sarsoo/Mixonomer
1
0
Fork 0
Code Issues 15 Pull Requests Actions Packages Projects Releases Wiki Activity

adding docstrings, restructured reST

Browse Source
This commit is contained in:
andy 2021-03-24 10:06:54 +00:00
parent 0a18597165
commit 7ade5ccaab
16 changed files with 351 additions and 223 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
README.md
View File

@ -6,6 +6,8 @@
Set of utility tools for Spotify and Last.fm.
Built on my other libraries for Spotify ([spotframework](https://github.com/Sarsoo/spotframework)), Last.fm ([fmframework](https://github.com/Sarsoo/pyfmframework)) and interfacing utility tools for the two ([spotfm](https://github.com/Sarsoo/pyfmframework)). Currently running on a suite of Google Cloud Platform services. An iOS client is currently under development [here](https://github.com/Sarsoo/Music-Tools-iOS).
Read the full documentation [here](https://sarsoo.github.io/Music-Tools/).
# Smart Playlists
Create smart playlists for Spotify including tracks from playlists, library and Spotify recommendations.

10
docs/index.rst
View File

@ -14,12 +14,16 @@ Music Tools
src/music.model
src/music.tasks
Music Tools
-------------
`Music Tools <https://music.sarsoo.xyz>`_
----------------------------------------------
.. image:: https://github.com/sarsoo/music-tools/workflows/test%20and%20deploy/badge.svg
Music Tools is a web app for creating smart Spotify playlists.
Music Tools is a web app for creating smart Spotify playlists. The app is based on `spotframework <https://github.com/Sarsoo/spotframework>`_ and `fmframework <https://github.com/Sarsoo/pyfmframework>`_ for interfacing with Spotify and Last.fm. The app is currently hosted on Google's Cloud Platform.
The system is composed of a Flask web server with a Fireo ORM layer and longer tasks dispatched to Cloud Tasks or Functions.
.. image:: Playlists.png
Indices and tables

133
docs/src/music.api.rst
View File

@ -1,73 +1,6 @@
music.api package
music.api
=================
Submodules
----------
music.api.admin module
----------------------
.. automodule:: music.api.admin
:members:
:undoc-members:
:show-inheritance:
music.api.api module
--------------------
.. automodule:: music.api.api
:members:
:undoc-members:
:show-inheritance:
music.api.decorators module
---------------------------
.. automodule:: music.api.decorators
:members:
:undoc-members:
:show-inheritance:
music.api.fm module
-------------------
.. automodule:: music.api.fm
:members:
:undoc-members:
:show-inheritance:
music.api.player module
-----------------------
.. automodule:: music.api.player
:members:
:undoc-members:
:show-inheritance:
music.api.spotfm module
-----------------------
.. automodule:: music.api.spotfm
:members:
:undoc-members:
:show-inheritance:
music.api.spotify module
------------------------
.. automodule:: music.api.spotify
:members:
:undoc-members:
:show-inheritance:
music.api.tag module
--------------------
.. automodule:: music.api.tag
:members:
:undoc-members:
:show-inheritance:
Module contents
---------------
@ -75,3 +8,67 @@ Module contents
:members:
:undoc-members:
:show-inheritance:
api.admin
----------------------
.. automodule:: music.api.admin
:members:
:undoc-members:
:show-inheritance:
api.api
--------------------
.. automodule:: music.api.api
:members:
:undoc-members:
:show-inheritance:
api.decorators
---------------------------
.. automodule:: music.api.decorators
:members:
:undoc-members:
:show-inheritance:
api.fm
-------------------
.. automodule:: music.api.fm
:members:
:undoc-members:
:show-inheritance:
api.player
-----------------------
.. automodule:: music.api.player
:members:
:undoc-members:
:show-inheritance:
api.spotfm
-----------------------
.. automodule:: music.api.spotfm
:members:
:undoc-members:
:show-inheritance:
api.spotify
------------------------
.. automodule:: music.api.spotify
:members:
:undoc-members:
:show-inheritance:
api.tag
--------------------
.. automodule:: music.api.tag
:members:
:undoc-members:
:show-inheritance:

21
docs/src/music.auth.rst
View File

@ -1,17 +1,6 @@
music.auth package
music.auth
==================
Submodules
----------
music.auth.auth module
----------------------
.. automodule:: music.auth.auth
:members:
:undoc-members:
:show-inheritance:
Module contents
---------------
@ -19,3 +8,11 @@ Module contents
:members:
:undoc-members:
:show-inheritance:
auth.auth
----------------------
.. automodule:: music.auth.auth
:members:
:undoc-members:
:show-inheritance:

37
docs/src/music.cloud.rst
View File

@ -1,25 +1,6 @@
music.cloud package
music.cloud
===================
Submodules
----------
music.cloud.function module
---------------------------
.. automodule:: music.cloud.function
:members:
:undoc-members:
:show-inheritance:
music.cloud.tasks module
------------------------
.. automodule:: music.cloud.tasks
:members:
:undoc-members:
:show-inheritance:
Module contents
---------------
@ -27,3 +8,19 @@ Module contents
:members:
:undoc-members:
:show-inheritance:
cloud.function
---------------------------
.. automodule:: music.cloud.function
:members:
:undoc-members:
:show-inheritance:
cloud.tasks
------------------------
.. automodule:: music.cloud.tasks
:members:
:undoc-members:
:show-inheritance:

19
docs/src/music.db.rst
View File

@ -1,8 +1,13 @@
music.db package
music.db
================
Submodules
----------
Module contents
---------------
.. automodule:: music.db
:members:
:undoc-members:
:show-inheritance:
music.db.database module
------------------------
@ -19,11 +24,3 @@ music.db.part\_generator module
:members:
:undoc-members:
:show-inheritance:
Module contents
---------------
.. automodule:: music.db
:members:
:undoc-members:
:show-inheritance:

69
docs/src/music.model.rst
View File

@ -1,41 +1,6 @@
music.model package
music.model
===================
Submodules
----------
music.model.config module
-------------------------
.. automodule:: music.model.config
:members:
:undoc-members:
:show-inheritance:
music.model.playlist module
---------------------------
.. automodule:: music.model.playlist
:members:
:undoc-members:
:show-inheritance:
music.model.tag module
----------------------
.. automodule:: music.model.tag
:members:
:undoc-members:
:show-inheritance:
music.model.user module
-----------------------
.. automodule:: music.model.user
:members:
:undoc-members:
:show-inheritance:
Module contents
---------------
@ -43,3 +8,35 @@ Module contents
:members:
:undoc-members:
:show-inheritance:
model.config
-------------------------
.. automodule:: music.model.config
:members:
:undoc-members:
:show-inheritance:
model.playlist
---------------------------
.. automodule:: music.model.playlist
:members:
:undoc-members:
:show-inheritance:
model.tag
----------------------
.. automodule:: music.model.tag
:members:
:undoc-members:
:show-inheritance:
model.user
-----------------------
.. automodule:: music.model.user
:members:
:undoc-members:
:show-inheritance:

19
docs/src/music.rst
View File

@ -1,4 +1,4 @@
music package
music
=============
Subpackages
@ -14,8 +14,13 @@ Subpackages
music.model
music.tasks
Submodules
----------
Module contents
---------------
.. automodule:: music
:members:
:undoc-members:
:show-inheritance:
music.music module
------------------
@ -24,11 +29,3 @@ music.music module
:members:
:undoc-members:
:show-inheritance:
Module contents
---------------
.. automodule:: music
:members:
:undoc-members:
:show-inheritance:

69
docs/src/music.tasks.rst
View File

@ -1,41 +1,6 @@
music.tasks package
music.tasks
===================
Submodules
----------
music.tasks.create\_playlist module
-----------------------------------
.. automodule:: music.tasks.create_playlist
:members:
:undoc-members:
:show-inheritance:
music.tasks.refresh\_lastfm\_stats module
-----------------------------------------
.. automodule:: music.tasks.refresh_lastfm_stats
:members:
:undoc-members:
:show-inheritance:
music.tasks.run\_user\_playlist module
--------------------------------------
.. automodule:: music.tasks.run_user_playlist
:members:
:undoc-members:
:show-inheritance:
music.tasks.update\_tag module
------------------------------
.. automodule:: music.tasks.update_tag
:members:
:undoc-members:
:show-inheritance:
Module contents
---------------
@ -43,3 +8,35 @@ Module contents
:members:
:undoc-members:
:show-inheritance:
tasks.create\_playlist
-----------------------------------
.. automodule:: music.tasks.create_playlist
:members:
:undoc-members:
:show-inheritance:
tasks.refresh\_lastfm\_stats
-----------------------------------------
.. automodule:: music.tasks.refresh_lastfm_stats
:members:
:undoc-members:
:show-inheritance:
tasks.run\_user\_playlist
--------------------------------------
.. automodule:: music.tasks.run_user_playlist
:members:
:undoc-members:
:show-inheritance:
tasks.update\_tag
------------------------------
.. automodule:: music.tasks.update_tag
:members:
:undoc-members:
:show-inheritance:

20
music/cloud/function.py
View File

@ -5,8 +5,14 @@ publisher = pubsub_v1.PublisherClient()
logger = logging.getLogger(__name__)
def update_tag(username, tag_id):
"""Queue serverless tag update for user"""
def update_tag(username: str, tag_id: str) -> None:
"""Queue serverless tag update for user
Args:
username (str): Subject username
tag_id (str): Subject tag ID
"""
logger.info(f'queuing {tag_id} update for {username}')
if username is None or tag_id is None:
@ -20,8 +26,14 @@ def update_tag(username, tag_id):
publisher.publish('projects/sarsooxyz/topics/update_tag', b'', tag_id=tag_id, username=username)
def run_user_playlist_function(username, playlist_name):
"""Queue serverless playlist update for user"""
def run_user_playlist_function(username: str, playlist_name: str) -> None:
"""Queue serverless playlist update for user
Args:
username (str): Subject username
playlist_name (str): Subject tag ID
"""
logger.info(f'queuing {playlist_name} update for {username}')
if username is None or playlist_name is None:

50
music/cloud/tasks.py
View File

@ -1,3 +1,6 @@
"""Functions for creating GCP Cloud Tasks for long running operatings
"""
import datetime
import json
import os
@ -48,8 +51,12 @@ def update_all_user_playlists():
seconds_delay += 30
def update_playlists(username):
"""Refresh all playlists for given user, environment dependent"""
def update_playlists(username: str):
"""Refresh all playlists for given user, environment dependent
Args:
username (str): Subject user's username
"""
user = User.collection.filter('username', '==', username.strip().lower()).get()
@ -73,8 +80,14 @@ def update_playlists(username):
seconds_delay += 6
def run_user_playlist_task(username, playlist_name, delay=0):
"""Create tasks for a users given playlist"""
def run_user_playlist_task(username: str, playlist_name: str, delay: int = 0):
"""Create tasks for a users given playlist
Args:
username (str): Subject user's username
playlist_name (str): Subject playlist name
delay (int, optional): Seconds to delay execution by. Defaults to 0.
"""
task = {
'app_engine_http_request': { # Specify the type of request.
@ -123,8 +136,12 @@ def refresh_all_user_playlist_stats():
logger.debug(f'skipping {iter_user.username}')
def refresh_user_playlist_stats(username):
"""Refresh all playlist stats for given user, environment dependent"""
def refresh_user_playlist_stats(username: str):
"""Refresh all playlist stats for given user, environment dependent
Args:
username (str): Subject user's username
"""
user = User.collection.filter('username', '==', username.strip().lower()).get()
if user is None:
@ -150,8 +167,13 @@ def refresh_user_playlist_stats(username):
logger.error('no last.fm username')
def refresh_user_stats_task(username, delay=0):
"""Create user playlist stats refresh task"""
def refresh_user_stats_task(username: str, delay: int = 0):
"""Create user playlist stats refresh task
Args:
username (str): Subject user's username
delay (int, optional): Seconds to delay execution by. Defaults to 0.
"""
task = {
'app_engine_http_request': { # Specify the type of request.
@ -172,8 +194,14 @@ def refresh_user_stats_task(username, delay=0):
tasker.create_task(task_path, task)
def refresh_playlist_task(username, playlist_name, delay=0):
"""Create user playlist stats refresh tasks"""
def refresh_playlist_task(username: str, playlist_name: str, delay: int = 0):
"""Create user playlist stats refresh tasks
Args:
username (str): Subject user's username
playlist_name (str): Subject playlist name
delay (int, optional): Seconds to delay execution by. Defaults to 0.
"""
track_task = {
'app_engine_http_request': { # Specify the type of request.
@ -230,7 +258,7 @@ def refresh_playlist_task(username, playlist_name, delay=0):
def update_all_user_tags():
"""Create user tag refresh task sfor all users"""
"""Create user tag refresh task for all users"""
seconds_delay = 0
logger.info('running')

35
music/db/database.py
View File

@ -1,6 +1,7 @@
from dataclasses import dataclass
import logging
from datetime import timedelta, datetime, timezone
from typing import Optional
from spotframework.net.network import Network as SpotifyNetwork, SpotifyNetworkException
from spotframework.net.user import NetworkUser
@ -11,7 +12,15 @@ from music.model.config import Config
logger = logging.getLogger(__name__)
def refresh_token_database_callback(user):
def refresh_token_database_callback(user: User) -> None:
"""Callback for handling when a spotframework network updates user credemtials
Used to store newly authenticated credentials
Args:
user (User): Subject user
"""
if isinstance(user, DatabaseUser):
user_obj = User.collection.filter('username', '==', user.user_id.strip().lower()).get()
if user_obj is None:
@ -29,7 +38,16 @@ def refresh_token_database_callback(user):
logger.error('user has no attached id')
def get_authed_spotify_network(user):
def get_authed_spotify_network(user: User) -> Optional[SpotifyNetwork]:
"""Get an authenticated spotframework network for a given user
Args:
user (User): Subject user to retrieve a network for
Returns:
Optional[SpotifyNetwork]: Authenticated spotframework network
"""
if user is not None:
if user.spotify_linked:
config = Config.collection.get("config/music-tools")
@ -62,7 +80,16 @@ def get_authed_spotify_network(user):
logger.error(f'no user provided')
def get_authed_lastfm_network(user):
def get_authed_lastfm_network(user: User) -> Optional[FmNetwork]:
"""Get an authenticated fmframework network for a given user
Args:
user (User): Subject user to retrieve a network for
Returns:
Optional[FmNetwork]: Authenticated fmframework network
"""
if user is not None:
if user.lastfm_username:
config = Config.collection.get("config/music-tools")
@ -75,5 +102,5 @@ def get_authed_lastfm_network(user):
@dataclass
class DatabaseUser(NetworkUser):
"""adding music tools username to spotframework network user"""
"""Adding Music Tools username to spotframework network user"""
user_id: str = None

44
music/db/part_generator.py
View File

@ -1,13 +1,27 @@
from music.model.user import User
from music.model.playlist import Playlist
import logging
from typing import List
from google.cloud.firestore import DocumentReference
logger = logging.getLogger(__name__)
class PartGenerator:
"""Resolve a playlists components from other referenced smart playlists
"""
def __init__(self, user: User = None, username: str = None):
"""Initialise with user to resolve for
Args:
user (User, optional): Subject user. Defaults to None.
username (str, optional): Subject username. Defaults to None.
Raises:
LookupError: No user returned when querying for username
NameError: No user provided
"""
self.queried_playlists = []
self.parts = []
@ -23,18 +37,35 @@ class PartGenerator:
raise NameError('no user info provided')
def reset(self):
"""Reset internal state for resolved playlists
"""
self.queried_playlists = []
self.parts = []
def get_recursive_parts(self, name):
def get_recursive_parts(self, name: str) -> List[str]:
"""Resolve and return a playlist's component Spotify playlist names
Args:
name (str): Subject smart playlist name
Returns:
List[str]: Resolved list of component playlists
"""
logger.info(f'getting part from {name} for {self.user.username}')
self.reset()
self.process_reference_by_name(name)
return [i for i in {i for i in self.parts}]
return list({i for i in self.parts})
def process_reference_by_name(self, name):
def process_reference_by_name(self, name: str) -> None:
"""Resolve a smart playlist by name, recurses into process_reference_by_reference
Args:
name (str): Subject playlist name
"""
playlist = Playlist.collection.parent(self.user.key).filter('name', '==', name).get()
@ -55,7 +86,12 @@ class PartGenerator:
else:
logger.warning(f'playlist reference {name} not found')
def process_reference_by_reference(self, ref):
def process_reference_by_reference(self, ref: DocumentReference):
"""Recursive resolution function for walking a playlist's dependencies by DocumentReference
Args:
ref (DocumentReference): Subject Firestore document for resolving
"""
if ref.id not in self.queried_playlists:
playlist_reference_object = ref.get().to_dict()

7
music/model/config.py
View File

@ -3,12 +3,19 @@ from fireo.fields import TextField, BooleanField, DateTime, NumberField, ListFie
class Config(Model):
"""Service-level config data structure for app keys and settings
"""
class Meta:
collection_name = 'config'
"""Set correct path in Firestore
"""
spotify_client_id = TextField()
spotify_client_secret = TextField()
last_fm_client_id = TextField()
playlist_cloud_operating_mode = TextField() # task, function
"""Determines whether playlist and tag update operations are done by Cloud Tasks or Functions
"""
secret_key = TextField()

17
music/tasks/create_playlist.py
View File

@ -1,12 +1,27 @@
import logging
from typing import Optional
import music.db.database as database
from spotframework.net.network import SpotifyNetworkException
from spotframework.model.playlist import FullPlaylist
from music.model.user import User
logger = logging.getLogger(__name__)
def create_playlist(user, name):
def create_playlist(user: User, name: str) -> Optional[FullPlaylist]:
"""Create a new playlist on the user's Spotify account
For creating new playlists, create and return a new playlist object
Args:
user (User): Subject user
name (str): Name of new playlist
Returns:
Optional[FullPlaylist]: New playlist object if created
"""
if user is None:
logger.error(f'username not provided')

22
music/tasks/run_user_playlist.py
View File

@ -10,6 +10,7 @@ from spotframework.filter.sort import sort_by_release_date
from spotframework.filter.deduplicate import deduplicate_by_name
from spotframework.net.network import SpotifyNetworkException
from spotframework.net.network import Network as SpotNetwork
from fmframework.net.network import Network
from spotfm.chart import map_lastfm_track_chart_to_spotify
@ -21,8 +22,25 @@ from music.model.playlist import Playlist
logger = logging.getLogger(__name__)
def run_user_playlist(user, playlist, spotnet=None, fmnet=None):
"""Generate and upadate a user's playlist"""
def run_user_playlist(user: User, playlist: Playlist, spotnet: SpotNetwork = None, fmnet: Network = None) -> None:
"""Generate and upadate a user's smart playlist
Args:
user (User): Subject user
playlist (Playlist): User's subject playlist
spotnet (SpotNetwork, optional): Spotframework network for Spotify operations. Defaults to None.
fmnet (Network, optional): Fmframework network for Last.fm operations. Defaults to None.
Raises:
NameError: No user provided
NameError: No playlist provided
AttributeError: Playlist has no URI
NameError: No spotframework network available
e: spotframework error when retrieving user playlists
Returns:
[type]: [description]
"""
# PRE-RUN CHECKS