adding docstrings, restructured reST

README.md

@@ -6,6 +6,8 @@
 Set of utility tools for Spotify and Last.fm. 
 Built on my other libraries for Spotify ([spotframework](https://github.com/Sarsoo/spotframework)), Last.fm ([fmframework](https://github.com/Sarsoo/pyfmframework)) and interfacing utility tools for the two ([spotfm](https://github.com/Sarsoo/pyfmframework)). Currently running on a suite of Google Cloud Platform services. An iOS client is currently under development [here](https://github.com/Sarsoo/Music-Tools-iOS).
 
+Read the full documentation [here](https://sarsoo.github.io/Music-Tools/).
+
 # Smart Playlists
 
 Create smart playlists for Spotify including tracks from playlists, library and Spotify recommendations.

docs/index.rst

@@ -14,12 +14,16 @@ Music Tools
    src/music.model
    src/music.tasks
 
-Music Tools
+`Music Tools <https://music.sarsoo.xyz>`_
--------------
+----------------------------------------------
 
 .. image:: https://github.com/sarsoo/music-tools/workflows/test%20and%20deploy/badge.svg
 
-Music Tools is a web app for creating smart Spotify playlists.
+Music Tools is a web app for creating smart Spotify playlists. The app is based on `spotframework <https://github.com/Sarsoo/spotframework>`_ and `fmframework <https://github.com/Sarsoo/pyfmframework>`_ for interfacing with Spotify and Last.fm. The app is currently hosted on Google's Cloud Platform.
+
+The system is composed of a Flask web server with a Fireo ORM layer and longer tasks dispatched to Cloud Tasks or Functions.
+
+.. image:: Playlists.png
 
 
 Indices and tables

docs/src/music.api.rst

@@ -1,73 +1,6 @@
-music.api package
+music.api
 =================
 
-Submodules
-----------
-
-music.api.admin module
-----------------------
-
-.. automodule:: music.api.admin
-   :members:
-   :undoc-members:
-   :show-inheritance:
-
-music.api.api module
---------------------
-
-.. automodule:: music.api.api
-   :members:
-   :undoc-members:
-   :show-inheritance:
-
-music.api.decorators module
----------------------------
-
-.. automodule:: music.api.decorators
-   :members:
-   :undoc-members:
-   :show-inheritance:
-
-music.api.fm module
--------------------
-
-.. automodule:: music.api.fm
-   :members:
-   :undoc-members:
-   :show-inheritance:
-
-music.api.player module
------------------------
-
-.. automodule:: music.api.player
-   :members:
-   :undoc-members:
-   :show-inheritance:
-
-music.api.spotfm module
------------------------
-
-.. automodule:: music.api.spotfm
-   :members:
-   :undoc-members:
-   :show-inheritance:
-
-music.api.spotify module
-------------------------
-
-.. automodule:: music.api.spotify
-   :members:
-   :undoc-members:
-   :show-inheritance:
-
-music.api.tag module
---------------------
-
-.. automodule:: music.api.tag
-   :members:
-   :undoc-members:
-   :show-inheritance:
-
 Module contents
 ---------------
 
@@ -75,3 +8,67 @@ Module contents
    :members:
    :undoc-members:
    :show-inheritance:
+
+api.admin
+----------------------
+
+.. automodule:: music.api.admin
+   :members:
+   :undoc-members:
+   :show-inheritance:
+
+api.api
+--------------------
+
+.. automodule:: music.api.api
+   :members:
+   :undoc-members:
+   :show-inheritance:
+
+api.decorators
+---------------------------
+
+.. automodule:: music.api.decorators
+   :members:
+   :undoc-members:
+   :show-inheritance:
+
+api.fm
+-------------------
+
+.. automodule:: music.api.fm
+   :members:
+   :undoc-members:
+   :show-inheritance:
+
+api.player
+-----------------------
+
+.. automodule:: music.api.player
+   :members:
+   :undoc-members:
+   :show-inheritance:
+
+api.spotfm
+-----------------------
+
+.. automodule:: music.api.spotfm
+   :members:
+   :undoc-members:
+   :show-inheritance:
+
+api.spotify
+------------------------
+
+.. automodule:: music.api.spotify
+   :members:
+   :undoc-members:
+   :show-inheritance:
+
+api.tag
+--------------------
+
+.. automodule:: music.api.tag
+   :members:
+   :undoc-members:
+   :show-inheritance:

docs/src/music.auth.rst

@@ -1,17 +1,6 @@
-music.auth package
+music.auth
 ==================
 
-Submodules
-----------
-
-music.auth.auth module
-----------------------
-
-.. automodule:: music.auth.auth
-   :members:
-   :undoc-members:
-   :show-inheritance:
-
 Module contents
 ---------------
 
@@ -19,3 +8,11 @@ Module contents
    :members:
    :undoc-members:
    :show-inheritance:
+
+auth.auth
+----------------------
+
+.. automodule:: music.auth.auth
+   :members:
+   :undoc-members:
+   :show-inheritance:

docs/src/music.cloud.rst

@@ -1,25 +1,6 @@
-music.cloud package
+music.cloud
 ===================
 
-Submodules
-----------
-
-music.cloud.function module
----------------------------
-
-.. automodule:: music.cloud.function
-   :members:
-   :undoc-members:
-   :show-inheritance:
-
-music.cloud.tasks module
-------------------------
-
-.. automodule:: music.cloud.tasks
-   :members:
-   :undoc-members:
-   :show-inheritance:
-
 Module contents
 ---------------
 
@@ -27,3 +8,19 @@ Module contents
    :members:
    :undoc-members:
    :show-inheritance:
+
+cloud.function
+---------------------------
+
+.. automodule:: music.cloud.function
+   :members:
+   :undoc-members:
+   :show-inheritance:
+
+cloud.tasks
+------------------------
+
+.. automodule:: music.cloud.tasks
+   :members:
+   :undoc-members:
+   :show-inheritance:

docs/src/music.db.rst

@@ -1,8 +1,13 @@
-music.db package
+music.db
 ================
 
-Submodules
+Module contents
-----------
+---------------
+
+.. automodule:: music.db
+   :members:
+   :undoc-members:
+   :show-inheritance:
 
 music.db.database module
 ------------------------
@@ -19,11 +24,3 @@ music.db.part\_generator module
    :members:
    :undoc-members:
    :show-inheritance:
-
-Module contents
----------------
-
-.. automodule:: music.db
-   :members:
-   :undoc-members:
-   :show-inheritance:

docs/src/music.model.rst

@@ -1,41 +1,6 @@
-music.model package
+music.model
 ===================
 
-Submodules
-----------
-
-music.model.config module
--------------------------
-
-.. automodule:: music.model.config
-   :members:
-   :undoc-members:
-   :show-inheritance:
-
-music.model.playlist module
----------------------------
-
-.. automodule:: music.model.playlist
-   :members:
-   :undoc-members:
-   :show-inheritance:
-
-music.model.tag module
-----------------------
-
-.. automodule:: music.model.tag
-   :members:
-   :undoc-members:
-   :show-inheritance:
-
-music.model.user module
------------------------
-
-.. automodule:: music.model.user
-   :members:
-   :undoc-members:
-   :show-inheritance:
-
 Module contents
 ---------------
 
@@ -43,3 +8,35 @@ Module contents
    :members:
    :undoc-members:
    :show-inheritance:
+
+model.config
+-------------------------
+
+.. automodule:: music.model.config
+   :members:
+   :undoc-members:
+   :show-inheritance:
+
+model.playlist
+---------------------------
+
+.. automodule:: music.model.playlist
+   :members:
+   :undoc-members:
+   :show-inheritance:
+
+model.tag
+----------------------
+
+.. automodule:: music.model.tag
+   :members:
+   :undoc-members:
+   :show-inheritance:
+
+model.user
+-----------------------
+
+.. automodule:: music.model.user
+   :members:
+   :undoc-members:
+   :show-inheritance:

docs/src/music.rst

@@ -1,4 +1,4 @@
-music package
+music
 =============
 
 Subpackages
@@ -14,8 +14,13 @@ Subpackages
    music.model
    music.tasks
 
-Submodules
+Module contents
-----------
+---------------
+
+.. automodule:: music
+   :members:
+   :undoc-members:
+   :show-inheritance:
 
 music.music module
 ------------------
@@ -24,11 +29,3 @@ music.music module
    :members:
    :undoc-members:
    :show-inheritance:
-
-Module contents
----------------
-
-.. automodule:: music
-   :members:
-   :undoc-members:
-   :show-inheritance:

docs/src/music.tasks.rst

@@ -1,41 +1,6 @@
-music.tasks package
+music.tasks
 ===================
 
-Submodules
-----------
-
-music.tasks.create\_playlist module
------------------------------------
-
-.. automodule:: music.tasks.create_playlist
-   :members:
-   :undoc-members:
-   :show-inheritance:
-
-music.tasks.refresh\_lastfm\_stats module
------------------------------------------
-
-.. automodule:: music.tasks.refresh_lastfm_stats
-   :members:
-   :undoc-members:
-   :show-inheritance:
-
-music.tasks.run\_user\_playlist module
---------------------------------------
-
-.. automodule:: music.tasks.run_user_playlist
-   :members:
-   :undoc-members:
-   :show-inheritance:
-
-music.tasks.update\_tag module
-------------------------------
-
-.. automodule:: music.tasks.update_tag
-   :members:
-   :undoc-members:
-   :show-inheritance:
-
 Module contents
 ---------------
 
@@ -43,3 +8,35 @@ Module contents
    :members:
    :undoc-members:
    :show-inheritance:
+
+tasks.create\_playlist
+-----------------------------------
+
+.. automodule:: music.tasks.create_playlist
+   :members:
+   :undoc-members:
+   :show-inheritance:
+
+tasks.refresh\_lastfm\_stats
+-----------------------------------------
+
+.. automodule:: music.tasks.refresh_lastfm_stats
+   :members:
+   :undoc-members:
+   :show-inheritance:
+
+tasks.run\_user\_playlist
+--------------------------------------
+
+.. automodule:: music.tasks.run_user_playlist
+   :members:
+   :undoc-members:
+   :show-inheritance:
+
+tasks.update\_tag
+------------------------------
+
+.. automodule:: music.tasks.update_tag
+   :members:
+   :undoc-members:
+   :show-inheritance:

music/cloud/function.py

@@ -5,8 +5,14 @@ publisher = pubsub_v1.PublisherClient()
 logger = logging.getLogger(__name__)
 
 
-def update_tag(username, tag_id):
+def update_tag(username: str, tag_id: str) -> None:
-    """Queue serverless tag update for user"""
+    """Queue serverless tag update for user
+
+    Args:
+        username (str): Subject username
+        tag_id (str): Subject tag ID
+    """
+
     logger.info(f'queuing {tag_id} update for {username}')
 
     if username is None or tag_id is None:
@@ -20,8 +26,14 @@ def update_tag(username, tag_id):
     publisher.publish('projects/sarsooxyz/topics/update_tag', b'', tag_id=tag_id, username=username)
 
 
-def run_user_playlist_function(username, playlist_name):
+def run_user_playlist_function(username: str, playlist_name: str) -> None:
-    """Queue serverless playlist update for user"""
+    """Queue serverless playlist update for user
+
+    Args:
+        username (str): Subject username
+        playlist_name (str): Subject tag ID
+    """
+
     logger.info(f'queuing {playlist_name} update for {username}')
 
     if username is None or playlist_name is None:

music/cloud/tasks.py

@@ -1,3 +1,6 @@
+"""Functions for creating GCP Cloud Tasks for long running operatings
+"""
+
 import datetime
 import json
 import os
@@ -48,8 +51,12 @@ def update_all_user_playlists():
             seconds_delay += 30
 
 
-def update_playlists(username):
+def update_playlists(username: str):
-    """Refresh all playlists for given user, environment dependent"""
+    """Refresh all playlists for given user, environment dependent
+
+    Args:
+        username (str): Subject user's username
+    """
 
     user = User.collection.filter('username', '==', username.strip().lower()).get()
 
@@ -73,8 +80,14 @@ def update_playlists(username):
             seconds_delay += 6
 
 
-def run_user_playlist_task(username, playlist_name, delay=0):
+def run_user_playlist_task(username: str, playlist_name: str, delay: int = 0):
-    """Create tasks for a users given playlist"""
+    """Create tasks for a users given playlist
+
+    Args:
+        username (str): Subject user's username
+        playlist_name (str): Subject playlist name
+        delay (int, optional): Seconds to delay execution by. Defaults to 0.
+    """
 
     task = {
         'app_engine_http_request': {  # Specify the type of request.
@@ -123,8 +136,12 @@ def refresh_all_user_playlist_stats():
             logger.debug(f'skipping {iter_user.username}')
 
 
-def refresh_user_playlist_stats(username):
+def refresh_user_playlist_stats(username: str):
-    """Refresh all playlist stats for given user, environment dependent"""
+    """Refresh all playlist stats for given user, environment dependent
+
+    Args:
+        username (str): Subject user's username
+    """
 
     user = User.collection.filter('username', '==', username.strip().lower()).get()
     if user is None:
@@ -150,8 +167,13 @@ def refresh_user_playlist_stats(username):
         logger.error('no last.fm username')
 
 
-def refresh_user_stats_task(username, delay=0):
+def refresh_user_stats_task(username: str, delay: int = 0):
-    """Create user playlist stats refresh task"""
+    """Create user playlist stats refresh task
+
+    Args:
+        username (str): Subject user's username
+        delay (int, optional): Seconds to delay execution by. Defaults to 0.
+    """
 
     task = {
         'app_engine_http_request': {  # Specify the type of request.
@@ -172,8 +194,14 @@ def refresh_user_stats_task(username, delay=0):
     tasker.create_task(task_path, task)
 
 
-def refresh_playlist_task(username, playlist_name, delay=0):
+def refresh_playlist_task(username: str, playlist_name: str, delay: int = 0):
-    """Create user playlist stats refresh tasks"""
+    """Create user playlist stats refresh tasks
+
+    Args:
+        username (str): Subject user's username
+        playlist_name (str): Subject playlist name
+        delay (int, optional): Seconds to delay execution by. Defaults to 0.
+    """
 
     track_task = {
         'app_engine_http_request': {  # Specify the type of request.
@@ -230,7 +258,7 @@ def refresh_playlist_task(username, playlist_name, delay=0):
 
 
 def update_all_user_tags():
-    """Create user tag refresh task sfor all users"""
+    """Create user tag refresh task for all users"""
 
     seconds_delay = 0
     logger.info('running')

music/db/database.py

@@ -1,6 +1,7 @@
 from dataclasses import dataclass
 import logging
 from datetime import timedelta, datetime, timezone
+from typing import Optional
 
 from spotframework.net.network import Network as SpotifyNetwork, SpotifyNetworkException
 from spotframework.net.user import NetworkUser
@@ -11,7 +12,15 @@ from music.model.config import Config
 logger = logging.getLogger(__name__)
 
 
-def refresh_token_database_callback(user):
+def refresh_token_database_callback(user: User) -> None:
+    """Callback for handling when a spotframework network updates user credemtials
+
+    Used to store newly authenticated credentials
+
+    Args:
+        user (User): Subject user
+    """
+
     if isinstance(user, DatabaseUser):
         user_obj = User.collection.filter('username', '==', user.user_id.strip().lower()).get()
         if user_obj is None:
@@ -29,7 +38,16 @@ def refresh_token_database_callback(user):
         logger.error('user has no attached id')
 
 
-def get_authed_spotify_network(user):
+def get_authed_spotify_network(user: User) -> Optional[SpotifyNetwork]:
+    """Get an authenticated spotframework network for a given user
+
+    Args:
+        user (User): Subject user to retrieve a network for
+
+    Returns:
+        Optional[SpotifyNetwork]: Authenticated spotframework network
+    """
+
     if user is not None:
         if user.spotify_linked:
             config = Config.collection.get("config/music-tools")
@@ -62,7 +80,16 @@ def get_authed_spotify_network(user):
         logger.error(f'no user provided')
 
 
-def get_authed_lastfm_network(user):
+def get_authed_lastfm_network(user: User) -> Optional[FmNetwork]:
+    """Get an authenticated fmframework network for a given user
+
+    Args:
+        user (User): Subject user to retrieve a network for
+
+    Returns:
+        Optional[FmNetwork]: Authenticated fmframework network
+    """
+
     if user is not None:
         if user.lastfm_username:
             config = Config.collection.get("config/music-tools")
@@ -75,5 +102,5 @@ def get_authed_lastfm_network(user):
 
 @dataclass
 class DatabaseUser(NetworkUser):
-    """adding music tools username to spotframework network user"""
+    """Adding Music Tools username to spotframework network user"""
     user_id: str = None

music/db/part_generator.py

@@ -1,13 +1,27 @@
 from music.model.user import User
 from music.model.playlist import Playlist
 import logging
+from typing import List
+from google.cloud.firestore import DocumentReference
 
 logger = logging.getLogger(__name__)
 
 
 class PartGenerator:
+    """Resolve a playlists components from other referenced smart playlists
+    """
 
     def __init__(self, user: User = None, username: str = None):
+        """Initialise with user to resolve for
+
+        Args:
+            user (User, optional): Subject user. Defaults to None.
+            username (str, optional): Subject username. Defaults to None.
+
+        Raises:
+            LookupError: No user returned when querying for username
+            NameError: No user provided
+        """
         self.queried_playlists = []
         self.parts = []
 
@@ -23,18 +37,35 @@ class PartGenerator:
             raise NameError('no user info provided')
 
     def reset(self):
+        """Reset internal state for resolved playlists
+        """
+
         self.queried_playlists = []
         self.parts = []
 
-    def get_recursive_parts(self, name):
+    def get_recursive_parts(self, name: str) -> List[str]:
+        """Resolve and return a playlist's component Spotify playlist names
+
+        Args:
+            name (str): Subject smart playlist name
+
+        Returns:
+            List[str]: Resolved list of component playlists
+        """
+
         logger.info(f'getting part from {name} for {self.user.username}')
 
         self.reset()
         self.process_reference_by_name(name)
 
-        return [i for i in {i for i in self.parts}]
+        return list({i for i in self.parts})
 
-    def process_reference_by_name(self, name):
+    def process_reference_by_name(self, name: str) -> None:
+        """Resolve a smart playlist by name, recurses into process_reference_by_reference
+
+        Args:
+            name (str): Subject playlist name
+        """
 
         playlist = Playlist.collection.parent(self.user.key).filter('name', '==', name).get()
 
@@ -55,7 +86,12 @@ class PartGenerator:
         else:
             logger.warning(f'playlist reference {name} not found')
 
-    def process_reference_by_reference(self, ref):
+    def process_reference_by_reference(self, ref: DocumentReference):
+        """Recursive resolution function for walking a playlist's dependencies by DocumentReference
+
+        Args:
+            ref (DocumentReference): Subject Firestore document for resolving
+        """
 
         if ref.id not in self.queried_playlists:
             playlist_reference_object = ref.get().to_dict()

music/model/config.py

@@ -3,12 +3,19 @@ from fireo.fields import TextField, BooleanField, DateTime, NumberField, ListFie
 
 
 class Config(Model):
+    """Service-level config data structure for app keys and settings
+    """
+
     class Meta:
         collection_name = 'config'
+        """Set correct path in Firestore
+        """
 
     spotify_client_id = TextField()
     spotify_client_secret = TextField()
     last_fm_client_id = TextField()
 
     playlist_cloud_operating_mode = TextField()  # task, function
+    """Determines whether playlist and tag update operations are done by Cloud Tasks or Functions
+    """
     secret_key = TextField()

music/tasks/create_playlist.py

@@ -1,12 +1,27 @@
 import logging
+from typing import Optional
 
 import music.db.database as database
 from spotframework.net.network import SpotifyNetworkException
+from spotframework.model.playlist import FullPlaylist
+
+from music.model.user import User
 
 logger = logging.getLogger(__name__)
 
 
-def create_playlist(user, name):
+def create_playlist(user: User, name: str) -> Optional[FullPlaylist]:
+    """Create a new playlist on the user's Spotify account
+
+    For creating new playlists, create and return a new playlist object
+
+    Args:
+        user (User): Subject user
+        name (str): Name of new playlist
+
+    Returns:
+        Optional[FullPlaylist]: New playlist object if created
+    """
 
     if user is None:
         logger.error(f'username not provided')

music/tasks/run_user_playlist.py

@@ -10,6 +10,7 @@ from spotframework.filter.sort import sort_by_release_date
 from spotframework.filter.deduplicate import deduplicate_by_name
 from spotframework.net.network import SpotifyNetworkException
 
+from spotframework.net.network import Network as SpotNetwork
 from fmframework.net.network import Network
 from spotfm.chart import map_lastfm_track_chart_to_spotify
 
@@ -21,8 +22,25 @@ from music.model.playlist import Playlist
 logger = logging.getLogger(__name__)
 
 
-def run_user_playlist(user, playlist, spotnet=None, fmnet=None):
+def run_user_playlist(user: User, playlist: Playlist, spotnet: SpotNetwork = None, fmnet: Network = None) -> None:
-    """Generate and upadate a user's playlist"""
+    """Generate and upadate a user's smart playlist
+
+    Args:
+        user (User): Subject user
+        playlist (Playlist): User's subject playlist
+        spotnet (SpotNetwork, optional): Spotframework network for Spotify operations. Defaults to None.
+        fmnet (Network, optional): Fmframework network for Last.fm operations. Defaults to None.
+
+    Raises:
+        NameError: No user provided
+        NameError: No playlist provided
+        AttributeError: Playlist has no URI
+        NameError: No spotframework network available
+        e: spotframework error when retrieving user playlists
+
+    Returns:
+        [type]: [description]
+    """
 
     # PRE-RUN CHECKS