[DRAFT] feat: API#924
Conversation
|
@FuzzyGrim is it possible to create a branch |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## dev #924 +/- ##
=======================================
Coverage 82.83% 82.83%
=======================================
Files 69 69
Lines 6923 6923
=======================================
Hits 5735 5735
Misses 1188 1188 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
|
Hi @66Bunz, Thanks for working on this! The spec file looks good to me. I've created the Also, I'd prefer a separate |
|
Thanks! |
…ion and pagination - Added Bearer and API Key authentication classes for secure access - Created serializers for media and item details - Started implementing API endpoints copying `app` implementations (could be useful to change some implementations in the future)
|
I've got an open questions: Looking at this list, is it ok to have the The history_id is not bound to the specific media |
- Introduced new `EpisodeSerializer` for handling episode-specific data - Updated `MediaSerializer` to include `parent_id` for nested relationships (tv serie -> season -> episode) - Updated `helpers.py` to start to support episodes - Added `history_processor.py` for managing history entries and processing timeline data - Continued implementation of routes - Modified URL patterns to accommodate new endpoints for media recommendations and history details - Improved pagination and serialization logic for media and history responses
|
Another option would be |
|
@FuzzyGrim I've got a question: Is it possible to get the media_id that is related to the history entry? I looked around but I wasn't able to find anything useful... |
…update pagination and schemas
|
Something like this: |
@FuzzyGrim 1 thing I noticed is that if the history entry is of a specific season/episode, I can only get the serie out of it, I wasn't able to get the specific season/episode. Am I missing something or this is it? |
|
It should work for both seasons and episodes. The |
|
Yes, nevermind, I was doing it wrong |
…; update serializers and helpers
…ructure and recommendations
There was a problem hiding this comment.
Pull request overview
This PR introduces a new /api/v1/ Django REST Framework API layer on top of the existing Yamtrack app implementations, plus supporting model changes (notably list item addressing) and a new API test suite.
Changes:
- Add a DRF-powered REST API app (
src/api/*) with endpoints for media, lists, calendar, changes history, search, health/info, and statistics. - Extend lists to support per-list sequential
list_item_idwith backfill migration and updated toggle behavior. - Add manual-provider search support with limit/offset/user inputs and wire up CI to run API tests.
Reviewed changes
Copilot reviewed 32 out of 37 changed files in this pull request and generated 7 comments.
Show a summary per file
| File | Description |
|---|---|
src/lists/views.py |
Switch list toggle to delete/create CustomListItem (to support list-scoped IDs). |
src/lists/models.py |
Add list_item_id, sequential assignment, renumber-on-delete, and list lookup helpers. |
src/lists/migrations/0004_customlistitem_list_item_id_and_more.py |
Backfill list_item_id for existing list items and add uniqueness constraint. |
src/lists/tests/test_views.py |
Update view tests to create CustomListItem directly. |
src/lists/tests/test_models.py |
Add tests for sequential IDs, bulk create behavior, renumbering, and list search. |
src/config/urls.py |
Register api/v1/ URL include. |
src/config/settings.py |
Add DRF + API app, default auth/permissions, JSON renderer, and API JSON error middleware. |
src/app/providers/services.py |
Extend provider search() signature and route manual searches to a new handler. |
src/app/providers/manual.py |
Add manual tracked-item search with pagination + caching; adjust manual metadata handling. |
src/app/models.py |
Add helper methods for seasons/episodes and list-membership mapping. |
src/app/helpers.py |
Add MODEL_MAP used for manual provider search across media types. |
src/api/urls.py |
Define API endpoint routing (optional trailing slash support). |
src/api/* (serializers/helpers/middleware/auth) |
Implement API serialization, pagination/sorting/helpers, JSON error conversion, and auth. |
src/api/tests/* |
Add endpoint contract tests and auth matrix tests for the new API. |
requirements.txt |
Add djangorestframework. |
pyproject.toml |
Ruff: ignore additional TODO rules. |
.github/workflows/app-tests.yml |
Run API tests in CI. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| def get_next_list_item_id(self, custom_list_id): | ||
| """Return the next sequential id for an item within a custom list.""" | ||
| current_max = ( | ||
| self.filter(custom_list_id=custom_list_id) | ||
| .aggregate(max_id=Max("list_item_id")) | ||
| .get("max_id") | ||
| ) | ||
| return 0 if current_max is None else current_max + 1 |
There was a problem hiding this comment.
get_next_list_item_id() uses MAX(list_item_id)+1 without any locking. With concurrent inserts into the same list, two transactions can compute the same next id and then hit the (custom_list, list_item_id) unique constraint on save/bulk_create. Consider locking the list (e.g., select_for_update() on the parent CustomList) or retrying on IntegrityError so concurrent adds don’t sporadically fail.
…ross serializers, views, and tests
… media core tests
Adds a new self-hosted tracker for Yamtrack using the new REST API introduced in FuzzyGrim/Yamtrack#924. - Bearer-token authentication with user-supplied host URL + API token - Search via /api/v1/search/manga/ with manual-entry fallback so users can track items that don't appear in upstream providers - Add / update / refresh / delete media via /api/v1/media/manga/... - Status mapping: Planning, In progress, Paused, Completed, Dropped - 0.0-10.0 score in 0.1 increments - Extended TrackingLoginDialog with optional pwStringRes so the token field reads "API token" instead of "Password"
The previous integration sent `status` as a display string (e.g. "In progress") and parsed the detail response as if `status`/`progress`/`score` lived at the top level. The actual REST API (FuzzyGrim/Yamtrack#924) expects `status` as an integer 0-4, keeps user tracking data inside `consumptions[0]`, and uses `max_progress`/`tracked` at the top level. This caused 400 errors on every PATCH and kept `bind()` from ever POSTing new items (GET returned 200 with provider metadata for any media, which we treated as "already tracked"). - Send `status` as Int (via Yamtrack.statusToApi) so validate_body accepts it. - Parse `tracked`, `max_progress`, and nested consumptions in YTMediaItem. - In bind()/refresh(), trigger add-to-library only when `tracked` is false.
|
Really love the API! Makes a lot of things possible, e. g. mobile apps. Created a (yes, fully vibe-coded) iOS app here. Swift 6, iOS 26, Liquid Glass, decent test coverage. Wanted to have an easier way to add media "on-the-go" and tried I far I can get with codex. Looking forward to the official release of the API! |
API Implementation
Roadmap
The roadmap I have in mind for this project is this:
develop API based on current app implementations (this PR)
move all implementations from the app to the API making them more "agnostic" (not so view driven)
change app to use API backend manipulating its data
since API is available, more "plugins", "integrations", "providers" can be implemented between various services (Jellyfin, seer, Trakt, ecc ecc)
Fixes #488
Fixes #543
Fixes #582
Fixes #698
To-Dos
/api/v1/calendar//api/v1/calendar/update//api/v1/changes_history/{media_type}/{history_id}:/api/v1/lists//api/v1/lists/{list_id}//api/v1/lists/{list_id}/items//api/v1/lists/{list_id}/items/{item_id}//api/v1/media//api/v1/media/{media_type}//api/v1/media/{media_type}/{source}/{media_id}//api/v1/media/{media_type}/{source}/{media_id}/changes_history//api/v1/media/{media_type}/{source}/{media_id}/history//api/v1/media/{media_type}/{source}/{media_id}/history/{consumption_id}//api/v1/media/{media_type}/{source}/{media_id}/lists//api/v1/media/{media_type}/{source}/{media_id}/lists/{list_id}/api/v1/media/{media_type}/{source}/{media_id}/recommendations//api/v1/media/{media_type}/{source}/{media_id}/seasons//api/v1/media/{media_type}/{source}/{media_id}/sync//api/v1/media/{media_type}/{source}/{media_id}/{season}//api/v1/media/{media_type}/{source}/{media_id}/{season}/episodes//api/v1/media/{media_type}/{source}/{media_id}/{season}/changes_history//api/v1/media/{media_type}/{source}/{media_id}/{season}/history//api/v1/media/{media_type}/{source}/{media_id}/{season}/history/{consumption_id}//api/v1/media/{media_type}/{source}/{media_id}/{season}/lists//api/v1/media/{media_type}/{source}/{media_id}/{season}/lists/{list_id}/api/v1/media/{media_type}/{source}/{media_id}/{season}/sync//api/v1/media/{media_type}/{source}/{media_id}/{season}/{episode}//api/v1/media/{media_type}/{source}/{media_id}/{season}/{episode}/changes_history//api/v1/media/{media_type}/{source}/{media_id}/{season}/{episode}/history//api/v1/media/{media_type}/{source}/{media_id}/{season}/{episode}/history/{consumption_id}//api/v1/media/{media_type}/{source}/{media_id}/{season}/{episode}/lists//api/v1/media/{media_type}/{source}/{media_id}/{season}/{episode}/lists/{list_id}/api/v1/media/{media_type}/{source}/{media_id}/{season}/{episode}/sync//api/v1/search/{media_type}//api/v1/statistics//api/v1/health//api/v1/info/feat/add-apibranchdevbranchDisclaimer
This is my first time working with Django, so I'm not really familiar with all the best practices, I tried to follow the existing code style.
If someone has suggestions on how to improve something let me know!
Main changes
Implementations
Since all of the features of the api are already implemented for the WebApp, I just referenced basically all the existing methods from the
appviews copying the implementations, or referencing directly the methods used in theappimplementations.Basically nothing was duplicated, only the changes_history_processor, because there were some things that were implemented to be human-readable, but not as api-friendly.
In the future, if there will be the decision that the WebApp should use the API directly, all the implementations should be moved from the
appto theapi.Urls
For urls I used the prefix
/api/v1/just to be on the safe side in case of future updates of the api.I choose to use the re_path in the urls.py to allow the ending with or without a trailing slash (both
/api/v1/media/and/api/v1/mediaare accepted). This makes the paths a bit more complex to write, but I think it's more friendly for the end user./media endpoint
The /media endpoint by default excludes seasons and episodes from the list of tracked items, to declutter the results.
Specific media endpoints
The main change I did in structuring the API routes for media items is inverting the media type and the source, because I think it makes more sense to group by media type first. So instead of
/{source}/{media_type}/{media_id}/as in the WebApp, I used/{media_type}/{source}/{media_id}/.It's true that both the media type can have different sources and a source can have differrent media types, but mainly the media type is the primary categorization of tracked items (e.g., when listing tracked items, filtering, searching, etc.), so it makes more sense to have it first in the path.
I decided that for the media types to follow a parent-child relationship, so seasons and episodes are sub-resources of tv series, so
media_typein the request URL cannot beseasonorepisode, but onlytv,movieecc ecc. This was made to simplify the API structure and make it more intuitive to use. So to query seasons or episodes, the user has to specify the parent tv serie first (e.g.,/api/v1/media/tv/{source}/{media_id}/{season}/{episode}/).Handling multiple consumptions
A single consumption entry is identified by
/history/{consumption_id}/subpath, so to get/edit/delete a specific consumption entry, the user has to specify the consumption id.The consumption id is the id of the media in the database, so it's incremental between all medias of the same type, and it's not related to the media id of the tracked item, because a tracked item can have multiple consumption entries, so they need to be identified by a unique id.
When listing the consumption history of an item, all the consumption entries are returned with their consumption id, so the user can use that id to get/edit/delete a specific consumption entry.
Operations on specific media items
GET on /media/{media_type}/{source}/{media_id}/ returns the tracked item with all its data.
DELETE on /media/{media_type}/{source}/{media_id}/ deletes the tracked item completely, with all its consumptions.
PATCH on /media/{media_type}/{source}/{media_id}/ edits the last consumption of the tracked item.
All of these operations are available for single consumptions, to get/delete/edit a specific consumption entry, the user has to use the /history/{consumption_id}/ subpath.
All of the previous operations are available for seasons and episodes as well.
Changes history endpoints
A thing to note is that history entries of children can appear in the changes history of the parent item when querying the parent changes history endpoint. For example, when getting a changes history entry of a season by id, that same id is present when querying the changes history entry of the tv serie, but this means that the item id of the entry is not the correct one because it was built for the parent and not the child.
Lists
Since the current implementation of lists doesn't have a field to identify the element in the list (0 for the first element added to the list, 1 for the second one and so on), I added a new field
list_item_idto the CustomListItem model, which is the id of the item in the list, so it can be used to identify the item in the list and perform operations on it (e.g., delete a specific item from the list).I implemented also a logic to update the
list_item_idof the items when items are removed, to not have "gaps" in the numeration. This change could be leveraged in the future to implement custom sorting for lists (out of scope for this PR).Sorting
Sorting for media items was implemented only partially, because there already are some ways to sort items in the existing codebase functions, so I just referenced those functions.
For lists, I had to implement them from scratch, because there were no existing implementations. They are implemented in the helpers.py file.
Probably in the future it could be useful to re-implement sorting for media items too, to have more control over it. At that point probably a new sorting.py file could be created to handle all the sorting logic in one place.
Search
The search is only implemented for providers by type, so the user has to specify the media type in the url, and cannot search only between already tracked items.
Search should be done globally, returning tracked and not tracked elements, maybe adding a tracked boolean field in the element, it'll come in the future.
Also Search should support more filters in the future, and also not be binded to media type, but a global search with elements from all providers could be implemented.
Statistics
Right now the statistics endpoint returns exactly the same data as the WebApp statistics page, without any modification. This is not really the best thing as they should be computed specifically for the API, so they can be computed as the API implementer wants. I wasn't really sure what to include, so I just left it as is for now.
Episode sync
Since in the
appthe episode sync method was not implemented, I just referenced the season sync method for now. In the future, if there will be the need to have a specific episode sync implementation, it can be changed easily.Authentication
The API uses two authentication methods implemented in authentication.py:
Authorization: Bearer <token>- Standard OAuth-style bearer tokensX-API-Key: <token>- Custom header for simpler API key authBoth methods look up the user by the
tokenfield in the User model. The user's existing token (from the users app) is reused for API authentication. No separate API key generation was implemented yet.Note: Currently there's no way to generate or regenerate API tokens via the API itself - users must use the web interface.
Pagination
All list endpoints use a consistent pagination approach via
parse_limit_offsetandpaginate_datahelpers in helpers.py:limit(default 20, max 200) andoffset(default 0)paginationobject withtotal,limit,offset,next, andpreviousURLsnextandpreviousURLs preserve all original query params (filters, sort, search)Consideration: This is offset-based pagination which can have performance issues with big lists of items.
Error Handling
A custom middleware ApiJsonErrorMiddleware converts Django HTML error responses to JSON for all
/api/paths:{"detail": "message"}structureprocess_exceptionto catch unhandled exceptions and return JSON errorsThis is because Django by default returns HTML error pages, but APIs shouldn't return HTML data.
Serializers
I added custom serializers in serializers.py to format the API responses consistently, each resource (Media, History, Season, Episode, List, etc.) has its own serializer class, nested relationships (e.g., Media with Seasons and Episodes) are handled correctly.
Testing
Still missing
Known Issues & Considerations
Future Improvements
Field filtering: Allow clients to request only specific fields (
?fields=title,score)Batch operations: Add support for bulk operations (e.g., update multiple items in one request)
Webhooks: Event subscription for calendar updates, new releases
Rate limiting: Proper per-user rate limiting with headers (X-RateLimit-*)
Expand filters: More granular filtering (score range, date range, tags)
Sorting: More sort options and multi-field sorting directly on the db query
Provider integrations: Endpoints to import tracked media from external providers (#1069)
Improve media user fields: Allow setting notes, progress, status, score and dates to every media, because not every media supports it, see table below.
Custom lists ordering: Allow users to create a custom order to the lists.
Administration commands: Add administration commands
table
Implement admin commands to manage the server
Dependencies
Only
djangorestframeworkwas added as a dependency, no additional dependencies were needed.