Add dedicated interactive Celery worker for user-triggered cache rebuilds - #210
Merged
dannyvfilms merged 1 commit intoMay 21, 2026
Merged
Conversation
…ilds Previously all tasks ran on a single Celery worker with concurrency 1, so user-triggered statistics and history cache refreshes would queue behind long-running background tasks (metadata backfills, nightly reconciles), causing the UI to spin indefinitely waiting for a refresh that could be delayed by minutes. This adds a second `celery-interactive` worker that listens exclusively on the `interactive` queue, and routes `refresh_statistics_cache_task` and `refresh_history_cache_task` to that queue so they are always processed immediately regardless of background task load. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
nakenyon
marked this pull request as ready for review
May 19, 2026 11:11
dannyvfilms
pushed a commit
that referenced
this pull request
Aug 10, 2026
…ntext per test Port the browser-test fix from PR #617 (fix/ci-baseline-regressions) onto current latest, which already has #617's TVDB test-settings fix merged via #626 — only the browser-stabilization hunk was still needed. This diff is Ryan Winkler's from #617, applied verbatim to a rebased base. test_season_progress_edit (App Tests run #210) failed because the Playwright suite shared one browser page across the whole test class while each test flushes the SQLite test DB, and drove a live, unmocked TMDB search that could be slow/rate-limited. Give each test its own browser context (closed before Django's flush) and mock the TMDB calls the search-and-track flow makes, reusing the existing _tv_with_seasons_payload fixture helper from test_track_modal.py. Co-Authored-By: Ryan Winkler <28455297+ryan-winkler@users.noreply.github.com> Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
dannyvfilms
pushed a commit
that referenced
this pull request
Aug 11, 2026
The 15s expect() timeout bump didn't fix this run's failure: test_season_progress_edit failed instantly on the very first search step, showing the exact symptom from the original bug report (App Tests run #210) - the local-only "Nothing saved matches" suggestions panel instead of the /search results page. Root cause: the search form's Alpine.js handleFormSubmit() only reliably allows submission when it can identify the submit event's source unambiguously (submitter === the search button, or no submitter with the input focused). The search input also has hx-trigger="keyup changed delay:250ms, search" - pressing Enter in an <input type="search"> fires a native `search` DOM event on top of the keydown/keyup sequence, so Playwright's synthesized Enter key firing that event alongside form submission is exactly the kind of ambiguous, near-simultaneous native-event race handleFormSubmit's submitter check is fragile against. Click the actual search button instead: unambiguous submitter, no implicit-submission semantics to race. Verified locally end-to-end (with the episode-level TMDB call also mocked, matching what real CI's live network resolves) - full IntegrationTest class passes.
dannyvfilms
added a commit
that referenced
this pull request
Aug 11, 2026
Fixes the App Tests run #210 failure: the global-search-by-Enter flow racing the search form's Alpine.js submit guard against a native `search` DOM event, falling back to the local-only "Nothing saved matches" state instead of /search. Root-caused past what #617 (Ryan Winkler) started, including shared-mutable-mock corruption, missing test-fixture fields, an anonymous-user crash in media_search, and the actual Enter-key submission race. Also rolls in @tag("network") fixes for ~30 tests across several files that hit real, unmocked provider APIs, found across multiple review passes. Co-authored-by: Ryan Winkler <28455297+ryan-winkler@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
All tasks share a single Celery worker (
concurrency=1), so user-triggered cache refreshes — Statistics page ranges, History index rebuilds — queue behind long-running background tasks like nightly metadata backfills and reconcile jobs. The result is the UI spinning for minutes showing "Refreshing statistics in background…" even though the user is actively waiting.Solution
Add a second
celery-interactivesupervisord program that runs a dedicated Celery worker listening exclusively on theinteractivequeue. Route the two user-triggered tasks to that queue viaCELERY_TASK_ROUTES:app.tasks.refresh_statistics_cache_taskapp.tasks.refresh_history_cache_taskThe existing
celeryworker is pinned to theceleryqueue (explicit--queues celery) so background tasks continue unaffected. The interactive worker picks up cache rebuild tasks immediately regardless of background load.Changes
supervisord.conf— pin existing worker to--queues celery, add[program:celery-interactive]on--queues interactivesrc/config/settings.py— add routes for the two interactive tasks inCELERY_TASK_ROUTESTest plan
celery-interactiveworker appears in container logs on startupceleryworker🤖 Generated with Claude Code