fix(docker): always start the RQ scheduler

[XenuIsWatching]

Description

^{Explain the changes or enhancements you are proposing with this pull request.}

The RQ scheduler only starts when one of four ENABLE_SCHEDULED_* flags is true:

# only start the scheduler if enabled
if [[ ${ENABLE_SCHEDULED_RESCAN} == "true" || ${ENABLE_SCHEDULED_UPDATE_SWITCH_TITLEDB} == "true" \
   || ${ENABLE_SCHEDULED_UPDATE_LAUNCHBOX_METADATA} == "true" || ${ENABLE_SCHEDULED_CLEANUP_ORPHANED_RESOURCES} == "true" ]]; then
    watchdog_process_pid rq_scheduler
fi

Nothing else respects that condition, and the list has fallen behind in three separate ways:

• startup.py registers three cleanups unconditionally. cleanup_netplay (every 30 min), cleanup_upload_tmp (hourly) and cleanup_zip_cache (daily) are declared enabled=True with cron strings and init()ed regardless of any flag. On a default install none of them ever run.
• Three more flags gate periodic tasks but are absent from the condition: ENABLE_SCHEDULED_CONVERT_IMAGES_TO_WEBP, ENABLE_SCHEDULED_RETROACHIEVEMENTS_PROGRESS_SYNC and ENABLE_SYNC_PUSH_PULL. Enabling any one of those alone silently does nothing.
• The filesystem watcher defers its rescans through the scheduler (watcher.py:214, tasks_scheduler.enqueue_in(...)), but ENABLE_RESCAN_ON_FILESYSTEM_CHANGE is not in the list either.

entrypoint.sh starts the scheduler unconditionally and always has, so the two entry paths already disagree about this.

The user-visible failure

Reported in Discord on 5.1.0: clicking Scan is refused immediately and permanently with 🛑 Scan already in progress, ignoring request, with no scan running.

With the watcher on and none of the four listed tasks enabled, the watcher runs but the scheduler does not. A filesystem change leaves a delayed scan_platforms entry in the scheduler registry that nothing will ever execute, and #3974's concurrent scan guard counts it as a queued scan, so every manual scan is refused.

Only one entry is needed to cause this. The watcher deduplicates before scheduling (get_pending_scan_jobs() in watcher.py, which bails when a full rescan is already pending and skips per-platform slugs that already have one), so entries do not pile up. A single stuck one blocks indefinitely.

That matches the report: restarting the container does not help, because it is persisted Redis state rather than a running process; clearing redis-data does help; and the block returns once the next scan finishes, because that scan writes resources, the watcher fires, and a fresh entry is scheduled.

Reproduced on a live instance by creating one watcher-style entry:

queued scans BEFORE: []
created scheduler entry: a1f0a558-… | func: endpoints.sockets.scan.scan_platforms
queued scans AFTER : ['a1f0a558-…']
>>> GUARD WOULD BLOCK: True

The reporter's configuration, confirmed after the fact, is ENABLE_RESCAN_ON_FILESYSTEM_CHANGE, ENABLE_SCHEDULED_CONVERT_IMAGES_TO_WEBP and ENABLE_SCHEDULED_RETROACHIEVEMENTS_PROGRESS_SYNC all true. Three scheduler-dependent features enabled, none of which starts the scheduler.

The silent variant is visible on any default install. These are the unconditional cleanups on my own instance, all overdue, because no scheduler exists to run them:

cleanup_netplay             scheduled_for=2026-07-25 06:30:00
cleanup_upload_tmp          scheduled_for=2026-07-25 07:00:00
cleanup_zip_cache           scheduled_for=2026-07-30 04:00:00
cleanup_orphaned_resources  scheduled_for=2026-07-30 05:00:00

Changes

Start the scheduler unconditionally, matching entrypoint.sh. A hand-maintained boolean chain in bash is what drifted here, and since three periodic tasks are registered regardless of configuration, there is no case left for the condition to express.

No cleanup step is needed for instances already stuck: rq-scheduler enqueues past-due jobs as soon as it starts, so the orphaned entry is picked up, run, and gone.

Testing

• Traced on a live 5.1.0 instance: reproduced the block by creating a single watcher-style scheduler entry, and confirmed rqscheduler is absent from the process list with the default configuration.
• bash -n, shellcheck, shfmt 3.6.0 clean.

No new tests. The change is one line of the init script, which has no test harness in-repo, and the guard behaviour it feeds is already covered by TestScanConcurrency.

Note for #4019

The s6-overlay rewrite copies the same four-flag condition verbatim into docker/s6-overlay/s6-rc.d/romm-rq-scheduler/run and calls disable_service, so it currently reintroduces all of the above. Commented there. Whichever lands second should carry this across.

Checklist

^{Please check all that apply.}

• I've tested the changes locally
• I've updated relevant comments
• I've assigned reviewers for this PR
• I've added unit tests that cover the changes

AI assistance disclosure

Per CONTRIBUTING.md: this change was written with AI assistance (Claude Code). The root cause was traced from the Discord report and reproduced on my own instance; the AI wrote the patch, and I reviewed the result.

[greptile-apps]

Greptile Summary

This PR starts the production RQ scheduler unconditionally and changes scan concurrency discovery to ignore scheduler entries more than 15 minutes overdue.

Confidence Score: 4/5

This PR should not merge until overdue delayed scans remain protected against later scheduler recovery.

A temporary scheduler outage lasting beyond the fixed grace period makes a still-executable delayed scan invisible to both concurrency prevention and cancellation, allowing it to overlap a manual scan or run after a stop request.

Files Needing Attention: backend/endpoints/sockets/scan.py

Important Files Changed

Filename	Overview
backend/endpoints/sockets/scan.py	Adds timestamp-aware scheduler discovery, but assumes an overdue persisted entry cannot execute after scheduler recovery.
backend/tests/endpoints/sockets/test_scan.py	Adds naive/aware timestamp coverage and the intended stale-entry behavior, but does not cover scheduler recovery after the grace period.
docker/init_scripts/init	Starts the scheduler unconditionally so delayed and periodic jobs are consistently drained.

Prompt To Fix All With AI

### Issue 1
backend/endpoints/sockets/scan.py:153
**Overdue scans remain executable**

When the scheduler recovers after a delayed watcher scan is more than 15 minutes overdue, this filter hides the still-executable entry from both the concurrency guard and cancellation path, allowing it to overlap a manual scan or run after a stop request.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

_{Reviews (1): Last reviewed commit: "fix(docker): always start the RQ schedul..." | Re-trigger Greptile}

[XenuIsWatching]

Confirmation from the reporter in Discord. Their configuration is:

ENABLE_RESCAN_ON_FILESYSTEM_CHANGE=true
ENABLE_SCHEDULED_CONVERT_IMAGES_TO_WEBP=true
ENABLE_SCHEDULED_RETROACHIEVEMENTS_PROGRESS_SYNC=true

Three scheduler-dependent features enabled, and not one of them starts the scheduler. The condition on master checks only ENABLE_SCHEDULED_RESCAN, ENABLE_SCHEDULED_UPDATE_SWITCH_TITLEDB, ENABLE_SCHEDULED_UPDATE_LAUNCHBOX_METADATA and ENABLE_SCHEDULED_CLEANUP_ORPHANED_RESOURCES.

So the watcher runs, defers rescans through a scheduler that does not exist, and the entries accumulate until the scan guard refuses every manual scan. That is the reported symptom, reached through a configuration I had not specifically predicted, which I think makes the case for dropping the condition rather than extending it: any fixed list here is one more thing to keep in sync, and this one has already drifted three times.

Two further consequences on that same instance, both silent:

• RetroAchievements progress sync has never run. Nothing outside the scheduler triggers it.
• WebP conversion only half works. startup.py:_enqueue_convert_images_to_webp() puts a one-off backfill directly on the low priority queue, which the worker picks up, and the worker does run. So covers present at startup get converted and the feature looks healthy. The recurring cron never fires, so anything added between restarts has no .webp sibling until the next container start.

That second one is probably worth noting for its own sake: the backfill masking the missing cron is a good part of why this went unnoticed for so long.

Worth adding that the workaround is not "enable any scheduled task", which is the natural assumption and the one I gave them first. It has to be one of the four in the condition. On this instance, ENABLE_SCHEDULED_CLEANUP_ORPHANED_RESOURCES=true plus a restart starts the scheduler and drains the backlog.

[XenuIsWatching]

Good catch, and correct. _get_queued_scan_jobs() also feeds stop_scan_handler, so the grace filter hid overdue entries from cancellation as well as from the guard, and an overdue entry is not dead: rq-scheduler enqueues past-due jobs the moment it comes up. Hiding it meant a manual scan could overlap it, and Stop would not cancel it.

I have dropped that part entirely rather than patch around it. It was defence in depth against a scheduler that stalls anyway, and it bought that at the cost of weakening a guard that works. The PR is now just the init script change.

Nothing is needed in its place: once the scheduler always starts, a stuck entry resolves itself, because the scheduler enqueues past-due jobs at startup and the worker runs them.

Separately, I have corrected the description. I had written that watcher entries accumulate. They do not, get_pending_scan_jobs() in watcher.py deduplicates before scheduling, bailing when a full rescan is already pending and skipping per-platform slugs that already have one. It is a single stuck entry that blocks indefinitely, not a growing pile. Same symptom, wrong mechanism on my part.

[gantoine]

yes 100%

[Copilot]

Pull request overview

This PR fixes a Docker startup inconsistency by ensuring the RQ scheduler is always supervised and running in the legacy docker/init_scripts/init path, matching entrypoint.sh behavior. This prevents scheduler-backed features (watcher-delayed rescans and periodic cleanups) from silently never executing due to a drifted, incomplete flag-gated condition.

Changes:

• Start/supervise rq_scheduler unconditionally in the main watchdog loop.
• Remove the brittle multi-flag conditional that had fallen out of sync with actual scheduled-task usage.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.