feat(local): watch the music folders and refresh automatically (#409) - #620
Draft
TheZupZup wants to merge 2 commits into
Draft
feat(local): watch the music folders and refresh automatically (#409)#620TheZupZup wants to merge 2 commits into
TheZupZup wants to merge 2 commits into
Conversation
Depends on #619 (incremental local scans). Branched from it rather than main, because a watcher that triggered a full rescan on every change would be worse than no watcher at all: this is only affordable because a refresh now re-reads only the files whose size or mtime moved. On Linux the selected music folders are watched recursively, and a change under one of them refreshes the library. Adding an album, deleting a track or renaming a folder shows up without pressing Rescan. The watcher owns no catalog logic on purpose. A filesystem change turns into scanFolders(...), the same call the Rescan button makes. There is exactly one way the local catalog is written, and a watcher with its own idea of how to apply a change would be a second one that could disagree with the first. Coalescing is the design: * an event starts, or extends, a quiet period, so a burst collapses into one scan. Copying a 12-track album is 37 events and one scan; * a burst that never goes quiet, which is what a copy over a slow network mount looks like, is not allowed to postpone the refresh forever: past a maximum window it refreshes anyway and starts a new one, so a long copy fills in while it runs rather than only at the end; * events during a refresh schedule exactly one more, not one per event. Noise is filtered before any of that. A music folder is full of things that change constantly while music is being added and cannot change the catalog: a downloader's .part files, cover art, .nfo/.log/.cue sidecars, editor swap files, and the bookkeeping folders sync tools scatter around. A path with an extension has to be one Linthra can import; a path without one is accepted, because that is what a directory looks like and a directory event is how a new or renamed album folder is noticed. Failure is expected rather than exceptional. inotify has a per-user watch limit, a folder can vanish, and many network filesystems cannot be watched at all. Any of those marks that one folder unwatched, leaves the others watching, and changes nothing about manual refresh. A watch that merely *ends* counts as a failure too: nothing under it will be reported again, and calling that "still watching" is a lie the user cannot see through. Re-syncing retries, because the reason it failed is usually temporary. Modules: * lib/core/sources/local/local_directory_watch.dart is the OS seam. The unsupported implementation is what Android gets: its local library is a SAF tree or a MediaStore query, not a directory, so there is no path to watch. * lib/core/sources/local/local_library_watcher.dart is the debounce, the relevance filter, the failure handling and the disposal. No Flutter, no I/O. * lib/features/library/local_library_watch_service.dart keeps the watched set in step with the selection and points a change at the incremental scan. Bootstrap reads it once, and the container's disposal releases the watches, so a shutdown gives its inotify descriptors back rather than leaving them to the process exit. Nothing here writes to the user's folders, so there is no feedback loop to guard against: Linthra never creates, moves or deletes a file under a music folder, and the only thing a refresh does to disk is read. Tests: 39 new, 4972 passing overall. The behaviour is exercised against a synthetic filesystem, because pinning Linthra's own rules to inotify's timing would make them flaky, and one small suite exercises the real `Directory.watch` seam against a temp directory the OS chooses, so a wrong assumption about the real thing cannot survive the whole suite. That suite skips itself where the host cannot watch, which is the same honest outcome the app has there. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Lq4ZJtwtdVCMBuZLjFbyfV
… into feat/409-linux-library-watcher
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #409. Part of #376.
Depends on #619 (incremental local scans). This branch is based on
perf/411-incremental-local-scan, notmain, and the PR targets that branch so the diff shows only the watcher. Once #619 merges, retarget this tomainand it needs no rebase.The dependency is real, not bookkeeping: a watcher that triggered a full rescan on every change would be worse than no watcher at all. This is only affordable because a refresh now re-reads just the files whose size or mtime moved.
What changes
On Linux the selected music folders are watched recursively, and a change under one of them refreshes the library. Adding an album, deleting a track or renaming a folder shows up without pressing Rescan.
The watcher owns no catalog logic, deliberately. A filesystem change turns into
scanFolders(...), the same call the Rescan button makes. There is exactly one way the local catalog is written, and a watcher with its own idea of how to apply a change would be a second one that could disagree with the first, which is exactly the kind of drift that produces "the library is wrong but a rescan fixes it" bugs.Coalescing
Copying one album produces a burst of events (a create and several modifies per file, plus directory events), so a scan per event would be hundreds of scans of the whole library for one user action.
Noise is filtered before any of that. A music folder is full of things that change constantly while music is being added and cannot change the catalog: a downloader's
.partfiles, cover art,.nfo/.log/.cuesidecars, editor swap files, and the bookkeeping folders sync tools scatter around. A path with an extension has to be one Linthra can import; a path without one is accepted, because that is what a directory looks like and a directory event is how a new or renamed album folder is noticed.Failure is expected, not exceptional
fs.inotify.max_user_watchescan exhaust it.Any of those marks that one folder unwatched, leaves the others watching, and changes nothing about manual refresh. A watch that merely ends counts as a failure too: nothing under it will be reported again, and calling that "still watching" is a lie the user cannot see through. Re-syncing retries a failed folder, because the reason it failed (a drive that was not mounted, a watch budget that was full) is usually temporary and rescanning is exactly when someone expects another attempt.
unwatchableRoots/isDegradedare exposed for a UI to read. Surfacing them in Settings is the obvious follow-up and is deliberately not in this PR: the acceptance criterion is that a watcher failure does not break manual refresh, which is covered, and a Settings row is a separate, reviewable change.Files
lib/core/sources/local/local_directory_watch.dartis the OS seam. The unsupported implementation is what Android gets: its local library is a Storage Access Framework tree or a MediaStore query, not a directory Linthra may walk, so there is no path to watch.lib/core/sources/local/local_library_watcher.dartis the debounce, the relevance filter, the failure handling and the disposal. No Flutter, no I/O.lib/features/library/local_library_watch_service.dartkeeps the watched set in step with the selection and points a change at the incremental scan. Bootstrap reads it once, and the container's disposal releases the watches, so a shutdown gives its inotify descriptors back rather than leaving them to the process exit.Protecting user data
Nothing here writes to the user's folders, so there is no feedback loop to guard against: Linthra never creates, moves or deletes a file under a music folder, and the only thing a refresh does to disk is read.
Tests
Synthetic filesystem (
local_library_watcher_test.dart, 28 cases): only selected folders are watched and a nested one is not watched twice; re-syncing opens nothing new; removing a folder releases it; add / delete / rename / folder-rename / re-tag each refresh once;.partfiles, cover art and sidecars refresh nothing; an album copy is one scan; a burst that never goes quiet still refreshes; 20 events during a scan produce one rescan; a refusing folder, a watch that errors, a watch that just ends, retry, and a refresh that throws; and disposal releasing every watch, cancelling a scheduled refresh, being idempotent, and ignoring later syncs.Wired into the app graph (
local_library_watch_service_test.dart): the selection drives what is watched, a new file lands in the catalog with nobody pressing Rescan, a deleted one leaves it, Android watches nothing, and disposing the container releases the watches.Real filesystem (
io_directory_watch_test.dart): theDirectory.watchseam against a temp directory the OS chooses, so a wrong assumption about the real thing cannot survive the whole suite. It skips itself where the host cannot watch, which is the same honest outcome the app has there. No distro-specific path anywhere: a music folder lives wherever the user put it, and CI has no~/Music.Real-device / real-filesystem limits
Documented in
docs/local-music.md:sysctl fs.inotify.max_user_watches), not something an app can raise. A folder over the limit is simply not live; manual refresh is unaffected.Directory.watch(recursive: true)adds watches for subfolders as it sees them created, which is what makes a newly-copied album folder work. A folder created during a window where the watch was being re-opened could in principle be missed until the next scan.Checks
dart format,flutter analyze(clean),flutter test: 39 new tests, 4972 passing (on top of #619's).Relationship to the other three
🤖 Generated with Claude Code
https://claude.ai/code/session_01Lq4ZJtwtdVCMBuZLjFbyfV
Generated by Claude Code