Skip to content

feat(local): watch the music folders and refresh automatically (#409) - #620

Draft
TheZupZup wants to merge 2 commits into
perf/411-incremental-local-scanfrom
feat/409-linux-library-watcher
Draft

feat(local): watch the music folders and refresh automatically (#409)#620
TheZupZup wants to merge 2 commits into
perf/411-incremental-local-scanfrom
feat/409-linux-library-watcher

Conversation

@TheZupZup

Copy link
Copy Markdown
Owner

Closes #409. Part of #376.

Depends on #619 (incremental local scans). This branch is based on perf/411-incremental-local-scan, not main, and the PR targets that branch so the diff shows only the watcher. Once #619 merges, retarget this to main and it needs no rebase.

The dependency is real, not bookkeeping: a watcher that triggered a full rescan on every change would be worse than no watcher at all. This is only affordable because a refresh now re-reads just the files whose size or mtime moved.

What changes

On Linux the selected music folders are watched recursively, and a change under one of them refreshes the library. Adding an album, deleting a track or renaming a folder shows up without pressing Rescan.

The watcher owns no catalog logic, deliberately. A filesystem change turns into scanFolders(...), the same call the Rescan button makes. There is exactly one way the local catalog is written, and a watcher with its own idea of how to apply a change would be a second one that could disagree with the first, which is exactly the kind of drift that produces "the library is wrong but a rescan fixes it" bugs.

Coalescing

Copying one album produces a burst of events (a create and several modifies per file, plus directory events), so a scan per event would be hundreds of scans of the whole library for one user action.

  • An event starts, or extends, a quiet period. A burst collapses into one scan: 37 events, one scan in the test that reproduces an album copy.
  • A burst that never goes quiet, which is what a copy over a slow network mount looks like, is not allowed to postpone the refresh forever. Past a maximum window it refreshes anyway and starts a new one, so a long copy fills in while it runs rather than only at the end.
  • Events that arrive while a refresh is running schedule exactly one more, not one per event.

Noise is filtered before any of that. A music folder is full of things that change constantly while music is being added and cannot change the catalog: a downloader's .part files, cover art, .nfo/.log/.cue sidecars, editor swap files, and the bookkeeping folders sync tools scatter around. A path with an extension has to be one Linthra can import; a path without one is accepted, because that is what a directory looks like and a directory event is how a new or renamed album folder is noticed.

Failure is expected, not exceptional

  • inotify has a per-user watch limit and a big library on a machine with a low fs.inotify.max_user_watches can exhaust it.
  • A folder can vanish, or live on a filesystem that cannot be watched at all (NFS and SMB do not report changes another machine made).

Any of those marks that one folder unwatched, leaves the others watching, and changes nothing about manual refresh. A watch that merely ends counts as a failure too: nothing under it will be reported again, and calling that "still watching" is a lie the user cannot see through. Re-syncing retries a failed folder, because the reason it failed (a drive that was not mounted, a watch budget that was full) is usually temporary and rescanning is exactly when someone expects another attempt.

unwatchableRoots / isDegraded are exposed for a UI to read. Surfacing them in Settings is the obvious follow-up and is deliberately not in this PR: the acceptance criterion is that a watcher failure does not break manual refresh, which is covered, and a Settings row is a separate, reviewable change.

Files

  • lib/core/sources/local/local_directory_watch.dart is the OS seam. The unsupported implementation is what Android gets: its local library is a Storage Access Framework tree or a MediaStore query, not a directory Linthra may walk, so there is no path to watch.
  • lib/core/sources/local/local_library_watcher.dart is the debounce, the relevance filter, the failure handling and the disposal. No Flutter, no I/O.
  • lib/features/library/local_library_watch_service.dart keeps the watched set in step with the selection and points a change at the incremental scan. Bootstrap reads it once, and the container's disposal releases the watches, so a shutdown gives its inotify descriptors back rather than leaving them to the process exit.

Protecting user data

Nothing here writes to the user's folders, so there is no feedback loop to guard against: Linthra never creates, moves or deletes a file under a music folder, and the only thing a refresh does to disk is read.

Tests

Synthetic filesystem (local_library_watcher_test.dart, 28 cases): only selected folders are watched and a nested one is not watched twice; re-syncing opens nothing new; removing a folder releases it; add / delete / rename / folder-rename / re-tag each refresh once; .part files, cover art and sidecars refresh nothing; an album copy is one scan; a burst that never goes quiet still refreshes; 20 events during a scan produce one rescan; a refusing folder, a watch that errors, a watch that just ends, retry, and a refresh that throws; and disposal releasing every watch, cancelling a scheduled refresh, being idempotent, and ignoring later syncs.

Wired into the app graph (local_library_watch_service_test.dart): the selection drives what is watched, a new file lands in the catalog with nobody pressing Rescan, a deleted one leaves it, Android watches nothing, and disposing the container releases the watches.

Real filesystem (io_directory_watch_test.dart): the Directory.watch seam against a temp directory the OS chooses, so a wrong assumption about the real thing cannot survive the whole suite. It skips itself where the host cannot watch, which is the same honest outcome the app has there. No distro-specific path anywhere: a music folder lives wherever the user put it, and CI has no ~/Music.

Real-device / real-filesystem limits

Documented in docs/local-music.md:

  • inotify's watch limit is a system setting (sysctl fs.inotify.max_user_watches), not something an app can raise. A folder over the limit is simply not live; manual refresh is unaffected.
  • Many network filesystems cannot be watched. NFS and SMB do not report changes another machine made, so those folders need a manual rescan.
  • Hidden files and folders are not watched, so music inside a hidden folder is found by a rescan but not live.
  • Directory.watch(recursive: true) adds watches for subfolders as it sees them created, which is what makes a newly-copied album folder work. A folder created during a window where the watch was being re-opened could in principle be missed until the next scan.
  • Flatpak: watching works through the same portal document that exposes the folder, so it follows whatever that grant allows.
  • Android is not affected at all.

Checks

dart format, flutter analyze (clean), flutter test: 39 new tests, 4972 passing (on top of #619's).

Relationship to the other three

🤖 Generated with Claude Code

https://claude.ai/code/session_01Lq4ZJtwtdVCMBuZLjFbyfV


Generated by Claude Code

claude and others added 2 commits September 10, 2026 15:25
Depends on #619 (incremental local scans). Branched from it rather than main,
because a watcher that triggered a full rescan on every change would be worse
than no watcher at all: this is only affordable because a refresh now re-reads
only the files whose size or mtime moved.

On Linux the selected music folders are watched recursively, and a change under
one of them refreshes the library. Adding an album, deleting a track or
renaming a folder shows up without pressing Rescan.

The watcher owns no catalog logic on purpose. A filesystem change turns into
scanFolders(...), the same call the Rescan button makes. There is exactly one
way the local catalog is written, and a watcher with its own idea of how to
apply a change would be a second one that could disagree with the first.

Coalescing is the design:

* an event starts, or extends, a quiet period, so a burst collapses into one
  scan. Copying a 12-track album is 37 events and one scan;
* a burst that never goes quiet, which is what a copy over a slow network mount
  looks like, is not allowed to postpone the refresh forever: past a maximum
  window it refreshes anyway and starts a new one, so a long copy fills in
  while it runs rather than only at the end;
* events during a refresh schedule exactly one more, not one per event.

Noise is filtered before any of that. A music folder is full of things that
change constantly while music is being added and cannot change the catalog: a
downloader's .part files, cover art, .nfo/.log/.cue sidecars, editor swap
files, and the bookkeeping folders sync tools scatter around. A path with an
extension has to be one Linthra can import; a path without one is accepted,
because that is what a directory looks like and a directory event is how a new
or renamed album folder is noticed.

Failure is expected rather than exceptional. inotify has a per-user watch
limit, a folder can vanish, and many network filesystems cannot be watched at
all. Any of those marks that one folder unwatched, leaves the others watching,
and changes nothing about manual refresh. A watch that merely *ends* counts as
a failure too: nothing under it will be reported again, and calling that "still
watching" is a lie the user cannot see through. Re-syncing retries, because the
reason it failed is usually temporary.

Modules:

* lib/core/sources/local/local_directory_watch.dart is the OS seam. The
  unsupported implementation is what Android gets: its local library is a SAF
  tree or a MediaStore query, not a directory, so there is no path to watch.
* lib/core/sources/local/local_library_watcher.dart is the debounce, the
  relevance filter, the failure handling and the disposal. No Flutter, no I/O.
* lib/features/library/local_library_watch_service.dart keeps the watched set
  in step with the selection and points a change at the incremental scan.
  Bootstrap reads it once, and the container's disposal releases the watches,
  so a shutdown gives its inotify descriptors back rather than leaving them to
  the process exit.

Nothing here writes to the user's folders, so there is no feedback loop to
guard against: Linthra never creates, moves or deletes a file under a music
folder, and the only thing a refresh does to disk is read.

Tests: 39 new, 4972 passing overall. The behaviour is exercised against a
synthetic filesystem, because pinning Linthra's own rules to inotify's timing
would make them flaky, and one small suite exercises the real
`Directory.watch` seam against a temp directory the OS chooses, so a wrong
assumption about the real thing cannot survive the whole suite. That suite
skips itself where the host cannot watch, which is the same honest outcome the
app has there.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Lq4ZJtwtdVCMBuZLjFbyfV
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants