Auto-Library Rescan conditions

[GomuSkelly]

Feature Description

#14129

"Implementing an option, to define in seconds/minutes/hours the time between Mixxx automatically rescanning the Library would be of benefit."

In the MozAIk use case #14129, there can be new material generating every few seconds.

For users with vast Libraries, we're not sure how slowly a Library Rescan occurs.

In such scenarios, it may be preferrable to localise Library Rescans to a single directory under which newly generated material is being added, to negate the time/resources otherwise required to reprocess the entire Library.

Potentially having two options - Full Auto-Library Rescan and Auto-Directory Rescan could address this.

[ronso0]

Related requests:
#13989
#8532

[GomuSkelly]

Here's a 1080 video playing with generated material across 4 Decks:

https://www.youtube.com/watch?v=iJW8diFjkqY

In this experiment, the AutoDJ plays material on Decks 1+2 generated using the prompt "160bpm birds trap dubstep jungle" which creates sparse rhythmic material with no melody, and some rhythmic birdsong.

Decks 3+4 are being DJ'd manually with material generated from both sonic texture prompts (e.g "160bpm birds"/"160bpm male vocals") and music prompts (e.g"160bpm trap heaven church dubstep jungle"), but in contrast to what was described here #14129 in this instance the sonic texture also contains "160bpm" in the prompt which generates the sonic texture with rhythm.

[SchultzVV]

Hi! I'd like to volunteer to implement this. 👋

My use case is close to #14129: I have a terminal workflow that drops freshly downloaded + auto-tagged/BPM'd tracks into a library folder, and today they only appear after a manual Rescan Library or a restart.

A few questions for the team before I dive in:

Do you prefer a filesystem watcher over the timer-based periodic rescan also mentioned here, or should both be offered as separate options?
Is LibraryScanner the right entry point for a single-directory scan, or is there a preferred API for a scoped rescan?
Any known concerns with QFileSystemWatcher on large libraries / network mounts / the inotify max_user_watches limit that I should design around up front?
Happy to hash out the design on Zulip first if that's easier. Thanks!

[ronso0]

@SchultzVV Thanks for your interest to work on this!
You may be interested in #15940 which implements the watched Incoming folder.
For that it introduces the per-directory scan.

A while ago I also looked into the QFileSystemWatcher and learned that it only watches one specific directory (ie. not its sub-directories). Which means that in order to watch the entire library we'd need to install one watcher for each sub-directory -- which is a) overkill and b) might suffer race conditions in case for example a nested directory structure is created (or pasted? I don't remember).

A robust alternative to watch the entire library for file / dir changes might be platform-specific APIs.
I briefly looked into this with an agent and learned the buildings blocks are

• Win: ReadDirectoryChangesW + QWinEventNotifier
• Linux: inotify + QSocketNotifier
• macOS: FSEvents API / CFRunLoop

That's where I've quit : )
And implementation complexity (recursion and event handling) seems to vary greatly for the three.

Wanna look into this?

Discussing this on Zulip prior to any implementation is appreciated!
Though IMO this topic is low prio atm, so don't be disappointed in case you don't get many enthusiastic responses ; )