Summary
Idea: let users generate a playlist from a free-text description ("upbeat 90s rock", "chill stuff I haven't played in a while") using Nextcloud's Assistant/TaskProcessing infrastructure, rather than manually building Advanced Search criteria by hand.
Posting this as an RFC before writing any code — want feedback on the approach and whether it's a direction worth pursuing, especially given the maincontroller.js refactor already planned.
Why this seems tractable
The hard part of "AI playlist generation" is usually that you can't fit a whole library into an LLM's context window, so you'd normally need embeddings/vector search infrastructure that doesn't exist here. But Music already has a full structured filter engine in Advanced Search (AdvSearchRules.php, BaseMapper::findAllAdvanced) — {rule, operator, input} criteria (genre, artist, year, BPM, favorite, play count, date added, etc.) resolved server-side via SQL across the whole library. That's exactly the shape an LLM is good at producing from natural language, and it sidesteps the context-window problem entirely — the LLM only has to fill in a known schema, not reason about 70k+ tracks.
Proposed approach
- New UI affordance in Music: a free-text "Describe a playlist" input.
- New backend endpoint that:
- builds a prompt from the existing Advanced Search rule catalogue (rule keys, types, valid operators) + the user's text
- schedules a
core:text2text task via OCP\TaskProcessing\IManager, asking for a JSON response shaped like {conjunction, rules: [{rule, operator, input}, ...]}
- validates the returned JSON strictly against the known rule/operator list (LLM output isn't guaranteed to be valid JSON or to stay within the schema, so this needs to be defensive — reject/drop anything unrecognized)
- feeds the validated criteria into the existing
findAllAdvanced() path and builds the playlist via PlaylistBusinessLayer, reusing tested filtering logic rather than adding a new query engine
- Feature depends on the instance having an LLM provider configured for TaskProcessing (OpenAI/local model/etc. via whatever integration app is installed) — Music would check
hasProviders() / getAvailableTaskTypeIds() and hide the feature entirely when unavailable, rather than degrading awkwardly.
Open questions
- Where should this live in the UI — sidebar action, a variant of the existing smart-list panel, something else?
core:text2text has no structured-output guarantee. Is prompt-engineering + strict server-side validation good enough, or is this worth a dedicated TaskType instead (tradeoff: a custom type needs a provider that implements it — core providers won't, so in practice this would leave the feature unusable except for providers that specifically add support)?- Any concern about scope/overlap with the
maincontroller.js Promise refactor, given the new endpoint's frontend piece would touch similar view-controller wiring?
Not attempting this yet — wanted to get a read on the idea and the approach before spending time on an implementation.
Summary
Idea: let users generate a playlist from a free-text description ("upbeat 90s rock", "chill stuff I haven't played in a while") using Nextcloud's Assistant/TaskProcessing infrastructure, rather than manually building Advanced Search criteria by hand.
Posting this as an RFC before writing any code — want feedback on the approach and whether it's a direction worth pursuing, especially given the
maincontroller.jsrefactor already planned.Why this seems tractable
The hard part of "AI playlist generation" is usually that you can't fit a whole library into an LLM's context window, so you'd normally need embeddings/vector search infrastructure that doesn't exist here. But Music already has a full structured filter engine in Advanced Search (
AdvSearchRules.php,BaseMapper::findAllAdvanced) —{rule, operator, input}criteria (genre, artist, year, BPM, favorite, play count, date added, etc.) resolved server-side via SQL across the whole library. That's exactly the shape an LLM is good at producing from natural language, and it sidesteps the context-window problem entirely — the LLM only has to fill in a known schema, not reason about 70k+ tracks.Proposed approach
core:text2texttask viaOCP\TaskProcessing\IManager, asking for a JSON response shaped like{conjunction, rules: [{rule, operator, input}, ...]}findAllAdvanced()path and builds the playlist viaPlaylistBusinessLayer, reusing tested filtering logic rather than adding a new query enginehasProviders()/getAvailableTaskTypeIds()and hide the feature entirely when unavailable, rather than degrading awkwardly.Open questions
core:text2texthas no structured-output guarantee. Is prompt-engineering + strict server-side validation good enough, or is this worth a dedicatedTaskTypeinstead (tradeoff: a custom type needs a provider that implements it — core providers won't, so in practice this would leave the feature unusable except for providers that specifically add support)?maincontroller.jsPromise refactor, given the new endpoint's frontend piece would touch similar view-controller wiring?Not attempting this yet — wanted to get a read on the idea and the approach before spending time on an implementation.