Proposal: Implement XDG Base Directory Support#18
Conversation
- Template structure with all required sections - Metadata populated with issue link and spec link - Placeholder content in each section body Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Explain what ~/.mixxx is and why it violates XDG conventions - Reference XDG Base Directory Specification with link - Cite cmdlineargs.cpp comment and issue #8090 - Note macOS/Windows already use platform-standard paths Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Home directory clutter from visible dotfile - No separation of concerns across file categories - Cache cleanup tools cannot reach ~/.mixxx - Read-only home directory breaks Mixxx - Inconsistent cross-platform behavior Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…onale - Complete table mapping all 18 entries (13 files + 5 directories) to XDG categories - Each row has Current Path, XDG Category, New Path, and Rationale columns - Notes section addresses sandbox.cfg, broadcast_profiles, effects.xml/samplers.xml reasoning, and legacy files Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Warns about BleachBit/systemd-tmpfiles deleting expensive-to-regenerate data - Documents regeneration cost (hours for large libraries) - Recommends cache placement per spec with documentation mitigation Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…ation - Platform matrix table with QStandardPaths enum per category per platform - Config/data split documented as Linux-only with macOS/Windows rationale - Removed placeholder paragraph Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- StateLocation Qt 6.7+ gap with compile-time guard code block - Fallback path table for Qt < 6.7 on all platforms - Flatpak XDG variable remapping with sandbox paths - Snap HOME remapping documented for completeness Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…or-nothing mode - Startup decision tree with --settings-path as first check - Legacy paths per platform table - --settings-path flag semantics documented - All-or-nothing mode selection with rationale Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- No automatic migration policy with daschuer beta-testing warning - Firefox 147 precedent and community consensus cited - Wiki migration guide outline with all sections - Action Plan items marked complete Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
… bridge - MixxxPathResolver API sketch with PathMode enum and typed QDir accessors - Call site inventory: 38 sites across 21 files grouped by XDG category - Three-phase deprecation bridge for getSettingsPath - Notable edge cases documented (Custom.kbd.cfg, trailing slash, path substitution) - Before/after code examples for data and cache categories Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Owen Williams <owen.williams@grafana.com>
|
Thanks for looking into this. Looks like the pre-commit hook is failing, could you please look into this before we start the review? |
|
"no newline at end of file" lol |
Signed-off-by: Owen Williams <owilliams@mixxx.org>
|
The main one is |
Signed-off-by: Owen Williams <owilliams@mixxx.org>
Signed-off-by: Owen Williams <owilliams@mixxx.org>
|
ok, all cleaned up |
| The `analysis/` directory stores pre-computed waveform data (waveforms, | ||
| beatgrids). This data is regenerable from the original audio files, | ||
| making it a cache by the XDG spec definition. However, regeneration is | ||
| expensive: several seconds per track on modern hardware, meaning a | ||
| library of 10,000 tracks could take many minutes to fully re-analyze | ||
| even in parallel. |
There was a problem hiding this comment.
this to me is the biggest question: is waveform data cache or data? On a fast computer, regenerating waveforms on demand during a set is not a big deal, but on a slow / raspberry pi, this could cause underruns. So I can see arguments either way.
I am tempted to call it cache, and then just warn users not to delete caches right before a big performance
There was a problem hiding this comment.
As per specification, I'd also say waveforms belong in cache. They're user specific, automatically regenerated and non-essential.
The spec doesn't go into detail what non-essential actually means, so the question is: is waveform data essential for user experience? And if so, should we keep them in a safer location in hopes of preventing accidental deletion?
If I understand correctly, the cache dir is not cleared unless the user manually deletes it, uses tmpfiles.d (or something similar) to automatically do so or remaps cache to a tmpfs mount. It does seem to take considerable effort, so a warning might be enough.
There was a problem hiding this comment.
On a fast computer, regenerating waveforms on demand during a set is not a big deal
Worth to say that this is not entirely true when considering STEM.
and then just warn users not to delete caches right before a big performance
I think this is a fair assumption recommendation!
| #include <QDir> | ||
| #include <QString> | ||
|
|
||
| class MixxxPathResolver { |
There was a problem hiding this comment.
we don't have to implement it 100% this way. If we start bikeshedding this code snippet, I'd prefer to just remove it and specify a rough API in this proposal
djantti
left a comment
There was a problem hiding this comment.
Overall this seems like a reasonable approach, even from a language agnostic perspective. And it's easy to understand what's happening with basic C++ knowledge. It's simple, effective and requires no actions from users. 👍
| sandbox. When Mixxx switches to XDG paths, this directive should be | ||
| removed. Alternatively, it can be retained temporarily to support | ||
| legacy detection during the transition period (see Phase 4). | ||
|
|
There was a problem hiding this comment.
Things are a bit more complicated. The --persist=.mixxx flag is used to bind mount ~/.mixxx from inside the sandbox to ~/.var/app/org.mixxx.Mixxx/.mixxx on the host. That means that Flatpak Mixxx will always see a legacy config directory. I'm pretty sure we can never remove the flag or existing legacy settings would be inaccessible.
So we need to check if ~/.mixxx is empty and only then use the new XDG paths. We can also check for container environment variable so that this only happens for Flatpak.
There was a problem hiding this comment.
That means that Flatpak Mixxx will always see a legacy config directory. I'm pretty sure we can never remove the flag or existing legacy settings would be inaccessible.
We could extend the migration path to detect whether the folder is also not empty.
However, I believe Flatpak will create the folder before binding it, so new install will still this to this empty folder being create I believe, and when we drop the binding it would remain there, orphaned.
I guess this is an acceptable side effect? We can always communicate cleanup instruction
There was a problem hiding this comment.
Yes, I think that is correct. With --persist the legacy config directory will always be created. It will be an empty directory for new installs and also recreated if manually removed. And flatpak-override / Flatseal can't touch it as long as it's defined directly in the build manifest. The directory can be deleted only after it's been dropped from the manifest.
But I'd say that is totally acceptable. Flatpak relies heavily on XDG paths, so this will be a really nice improvement.
acolombier
left a comment
There was a problem hiding this comment.
Thanks for getting this 11-yo issue moving forward!
| macOS and Windows already use platform-standard locations via | ||
| `QStandardPaths::AppLocalDataLocation` (`~/Library/Application | ||
| Support/Mixxx/` on macOS, `%LOCALAPPDATA%/Mixxx/` on Windows). | ||
| Linux and BSD are the only platforms where Mixxx uses a hardcoded | ||
| non-standard path. |
There was a problem hiding this comment.
I am not entirely familial with these platform, but I am not sure this is true? On Windows, there is also different folders, based of the type of data (e.g C:\Users\<Username>\AppData\Local\Temp and C:\Users\<Username>\AppData\Local\). Definitely not in the scope there, though we might want to make sure the approach we take facilitate better support of platform-specific policy, such as Android.
|
|
||
| - **Read-only home directory breaks Mixxx.** Users who set `$HOME` | ||
| to read-only (a practice for testing XDG compliance) cannot run | ||
| Mixxx, since it writes directly to `~/.mixxx`. |
There was a problem hiding this comment.
Wouldn't this be a problem with default XDG path anyway? Setting the home dir as readonly would also impact ~/.local/share or ~/.config, wouldn't it?
|
|
||
| - **Inconsistent cross-platform behavior.** macOS and Windows | ||
| already use platform-standard locations via `QStandardPaths`. | ||
| Only Linux and BSD use a hardcoded non-standard path. |
There was a problem hiding this comment.
This doesn't seem to be true looking at the code, as mentioned above. We use QStandardPaths::AppLocalDataLocation for all data, instead of leveraging QStandardPaths::CacheLocation and QStandardPaths::StateLocation,
| - Separate Mixxx files into the correct XDG categories (config, data, | ||
| state, cache) on Linux. | ||
| - Use platform-standard locations on macOS and Windows. | ||
| - Preserve backward compatibility for existing installations that use |
There was a problem hiding this comment.
Worth to say that keeping a "single folder" data persistency as option is also quite useful when you want to use different profile. For example, depending of the work I do, including testing, I would use --settings-path ~/.mixxx_dev/ for arbitrary feature tests, or --settings-path ~/.mixxx_3.0_dev/ for QML work.
I think it would be great to keep that functioning instead of only allowing migration-like approach (not sure if this is what is suggested?)
Edit: Already mentioned later, great!
| | `mixxx.log` | state | `~/.local/state/Mixxx/mixxx.log` | Current session log; the spec lists "action history (logs)" as state | | ||
| | `mixxx.log.1` through `mixxx.log.9` | state | `~/.local/state/Mixxx/mixxx.log.1` .. `.9` | Rotated log files; same rationale as current session log | |
There was a problem hiding this comment.
Wondering if it would make sense to keep in ~/.cache?
I know when I am running out of space of my system, I usually delete the ~/.cache to reclaim a few gigs.
Now Mixxx's log file may reach 100MB, and we keep up to 10, meaning that is up to a gig to be safely reclaimable. What do you think?
| 1. **Complexity.** Every file access would need try-legacy-then-XDG | ||
| logic, creating 30+ branch points across the 21 call sites that | ||
| reference `getSettingsPath()`. |
There was a problem hiding this comment.
I assume we will want to create an abstraction layer, to later support the multi directory on other platform as well
See suggestion in the interface
| **Community consensus:** acolombier proposed in #8090 to "support both | ||
| locations for now, and create in the new XDG compatible location on new | ||
| setup only." Krafting responded: "seems like a better idea than copying | ||
| data indeed!" daschuer suggested: "don't copy, use the old folder if | ||
| found, but default to the new if not." This proposal implements that | ||
| consensus. |
| In [issue #8090](https://github.com/mixxxdj/mixxx/issues/8090), | ||
| holzhaus proposed (October 2021): "As a first step, we should | ||
| introduce a class that allows accessing the different path types | ||
| (cache, data, config) and use it everywhere." The class sketched | ||
| here follows that approach. |
| #include <QDir> | ||
| #include <QString> | ||
|
|
||
| class MixxxPathResolver { |
There was a problem hiding this comment.
Perhaps an alternative proposition here would be to abstract the path resolution. As I understand with this interface, we expect the caller to apply the "try-legacy-then-XDG" logic, leading to a lot of duplicated logic.
Instead of expecting the caller to craft the path (currently by concatenating getSettingsPath + sub path), we expect something like
resolve(PathType::Config type, QString prefix): QString (or QFile)
This could allow:
- Abstracting the XDG/Legacy logic away
- Add per-type migration logic in the future
- Handle platform specific request only once (e.g sandboxing)
- Remove/abstract away QDir references (
configDir(),dataDir(),stateDir()andcacheDir()) so no risk of mishandling from the caller
wdyt?
| // Before (library/dao/analysisdao.cpp) | ||
| QDir analysisDir(m_pConfig->getSettingsPath() + "analysis"); | ||
| // After | ||
| QDir analysisDir(m_pathResolver.cacheDir().filePath("analysis")); |
There was a problem hiding this comment.
Just for reference here, here is my suggestion:
| QDir analysisDir(m_pathResolver.cacheDir().filePath("analysis")); | |
| QDir analysisDir(m_pathResolver.resolve(PathType::Cache, "analysis")); |
In case of analysis, this aligns well with the type being potentially conditional, whilst keeping the use nit and simple.
| QDir analysisDir(m_pathResolver.cacheDir().filePath("analysis")); | |
| QDir analysisDir(m_pathResolver.resolve(useStateInsteadOfCache? PathType::State : PathType::Cache, "analysis")); |
AI-assisted proposal for migrating Mixxx from ~/.mixxx to XDG Base Directory compliant paths on Linux/BSD, using QStandardPaths for platform-standard locations on all platforms. Per our discussions, proposes a detect-not-migrate strategy: existing ~/.mixxx users keep it, fresh installs get XDG paths. Users wishing to migrate can do so manually if they choose. Minimal code overhead required for maintaining the backwards compatibility forever. (Of note, this is the approach Firefox uses for the sames issues).
I've gone through the whole thing and touched it up, and tried to remove some extra verbosity.
There is one primary question for discussion: should waveform analysis be considered cache or data? There are good arguments for both. This proposal calls it cache, but does point out the cpu cost of regenerating the data.
There's also some stuff about QT 6.7 and the "state" directory, we can decide if we want that level of specificity or just remove it