Epic: HTDemucs ONNX Offline Integration #15495
Description
Epic: HTDemucs ONNX Integration into Mixxx
This is an epic issue to track all the work on integrating ONNX-based source separation (Demucs v4 Hybrid Transformer) into Mixxx.
The goal is to enable DJs to separate tracks into stems (vocals, drums, bass, other) during analysis, directly inside Mixxx.
Note that this epic focuses on integration with ONNXRuntime and the Demucs model exported during GSoC 2025. While other models may be supported in the future, this effort is scoped to Demucs as the first integration.
Tasklist
Offline Pre-processing
- Add ONNXRuntime as a build dependency (cross-platform)
- [onnxruntime] new ports for microsoft/onnxruntime v1.23.2 microsoft/vcpkg#36850 ONNX-Runtime VCPKG port
- Merge upstream changes to 2.7 branch vcpkg#194 Merge upstream VCPKG with onnxruntime support into our repo
- Add onnxruntime port to our VCPKG manifest file
- Update VCPKG buildenv in mixxxdj/mixxx
- Ubuntu has a ONNX runtime packaoe since Version 25.04 (see https://repology.org/project/onnxruntime/versions)
- Detect available execution providers (either pick best automatically or list all available to choose from)
- Implement C++ wrapper for running Demucs ONNX inference
- Define pipeline for handling separated stems
- Add basic GUI controls (toggle separation (during analysis), trigger separation (individual tracks))
Planned COs (Control Objects)
These will be proposed to control separated stems inside Mixxx (inspired by stem mixing COs):
| CO | Description | Status |
|---|---|---|
[ChannelX],separation_trigger |
To trigger source separation for selected tracks | Not implemented |
[ChannelX],stem_count |
Number of separated stems available (read-only, 4 for Demucs) | Not implemented |
[ChannelX],separation_percentage |
Indicates percentage of the loaded track separated | Not implemented |
Limitations
- ONNXRuntime adds an external dependency, build system integration must be handled.
- Demucs model is large (~ 300 MB) → may be heavy for real-time use on older CPUs.
- Disk storage during analysis would quadruple the used storage space.
Use Case Log
- DJ loads a track into a deck.
- Toggles “Enable Source Separation.”
- Mixxx runs Demucs inference, providing 4 stems.
- User can solo/mute stems, apply effects separately, or mix stems live.
- Option to pre-split stems during track analysis.
- Separated stems cached on disk for later use in performance.
Relates to: (to be linked as subtasks are opened)