You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: content/news/2025-10-27-gsoc2025-demucs-to-onnx-dhunstack.md
+24-1Lines changed: 24 additions & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -261,7 +261,24 @@ To evaluate the exported ONNX model, we developed benchmarking scripts for both
261
261
## Benchmark Results
262
262
263
263
The quality of the model is expected to be equal or slightly worse when exported to ONNX. While there are plenty of ways of measuring the benchmarks models (another blog post incoming), we've chosen to measure our models with `SI-SDR` metric, Scale Invariant Signal To Distortion Ratio, on the MusDB dataset. This is the standard metric on which researchers report their source separation model's performance.
264
-
After export, the model's performance is nearly identical.
264
+
265
+
We conducted comprehensive benchmarking on both CPU and GPU platforms, evaluating 50 tracks from the MusDB test dataset (approximately 3.46 hours of audio).
266
+
267
+
### CPU Performance Comparison
268
+
269
+
| Metric | PyTorch Model | C++ ONNX Model | Improvement |
@@ -275,6 +292,12 @@ After export, the model's performance is nearly identical.
275
292
276
293
Typical high-performing models achieve around **7–9 dB** on vocals in the [MusDB](https://sigsep.github.io/datasets/musdb.html) benchmark - with 0 dB meaning no separation improvement over the original mixture.
277
294
295
+
**Key Findings:**
296
+
297
+
-**CPU Performance**: The C++ ONNX model delivers significant performance improvements (17.94% faster) while maintaining equivalent audio quality
298
+
-**GPU Performance**: PyTorch maintains a performance advantage on GPU (8.35% faster), likely due to optimized CUDA implementations
299
+
-**Audio Quality**: Both implementations produce nearly identical separation quality across all stems (< 0.1 dB difference)
300
+
278
301
Now that we have a running platform independent high quality ONNX Demucs model that can utilize hardware acceleration and be deployed with C++, we plan to integrate this into Mixxx DJ for future.
279
302
We've prepared example scripts for running the exported Demucs model, which can be used following the instructions documented in our READMEs.
0 commit comments