Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 11 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@

Portable Demucs v4 (`htdemucs`) inference built from exported ONNX models and a Rust orchestration layer. The repository includes a native CLI, Node binding, and fully client-side WASM web app.

The model artifacts are derived from the [upstream Demucs models](https://github.com/adefossez/demucs), and the export builds on the ONNX work in [adefossez/demucs#10](https://github.com/adefossez/demucs/pull/10).

The CLI separates WAV input into Demucs stems using locally exported ONNX models. Model export requires PyTorch, but inference uses the native Rust and ONNX Runtime stack.

## Usage
Expand All @@ -11,11 +13,16 @@ Prerequisites: Rust, `uv`, and `pnpm`.
One-time setup from the repository root:

```bash
# Install the pinned export environment and build the standard model. This downloads the
# upstream checkpoint, exports ONNX, and moves shared DFT data into one external file.
pnpm install
pnpm build:model htdemucs
```

Alternatively, download the prebuilt standard model using a tag from the [releases page](https://github.com/hi-ogawa/demucs-onnx/releases):

```bash
pnpm model-release download models-2026-07-11 htdemucs
```

Separate a WAV file into four stems:

```bash
Expand All @@ -32,7 +39,7 @@ Useful variants:
- `--shifts N` averages `N` seeded-offset passes. The default is one pass.
- `node crates/napi/cli.mjs separate ...` exposes the same flow through the Node binding.

The setup above creates the size-optimized standard model under `data/onnx-lean/`. Run `uv run python tools/model-export/build_models.py --all` to build the standard model and all fine-tuned specialists. Passing explicit member names builds an exact subset.
Both setup paths create the size-optimized standard model under `data/onnx-lean/`. Run `pnpm build:model --all` to build the standard model and all fine-tuned specialists locally. Passing explicit member names builds an exact subset. For release downloads, omit `htdemucs` to download all models. See [Model releases](docs/model-release.md) for partial downloads and maintainer publishing instructions.

## Web App

Expand All @@ -54,6 +61,7 @@ Open `http://localhost:5173`, choose a local audio file, and run separation. Aud
- `packages/app/` contains the fully client-side Vite app.
- `tools/model-export/` contains the uv-managed model export, DFT stripping, and parity tools.
- `docs/architecture.md` describes the runtime and artifact design.
- `docs/model-release.md` documents downloading and publishing model artifacts.
- `docs/demucs.md` documents upstream Demucs mechanics and prior art.
- `docs/development.md` records the chronological implementation process and measurements.
- `docs/history.md` preserves the original motivation and prototype completion timeline.
Expand Down
9 changes: 9 additions & 0 deletions docs/model-release-notes.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
Size-optimized Demucs ONNX models for native and browser inference. These are derived artifacts based on the [upstream Demucs models](https://github.com/adefossez/demucs) and the ONNX export work in [adefossez/demucs#10](https://github.com/adefossez/demucs/pull/10).

Download one of these sets:

- Standard four-stem and two-stem modes: `dft.bin` and `htdemucs.onnx`.
- Fine-tuned target/minus mode: `dft.bin` and the corresponding `htdemucs_ft_<source>.onnx` specialist.
- Full fine-tuned support: `dft.bin` and all four fine-tuned specialists.

Every ONNX model references the shared `dft.bin` external-data file. Keep both files in the same directory for native ONNX Runtime, or supply `dft.bin` through ONNX Runtime Web's external-data API.
54 changes: 54 additions & 0 deletions docs/model-release.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
# Model releases

GitHub Releases distribute the size-optimized ONNX models separately from the application. Normal local development downloads these artifacts; rebuilding them is only necessary when changing the export pipeline or preparing new model bytes.

## Download for local development

Prerequisites are Python, `gh` authenticated with access to the repository, and `pnpm`:

```bash
gh auth status
pnpm model-release download models-2026-07-11
```

The download command retrieves all five ONNX files and `dft.bin` into `data/onnx-lean/`. It replaces an existing model directory only after the requested assets have downloaded successfully.

Pass member names after the tag to download only a useful subset:

```bash
# Standard four-stem and two-stem modes.
pnpm model-release download models-2026-07-11 htdemucs

# Fine-tuned bass/minus mode.
pnpm model-release download models-2026-07-11 htdemucs_ft_bass
```

Every subset includes `dft.bin`. A partial download replaces `data/onnx-lean/`, so only the selected workflows remain available locally.

## Build and publish

Model maintainers need Python with `uv` in addition to the download prerequisites. Build the complete release set locally:

```bash
pnpm build:model --all
```

Create a release and upload the six model assets by choosing an explicit tag:

```bash
pnpm model-release release models-2026-07-11
```

Update an existing release by explicitly replacing its same-named assets:

```bash
pnpm model-release release models-2026-07-11 --update
```

Inspect a release with:

```bash
gh release view models-2026-07-11
```

Prefer a new date-based tag when model bytes change. Overwriting a published release should be reserved for correcting an upload before consumers depend on it.
1 change: 1 addition & 0 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@
"prepare": "vp config",
"bass-cover": "python3 tools/bass-cover.py",
"build:model": "uv run python tools/model-export/build_models.py",
"model-release": "python tools/model_release.py",
"build:wasm": "wasm-pack build crates/wasm --target web --release",
"cli-separate": "cargo run --release -p demucs-cli -- separate --models data/onnx-lean",
"dev": "pnpm -C packages/app dev",
Expand Down
136 changes: 136 additions & 0 deletions tools/model_release.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,136 @@
#!/usr/bin/env python3
"""Download or publish the shipped Demucs model artifacts."""

import argparse
import os
import shlex
import shutil
import subprocess
import tempfile
from pathlib import Path

REPO = "hi-ogawa/demucs-onnx"
MEMBERS = [
"htdemucs",
"htdemucs_ft_drums",
"htdemucs_ft_bass",
"htdemucs_ft_other",
"htdemucs_ft_vocals",
]
ASSETS = ["dft.bin", *(f"{member}.onnx" for member in MEMBERS)]
REPO_DIR = Path(__file__).resolve().parents[1]
MODELS_DIR = REPO_DIR / "data/onnx-lean"


def run(args: list[str], capture_output: bool = False) -> subprocess.CompletedProcess[str]:
print(shlex.join(args), flush=True)
return subprocess.run(
args,
check=True,
text=True,
capture_output=capture_output,
)


def download(args: argparse.Namespace) -> None:
members = list(dict.fromkeys(args.members or MEMBERS))
expected = ["dft.bin", *(f"{member}.onnx" for member in members)]
MODELS_DIR.parent.mkdir(parents=True, exist_ok=True)

with tempfile.TemporaryDirectory(
prefix=".onnx-lean.download-", dir=MODELS_DIR.parent
) as temporary:
staging = Path(temporary)
patterns: list[str] = []
for name in expected:
patterns.extend(("--pattern", name))
run(
[
"gh",
"release",
"download",
args.tag,
"--repo",
REPO,
"--dir",
str(staging),
*patterns,
]
)

for name in expected:
if not (staging / name).is_file():
raise SystemExit(f"release {args.tag} is missing asset: {name}")

if MODELS_DIR.exists():
shutil.rmtree(MODELS_DIR)
os.replace(staging, MODELS_DIR)

print(f"downloaded {len(members)} model(s) and dft.bin to {MODELS_DIR}")


def release(args: argparse.Namespace) -> None:
missing = [str(MODELS_DIR / name) for name in ASSETS if not (MODELS_DIR / name).is_file()]
if missing:
raise SystemExit("\n".join([
"missing release assets:",
*missing,
"build the complete set with: pnpm build:model --all",
]))

paths = [str(MODELS_DIR / name) for name in ASSETS]
if args.update:
run(
[
"gh",
"release",
"upload",
args.tag,
*paths,
"--clobber",
"--repo",
REPO,
]
)
else:
run(
[
"gh",
"release",
"create",
args.tag,
*paths,
"--repo",
REPO,
"--target",
"main",
"--title",
f"Demucs ONNX models ({args.tag})",
"--notes-file",
str(REPO_DIR / "docs/model-release-notes.md"),
]
)

run(["gh", "release", "view", args.tag, "--repo", REPO])


def main() -> None:
parser = argparse.ArgumentParser(description=__doc__)
subparsers = parser.add_subparsers(required=True)

download_parser = subparsers.add_parser("download", help="download release assets")
download_parser.add_argument("tag")
download_parser.add_argument("members", nargs="*", choices=MEMBERS)
download_parser.set_defaults(func=download)

release_parser = subparsers.add_parser("release", help="create or update a release")
release_parser.add_argument("tag")
release_parser.add_argument("--update", action="store_true")
release_parser.set_defaults(func=release)

args = parser.parse_args()
args.func(args)


if __name__ == "__main__":
main()