comparisons

Codec comparison

Reproduces the side-by-side reconstruction comparison on the GitHub Pages page and the tables in the top-level README. A speech-only comparison of STFT-VAE against Mimi, SNAC, and MioCodec over 8 LibriTTS-R clips (24 kHz, distinct public-domain speakers).

Layout

File	Runs in	What it does
fetch_speech.py	comparisons/.venv	download speech clips → comparisons/inputs/*.wav (24 kHz)
run.py	comparisons/.venv (GPU)	reconstruct through STFT-VAE / SNAC / Mimi → docs/samples/<clip>/*.wav
run_miocodec.py	comparisons/.venv-mio (CPU)	reconstruct through MioCodec, reading the original.wavs
metrics.py	comparisons/.venv	score every reconstruction → metrics.json
build_page.py	comparisons/.venv	emit the Pages page docs/index.html

MioCodec needs the real torchaudio (its encoder is torchaudio.pipelines WavLM), which has no build matching this repo's bleeding-edge torch — so it runs on CPU in a separate venv with a matched torch/torchaudio pair.

Setup & run

# main venv (GPU): torch + snac + transformers(Mimi) + metrics + data fetch
uv venv comparisons/.venv --python 3.12
uv pip install --python comparisons/.venv/bin/python \
  torch==2.13.0 numpy==2.1.3 "numba>=0.61" scipy soundfile librosa datasets \
  transformers huggingface_hub safetensors einops snac pesq pystoi

# MioCodec venv (CPU): matched torch/torchaudio
uv venv comparisons/.venv-mio --python 3.12
uv pip install --python comparisons/.venv-mio/bin/python \
  --index-url https://download.pytorch.org/whl/cpu torch==2.8.0 torchaudio==2.8.0
uv pip install --python comparisons/.venv-mio/bin/python \
  einops huggingface-hub json5 jsonargparse julius numpy ruamel.yaml safetensors \
  soundfile "git+https://github.com/Aratako/MioCodec"

# run the pipeline
comparisons/.venv/bin/python comparisons/fetch_speech.py
CUDA_VISIBLE_DEVICES=0 PYTHONPATH=. comparisons/.venv/bin/python comparisons/run.py
comparisons/.venv-mio/bin/python comparisons/run_miocodec.py
comparisons/.venv/bin/python comparisons/metrics.py
comparisons/.venv/bin/python comparisons/build_page.py

fetch_speech.py prefers LibriTTS-R (24 kHz native) and falls back to LibriSpeech dev-clean; it picks one clip per distinct speaker.

Metrics

• SI-SDR (dB, ↑) — scale-invariant waveform fidelity.
• Mel-L1 (dB, ↓) — L1 on log-mel spectrograms.
• PESQ-wb (↑) and STOI (↑) — wideband speech quality/intelligibility at 16 kHz.

All scores are computed after ±50 ms cross-correlation alignment so a constant codec latency doesn't distort them. The reconstructions are committed as lossless WAV under docs/samples/ (un-ignored in .gitignore) and served by the Pages page.