Skip to content

ASR Backends

Jump to bottom
rcspam edited this page May 12, 2026 · 17 revisions

🌐 Language: English | Français

ASR Backends

dictee supports 5 speech recognition backends, each with different trade-offs on accuracy, language coverage, hardware requirements, and streaming capabilities. The default and recommended backend is Parakeet-TDT. A fifth backend — Nemotron — is CLI-only (for long English streaming + diarization).

Switching between backends is instant at runtime via dictee-switch-backend or the Setup Wizard. All backends share the same audio pipeline (PipeWire → PulseAudio → ALSA fallback) and feed the same 12-step Post-Processing-Overview.

Setup wizard — ASR backend selection

Table of Contents

Comparison table

Backend Languages Model size Warm latency (GPU) Warm latency (CPU) Punctuation Built-in translation* Streaming
Parakeet-TDT 0.6B v3 25 ~2.5 GB ~0.16 s ~0.8 s ✅ Native ❌ ⚠ File-only
Canary-1B v2 25 ~5 GB ~0.7 s Impractical ✅ Native ✅ (48 pairs ↔ EN, same forward pass) ❌
faster-whisper 99 500 MB–3 GB ~0.3 s ~0.5 s ✅ Native ⚠ EN only (via task=translate) ❌
Vosk 20+ 50 MB–1.5 GB N/A ~1.5 s ❌ Optional model ❌ ⚠ Model capability, not exposed in dictee
Nemotron 0.6B (CLI only) 1 (EN) ~2.5 GB Streaming Impractical ✅ Native ❌ ✅ Native (chunked)

* "Built-in translation" = translation performed by the ASR model itself in a single forward pass. A ❌ in this column does not mean dictee cannot translate that backend's output — it means the backend doesn't translate natively. Translation for any backend (Parakeet, Vosk, Whisper…) is always available through one of dictee's separate translation backends (Google, Bing, LibreTranslate, Ollama). See Translation for details.

Warm latency = time from end-of-speech to transcription ready, for a 5-second utterance, after the daemon has loaded the model (first-call cold latency is 2–5× higher).

When to choose which

Situation Recommended backend
Default daily dictation (FR/EN/DE/ES/IT/PT/RU/UK + 17 more) Parakeet-TDT
Need translation as part of transcription Canary-1B (25 langs ↔ English)
Exotic languages (Japanese, Arabic, Chinese, Hindi…) faster-whisper (99 langs)
Minimal resources (no GPU, 1 GB RAM, Raspberry Pi…) Vosk
Long English file + diarization (meetings, podcasts) Nemotron 0.6B via transcribe-stream-diarize (true streaming, no duration cap)
Streaming / live subtitles in the dictation UI None of the backends currently exposes live subtitles in the main UI (Vosk's streaming is model-level only; Nemotron streams but is CLI-only for file transcription) — planned for v1.4+
Highest accuracy (accepting latency) Canary-1B on GPU
Lowest latency Parakeet-TDT on GPU

Parakeet-TDT

Parakeet logo

Parakeet-TDT 0.6B v3 is NVIDIA's Token-and-Duration Transducer model with a FastConformer encoder. It's the default backend since v1.3.

Highlights:

  • 25 European languages (French, English, German, Spanish, Italian, Portuguese, Dutch, Ukrainian, Russian, Polish, Czech, Slovak, Slovenian, Croatian, Bulgarian, Romanian, Hungarian, Greek, Estonian, Latvian, Lithuanian, Finnish, Swedish, Danish, Maltese)
  • Native punctuation and capitalization (no post-processing regex needed for basic output)
  • Excellent accuracy on real-world audio (WER ~5–8% on CommonVoice FR/EN)
  • ~0.16 s warm latency on RTX 4070 for a 5-second utterance
  • Runs on CPU (~0.8 s) or GPU (~0.16 s)

Trade-offs:

  • The raw command-line transcribe tool can't process audio longer than ~5:20 min on any GPU (a known bug in the Parakeet-TDT v3 model). dictee-transcribe (the graphical app) works around this since v1.3.4 by automatically splitting any longer file into smaller chunks — no manual action needed.
  • Cannot stream — needs the complete utterance before transcription starts
  • No built-in translation

Deep dive: Parakeet-TDT-Deep-Dive covers architecture, VRAM breakdown, duration limits, known quirks (first-word drop, Cyrillic misdetection), and the upcoming hotwords feature.

Canary-1B

Canary logo

NVIDIA Canary-1B v2 is an attention-encoder-decoder (AED) model that transcribes and translates in a single pass.

Highlights:

  • 25 European languages supported (same set as Parakeet: EN, FR, DE, ES, IT, PT, NL, UK, RU, PL, CS, SK, SL, HR, BG, RO, HU, EL, ET, LV, LT, FI, SV, DA, MT)
  • Built-in translation: each of the 25 ↔ English = 48 pairs (no non-English pairs like FR→DE)
  • Best-in-class accuracy on supported languages
  • Native punctuation and capitalization
  • Decoder-context aware (prompt tokens for source language + target)

Trade-offs:

  • Requires a GPU in practice (CPU inference is impractically slow)
  • 5 GB VRAM minimum
  • Translation hub is English (for FR→DE you'd need FR→EN then EN→DE)
  • No streaming

CLI example:

# Transcribe only (French)
dictee-switch-backend asr canary
DICTEE_LANG_SOURCE=fr dictee

# Transcribe + translate FR → EN
DICTEE_LANG_SOURCE=fr DICTEE_LANG_TARGET=en dictee --translate

The Rust Canary implementation (in src/canary.rs) was originally ported from onnx-asr and is now fully self-contained.

faster-whisper

faster-whisper is a CTranslate2-optimized port of OpenAI's Whisper family, maintained by SYSTRAN.

Highlights:

  • 99 languages (widest coverage of any backend)
  • Multiple model sizes — tiny (39 MB), base (74 MB), small (244 MB), medium (769 MB), large-v3 (1.5 GB), large-v3-turbo (809 MB), distil-large-v3 (600 MB)
  • Runs well on both CPU and GPU
  • Native punctuation and capitalization
  • Optional translation via task="translate" → English only

Trade-offs:

  • Known to hallucinate on short/silent audio
  • Timestamps are inaccurate on large-v3 and turbo (DTW degraded) — audio-context continuation skipped
  • No streaming

Model selection:

# Set in dictee.conf or via environment
DICTEE_WHISPER_MODEL=large-v3-turbo dictee

Recommended for daily use: large-v3-turbo (best quality/speed trade-off) or distil-large-v3 (5× faster, ~95% of large-v3 accuracy).

Vosk

Vosk is a lightweight Kaldi-based offline recognizer maintained by Alpha Cephei.

Highlights:

  • Tiny models (50 MB typical, down to ~40 MB for mobile variants)
  • Streaming-capable model — Vosk's API exposes PartialResult() during recording, but dictee's daemon currently only calls FinalResult() (transcribe-daemon-vosk:125). Live text display is not yet surfaced in the UI.
  • Runs on CPU with no special hardware (Raspberry Pi 4 tested)
  • Strict offline — no network needed, no telemetry
  • 20+ language packs available at alphacephei.com/vosk/models

Trade-offs:

  • No native punctuation in most models (requires post-processing or the vosk-recasepunc add-on)
  • Lower accuracy than Parakeet/Canary/Whisper (WER ~15–20% on CommonVoice)
  • Model selection lottery: dictee picks alphabetical first if multiple are present (known issue)

Model directory:

ls ~/.cache/dictee/vosk/
# → vosk-model-fr-0.22/
# → vosk-model-small-en-us-0.22/

Nemotron 0.6B (CLI-only streaming)

NVIDIA Nemotron-Speech-Streaming-EN 0.6B is a specialized streaming ASR model, English-only, designed for long-form transcription with natural chunking.

Highlights:

  • True streaming — processes audio in 10-second chunks with internal overlap; no need to buffer the full utterance
  • English only (1 language)
  • ~2.5 GB model (encoder.onnx + decoder_joint.onnx)
  • Native punctuation and capitalization
  • No hard duration cap — can process multi-hour files with bounded VRAM

Where it's used:

  • The transcribe-stream-diarize Rust binary pairs Nemotron with Sortformer for streaming-friendly long-English-file diarization
  • Not exposed through dictee-switch-backend — you don't select "Nemotron" as your dictation backend
  • CLI-only: invoked directly for batch transcription, not for interactive dictation

📌 Since v1.3.0, the Nemotron model is no longer offered in dictee-setup — the streaming UI flow is parked until v1.4. The transcribe-stream-diarize binary is still shipped in the packages, so power users can still use the pipeline by downloading the model manually:

mkdir -p ~/.local/share/dictee/nemotron
cd ~/.local/share/dictee/nemotron
for f in encoder.onnx encoder.onnx.data decoder_joint.onnx tokenizer.model; do
  wget "https://huggingface.co/altunenes/parakeet-rs/resolve/main/nemotron-speech-streaming-en-0.6b/$f"
done

The interactive UI hookup (live captions in dictee-transcribe / dictee-tray) is on the v1.4 roadmap.

When to use:

  • Long English recordings (1h+ podcast, conference, lecture) where Parakeet/Canary would OOM
  • Multi-speaker English transcripts (Nemotron + Sortformer pipeline)

Trade-offs:

  • English only — use Parakeet/Canary/Whisper for other languages
  • No translation (combine with another translation backend post-transcription)
  • Not wired into the daemon / plasmoid / tray — purely a CLI tool

CLI example:

transcribe-stream-diarize meeting.wav \
  --nemotron-model ~/.local/share/dictee/nemotron/ \
  --sortformer-model /usr/share/dictee/sortformer/ \
  --format=plain

See Diarization for pipeline details.

Benchmarks

Measured on a TUXEDO InfinityBook Pro Gen8 (MK2) — Intel Core i7-13700H, RTX 4070 Laptop 8 GB, TUXEDO OS (kernel 6.17, NVIDIA 590.48.01), 5-second utterance in French, warm model:

Backend GPU latency CPU latency GPU VRAM CPU RAM
Parakeet-TDT 0.16 s 0.8 s 1.2 GB 3.1 GB
Canary-1B 0.7 s 4.5 s 5.1 GB 5.8 GB
faster-whisper (large-v3-turbo) 0.3 s 0.5 s 2.1 GB 2.8 GB
Vosk (fr-0.22) N/A 1.5 s N/A 1.9 GB

Cold latency (first call after daemon start): add ~2 s for Parakeet/Whisper/Vosk, ~5 s for Canary.

Switching at runtime

No restart needed — backend switching is instant via the dictee-switch-backend helper:

# Show current backends
dictee-switch-backend status
# → ASR: parakeet (dictee.service, active)
# → Translate: google (trans)

# Switch ASR
dictee-switch-backend asr canary
dictee-switch-backend asr whisper
dictee-switch-backend asr vosk
dictee-switch-backend asr parakeet

# List available
dictee-switch-backend list

The Tray-Icon and Plasmoid-Widget include backend sub-menus — no terminal needed.

Under the hood

Each dictation backend runs as a systemd user service:

Backend Service unit Daemon binary
Parakeet dictee.service transcribe-daemon (Rust)
Canary dictee-canary.service transcribe-daemon --canary (Rust)
Whisper dictee-whisper.service dictee-transcribe (Python, faster-whisper)
Vosk dictee-vosk.service dictee-transcribe (Python, vosk-api)
Nemotron (no daemon — CLI only) transcribe-stream-diarize (Rust)

All services listen on the same Unix socket at $XDG_RUNTIME_DIR/transcribe.sock, with a simple request/response protocol. Only one service is active at a time (the others are mutually exclusive via Conflicts= in their systemd units).

For the full protocol and daemon internals, see Developer-Guide.

Next steps

📖 dictee Wiki

🇬🇧 Home · 🇫🇷 Accueil

Getting started / Premiers pas

Speech recognition / ASR

Translation / Traduction

Post-processing / Post-traitement

CLI

Reference / Référence

🏠 Repo · 📦 Releases · 🐛 Issues

Clone this wiki locally