feat(asr): add Parakeet TDT 0.6B engine via sherpa-onnx - #771
Open
AlexThunder1989 wants to merge 5 commits into
Open
feat(asr): add Parakeet TDT 0.6B engine via sherpa-onnx#771AlexThunder1989 wants to merge 5 commits into
AlexThunder1989 wants to merge 5 commits into
Conversation
Adds a "parakeet" engine backed by sherpa-onnx, running NVIDIA Parakeet TDT 0.6B int8 models on CPU: v3 (25 European languages, auto-detected) and v2 (English). Output is punctuated and capitalized. On an Intel Core Ultra 7 155H it decodes at about 12x realtime on CPU (RTF 0.08), roughly 8x faster than whisper.cpp large-v3-turbo over the same clips with its Vulkan GPU backend (RTF 0.59). Being a transducer it also has no 30s window to pad, so a short push-to-talk utterance returns in ~0.3s instead of ~2.4s. Follows the existing engine layout: utils/parakeet_model_info.py holds the catalog and the get_model_path / is_model_downloaded / list_downloaded_models / delete_model helpers, while _init_parakeet / _transcribe_with_parakeet / _download_parakeet_model follow the same shape as their whisper.cpp counterparts and reuse _stream_model_download for progress and cancellation. Registered in the engine dispatch, the settings engine/model/language pickers, the model info card, the download prompts, the unused-downloads list and install.sh. Models download to <models_dir>/parakeet/<model>/ via temp files renamed on success; is_model_downloaded requires all four files, so an interrupted download is retried rather than loaded. Requires the new "parakeet" extra (sherpa-onnx).
✅ Deploy Preview for voca-linux canceled.
|
|
Thanks @AlexThunder1989. A maintainer will review it.
|
github-actions
Bot
added
documentation
|
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
|
| Filename | Overview |
|---|---|
| src/vocalinux/speech_recognition/recognition_manager.py | Adds Parakeet download, initialization, transcription, and lifecycle dispatch paths. |
| src/vocalinux/utils/parakeet_model_info.py | Defines the two model bundles, pinned revisions, expected sizes, paths, and deletion helpers. |
| src/vocalinux/ui/settings_dialog.py | Integrates Parakeet into engine selection, model prompts, metadata, and local-model management. |
| install.sh | Adds Parakeet dependency installation, initial configuration, and display metadata. |
| pyproject.toml | Adds the optional sherpa-onnx dependency extra. |
| uv.lock | Locks sherpa-onnx and records the new Parakeet extra. |
Flowchart
%%{init: {'theme': 'neutral'}}%%
flowchart LR
A[Settings or CLI selects Parakeet] --> B[Recognition manager]
B --> C{Model bundle present?}
C -- No --> D[Download four model files]
D --> E[Initialize sherpa-onnx recognizer]
C -- Yes --> E
E --> F[Decode 16 kHz PCM]
F --> G[Punctuated transcript]
Reviews (5): Last reviewed commit: "fix(asr): reject a truncated Parakeet do..." | Re-trigger Greptile
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
The parakeet branch duplicated a full default config.json. ConfigManager deep-merges DEFAULT_CONFIG on load, so every other key was redundant: a config holding only "engine": "parakeet" still resolves parakeet_model_size to v3-european along with the VAD, shortcut and text-injection defaults.
Writing only the engine key left config.json without a shortcuts section, which _migrate_shortcuts_config reads as a pre-push-to-talk upgrade and pins back to ctrl+ctrl/toggle - contradicting the installer's own "Hold Right Alt while you speak" message. Seed that section too. Point the config at parakeet with set_configured_engine rather than only on first create, so --engine=parakeet also switches a machine that already has a config.json, and give print_welcome_message a parakeet arm so the summary is not blank. The interactive picker overwrites SELECTED_ENGINE, so document the flag combination that reaches this branch.
Pin each bundle to a Hugging Face commit instead of resolve/main, so a file replaced upstream cannot change what an install downloads. This is the same approach whispercpp_model_info takes with its pinned revision. Remap each file's progress into its slice of the bundle. The four files were reported through the single-file downloader independently, so the fraction ran to 1.0 and restarted four times; it now advances once and names the file being fetched.
_stream_model_download only rejects an empty body, so a truncated but nonempty response was renamed into the model directory and then reported as installed, because is_model_downloaded checks existence. Record the byte size of each file at the pinned revision and check it before the rename, the same order whisper.cpp verifies its digest in. A short file is a truncated transfer rather than an upstream change, since the revision is pinned, so it raises and the temp file is removed.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Adds a
parakeetspeech recognition engine backed by sherpa-onnx, running NVIDIA Parakeet TDT 0.6B int8 models on CPU:Output is punctuated and capitalized without post-processing.
On an Intel Core Ultra 7 155H it decodes at about 12x realtime on CPU (RTF 0.08), roughly 8x faster than whisper.cpp
large-v3-turboover the same clips with its Vulkan GPU backend (RTF 0.59). Being a transducer it has no 30 s window to pad, so a short push-to-talk utterance returns in ~0.3 s rather than ~2.4 s. Measured over four EN/DE/ES/FR clips.utils/parakeet_model_info.pyholds the catalog plusget_model_path/is_model_downloaded/list_downloaded_models/delete_model, mirroringvosk_model_info.pyandwhispercpp_model_info.py._init_parakeet/_transcribe_with_parakeet/_download_parakeet_modelfollow the same shape as their whisper.cpp counterparts and reuse_stream_model_download, so progress, speed, ETA and cancellation behave as for the other engines.model_sizeis validated with a fallback as in_init_whispercpp.Registered in the engine dispatch (init, transcribe, reconfigure, resume-reinit), the Settings engine/model/language pickers, the model info card, both download prompts, the unused-downloads list and delete path,
install.shand the README engine list.Models download to
<models_dir>/parakeet/<model>/as temp files renamed on success;is_model_downloadedrequires all four bundle files, so an interrupted download is retried rather than half-loaded._init_parakeetdoes not take_model_lock, becausereconfigure()andreinitialize_after_resume()already hold that non-reentrant lock when they call it.Installed via the new
parakeetextra (sherpa-onnx);uv.lockregenerated.Related Issue
Type of Change
Checklist