You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
A local speech transcription pipeline for English, Hindi, and Hinglish audio.
4
-
The orchestrator owns chunking, diarization, repair, and exports; ASR inference is **externalized** behind an OpenAI-compatible `/v1/audio/transcriptions` endpoint that you choose (vLLM, hosted APIs including OpenAI and Open Router API, faster-whisper wrapper, Parakeet wrapper, etc.).
3
+
**The only Speech-To-Text pipeline you need.**
4
+
5
+
A provider-agnostic speech-to-text pipeline that plugs into **any OpenAI-compatible ASR endpoint** — OpenAI, OpenRouter, vLLM, faster-whisper, or the bundled Qwen worker — and returns rich, timestamped transcripts with speaker labels and validated metadata. Resilient STT handles everything around inference: audio preprocessing, voice-activity detection, intelligent chunking, pyannote diarization, and LLM-powered transcript repair. Tested on English, Hindi, and Hinglish but it is designed to support all the languages the ASR model of your choice supports.
6
+
7
+
## Features
8
+
9
+
-**Any ASR provider** — Point at any service that implements `POST /v1/audio/transcriptions` (or OpenRouter STT); swap models with flags, not code changes.
10
+
-**Resilient ASR discovery** — Auto-detects local workers and hosted APIs, retries failed requests, and starts a bundled Qwen fallback when nothing else is reachable.
11
+
-**Audio preprocessing** — ffmpeg normalization to mono 16 kHz WAV with optional enhancement before transcription.
12
+
-**Voice activity detection** — Silero, webrtcvad, or RMS fallback to skip silence and focus compute on speech.
13
+
-**Smart chunking & stitching** — Fixed or pause-aligned chunks for long files, with global word and segment timestamps stitched back together.
14
+
-**Speaker diarization** — pyannote on the full audio, with word-level speaker assignment (IoU + segment fallback).
15
+
-**LLM transcript repair** — Optional two-pass repair via any OpenAI-compatible chat endpoint, with validation to preserve timing and mixed-script Hinglish.
16
+
-**Rich exports** — JSON, SRT, and VTT with segments, words, speakers, and pipeline metadata.
17
+
-**Lightweight orchestrator** — No ASR model weights in the core package; inference stays in the microservice you choose.
18
+
-**CLI and Python API** — `resilient-stt` from the terminal, or `pipeline.run(JobConfig)` for programmatic use.
5
19
6
20
> **Experimental:** This project is under active development and is **not production-ready**. Expect breaking changes, incomplete features, and behavior that may shift between releases. Use for evaluation and prototyping only.
|[qwen_transformers_service](workers/qwen_transformers_service/README.md)|`http://127.0.0.1:8002/v1`|`Qwen/Qwen3-ASR-0.6B`|**Bundled** — auto-started when no other ASR is reachable |
129
+
|[qwen_transformers_service](src/resilient_stt/workers/qwen_transformers_service/README.md)|`http://127.0.0.1:8002/v1`|`Qwen/Qwen3-ASR-0.6B`|**Bundled** — auto-started when no other ASR is reachable |
91
130
|[qwen_vllm_service](workers/qwen_vllm_service/README.md)|`http://127.0.0.1:8001/v1`|`Qwen/Qwen3-ASR-1.7B`|**Bundled bootstrap** — Linux + NVIDIA GPU |
92
131
| Custom (faster-whisper, vLLM, hosted proxy, etc.) | Your URL | Your model id |**Supported** — implement or deploy separately |
Today, alignment runs only when `--align` is set or ASR returned weak
128
167
timestamps; the default aligner is a no-op pass-through.
129
168
130
-
## Install (uv)
169
+
## Install (PyPI)
170
+
171
+
Prerequisites: **Python 3.11 or 3.12**, **ffmpeg** on PATH.
172
+
173
+
```bash
174
+
# Minimal orchestrator (webrtcvad VAD; ASR via API or bundled qwen worker)
175
+
pip install resilient-stt
176
+
177
+
# Recommended on Apple Silicon / Linux (Silero VAD + pyannote diarization + torch)
178
+
pip install "resilient-stt[full]"
179
+
180
+
# Contributors
181
+
pip install "resilient-stt[full,dev]"
182
+
```
183
+
184
+
Verify the CLI:
185
+
186
+
```bash
187
+
resilient-stt --help
188
+
```
189
+
190
+
### Usage after `pip install`
191
+
192
+
Run from any directory (creates `data/work/` under the current working dir unless you pass `--work-root`):
193
+
194
+
```bash
195
+
resilient-stt \
196
+
--audio /path/to/audio.wav \
197
+
--output /path/to/output-dir \
198
+
--language hi
199
+
```
200
+
201
+
With diarization and Silero VAD you need the `[full]` extra and usually `HF_TOKEN` in `.env` (see [Environment variables](#environment-variables)). Quick smoke test without pyannote:
202
+
203
+
```bash
204
+
resilient-stt \
205
+
--audio /path/to/audio.wav \
206
+
--output /path/to/output-dir \
207
+
--language hi \
208
+
--skip-diarization \
209
+
--repair false
210
+
```
211
+
212
+
Hosted ASR (no local qwen worker) — set `OPENAI_API_KEY` or `OPENROUTER_API_KEY` in `.env`:
213
+
214
+
```bash
215
+
resilient-stt \
216
+
--audio /path/to/audio.wav \
217
+
--output /path/to/output-dir \
218
+
--no-asr-fallback \
219
+
--skip-diarization
220
+
```
221
+
222
+
**Bundled qwen-asr worker:** On first auto-start, the CLI creates an isolated venv at
223
+
`~/.cache/resilient-stt/qwen-transformers-worker/.venv` (requires network to download
224
+
`qwen-asr` and model weights). If inference fails on Apple Silicon, start the worker
225
+
manually with `python scripts/bootstrap_qwen_asr_fallback.py --no-aligner` (from a git
226
+
checkout) or see the [bundled worker README](src/resilient_stt/workers/qwen_transformers_service/README.md).
227
+
228
+
Platform notes for `[full]` match [Platform notes (torch / diarization)](#platform-notes-torch--diarization) below (Intel Mac: base install + `--skip-diarization`).
229
+
230
+
## Install from source (uv)
131
231
132
232
Prerequisites: [uv](https://docs.astral.sh/uv/), **ffmpeg** on PATH. ASR is optional to
133
233
configure manually — see [ASR auto-detection](#asr-auto-detection) below.
From a **git checkout**, prefix commands with `uv run` (or activate `.venv` and use `resilient-stt` directly). After **`pip install`**, use `resilient-stt` only.
296
+
195
297
Minimal run (no ASR setup — auto-starts local **qwen-asr** on CPU/MPS when needed):
196
298
197
299
```bash
198
-
uv run python -m orchestrator.main \
300
+
uv run resilient-stt \
199
301
--audio data/input/meeting.mp3 \
200
302
--output data/output/meeting \
201
303
--language hi
202
304
```
203
305
204
-
With an external ASR service (vLLM, hosted API, etc.):
306
+
With an external ASR service (vLLM, hosted API, etc.) — the endpoint must already be running and respond to `GET {base}/v1/models`:
205
307
206
308
```bash
207
-
uv run python -m orchestrator.main \
309
+
uv run resilient-stt \
208
310
--audio data/input/meeting.mp3 \
209
311
--asr-endpoint http://localhost:8001/v1 \
210
312
--model Qwen/Qwen3-ASR-1.7B \
@@ -218,14 +320,14 @@ auto-detect (no local ASR on `:8001`/`:8002`) or point at the API explicitly:
218
320
219
321
```bash
220
322
# Auto-detect when no local ASR is running (--no-asr-fallback avoids starting qwen-asr)
221
-
uv run python -m orchestrator.main \
323
+
uv run resilient-stt \
222
324
--audio data/input/speech.wav \
223
325
--output data/output/openai \
224
326
--no-asr-fallback \
225
327
--skip-diarization
226
328
227
329
# Explicit endpoint
228
-
uv run python -m orchestrator.main \
330
+
uv run resilient-stt \
229
331
--audio data/input/speech.wav \
230
332
--output data/output/openai \
231
333
--asr-endpoint https://api.openai.com/v1 \
@@ -237,7 +339,7 @@ uv run python -m orchestrator.main \
237
339
the key via `ASR_API_KEY`:
238
340
239
341
```bash
240
-
uv run python -m orchestrator.main \
342
+
uv run resilient-stt \
241
343
--audio data/input/speech.wav \
242
344
--output data/output/openrouter/chirp \
243
345
--asr-endpoint https://openrouter.ai/api/v1 \
@@ -257,10 +359,12 @@ When `--asr-endpoint` is omitted (and `ASR_BASE_URL` / `ASR_ENDPOINT` are unset)
257
359
2. Probe an existing **qwen-asr** worker at `http://127.0.0.1:8002/v1`
258
360
3.**OpenRouter** when `OPENROUTER_API_KEY` is set (no `--model`)
259
361
4.**OpenAI** when `OPENAI_API_KEY` is set (no `--model`)
260
-
5. Otherwise **start** the local qwen-asr fallback ([workers/qwen_transformers_service/README.md](workers/qwen_transformers_service/README.md)) — slow on CPU/MPS but needs no NVIDIA GPU
362
+
5. Otherwise **start** the local qwen-asr fallback ([bundled worker](src/resilient_stt/workers/qwen_transformers_service/README.md)) — slow on CPU/MPS but needs no NVIDIA GPU
261
363
262
364
Use `--no-asr-fallback` to require an explicit or already-running ASR service.
263
365
366
+
If you pass `--asr-endpoint` explicitly, that URL must respond to `GET …/v1/models` before the pipeline runs (otherwise the CLI exits with “endpoint configured but unreachable”).
367
+
264
368
Useful flags:
265
369
266
370
@@ -340,13 +444,29 @@ Final exports land under `--output`: `transcript.json`, `transcript.srt`,
340
444
341
445
## Tests
342
446
447
+
From source:
448
+
343
449
```bash
450
+
uv sync --extra dev
344
451
uv run pytest
345
452
```
346
453
454
+
After `pip install "resilient-stt[dev]"` from a checkout (with `src/` on `PYTHONPATH`) or when developing in the repo, the same `pytest` command applies if `src` is configured in `pyproject.toml` (`pythonpath = ["src"]`).
455
+
347
456
The included tests use synthetic fixtures and mocked HTTP responses; they do
348
457
not invoke ffmpeg, pyannote, or any LLM.
349
458
459
+
## Publishing (maintainers)
460
+
461
+
1.**Commits on `main`** — use [Conventional Commits](https://www.conventionalcommits.org/) in PR titles or squash messages (`feat:`, `fix:`, `chore:`, etc.).
462
+
2.**release-please** (`.github/workflows/release-please.yml`) — opens/updates a **Release PR** that bumps `pyproject.toml`, `CHANGELOG.md`, and `.release-please-manifest.json`.
463
+
3.**Ship** — merge the Release PR; release-please creates GitHub Release + tag `vX.Y.Z`.
464
+
4.**PyPI** (`.github/workflows/publish.yml`) — runs on that tag when it matches `version` in `pyproject.toml`.
465
+
466
+
One-time: configure a [PyPI trusted publisher](https://docs.pypi.org/trusted-publishers/) for workflow `publish.yml` on repo `gitcommitshow/resilient-stt`. In the org/repo settings, allow GitHub Actions to create and approve pull requests if Release PRs do not appear.
467
+
468
+
See [CHANGELOG.md](CHANGELOG.md) for release history.
469
+
350
470
## License
351
471
352
472
This project is licensed under the [GNU General Public License v3.0 or
0 commit comments