fix: make Cohere Transcribe usable and stop buffered streaming from crawling

[Mr-Sunglasses]

Two reported problems, both reproduced against the real models on disk.

1. Cohere Transcribe never transcribed anything

Not a download or model problem — the model works fine once told a language:

cohere-transcribe-14-lang-int8  lang="en"    OK  infer=2304ms
  "Ask not what your country can do for you, ask what you can do for your country."
cohere-transcribe-14-lang-int8  lang="auto"  FAIL LanguageUnsupportedError

Its decoder refuses to run without an explicit language, and passing auto through
yields an empty transcript rather than a fallback:

Invalid language: 'auto'. Supported values: ar, de, el, en, es, fr, it, ja, ko, nl, pl, pt, vi, zh
lang='auto' en.wav: ''
lang=''     en.wav: ''   ("Please specify a language for Cohere Transcribe")

But every client defaults that field to auto — CreateSessionRequest.language,
the test-transcription query parameter, and the test panel's first <option>. So
the model failed with language_unsupported on any default-configured client.

Fix. CatalogModel.requires_explicit_language, the mirror of
detects_language_automatically, is set on Cohere and reported by /health so mobile
clients can drop "Detect language" too. syncTestLanguages() in the web UI reads it,
hides that option and selects the first supported language instead. It also greys out
languages outside a model's coverage when the model does not self-detect, which heads
off the same class of failure (picking Hindi on Cohere). An empty languages list
still means "unknown", so nothing is restricted there. The engine-side guard stays as
the backstop.

2. "Streaming 560 ms" was ~20× slower than the batch export, not faster

Same audio, same machine, same 8 threads:

model	inference
parakeet-unified-en-0.6b-int8 (non-streaming)	373 ms
parakeet-unified-en-0.6b-560ms-int8 (streaming)	7398 ms

The encoder's ONNX metadata explains it:

buffered_streaming     = 1
left_feature_frames    = 560
chunk_feature_frames   = 16
right_feature_frames   = 40
inputs: audio_signal, length      <- 2 inputs, no cache tensors

No cache state: this is a sliding-window export, not a cache-aware streaming
Conformer. Every step re-encodes 560+16+40 = 616 feature frames (~6.2 s of audio) to
emit 16 new frames (160 ms). Measured: 47 decode steps for 7.4 s of audio
(7.435 / 0.160 ≈ 46.5 ✔), ~152 ms each — about 290 s of audio pushed through the
encoder to transcribe 7.4 s. Thread scaling confirms it is compute, not overhead:

threads=1  decode=20.15s  429ms/chunk  rtf=2.71
threads=2  decode=12.17s  259ms/chunk  rtf=1.64
threads=4  decode= 8.33s  177ms/chunk  rtf=1.12
threads=8  decode= 7.14s  152ms/chunk  rtf=0.96

So "560 ms" is the latency before a word appears, not throughput. The model barely
keeps up with live audio and costs ~20× for a finished file, where there are no
partials to deliver early. There is no config knob for this.

Fix. Whole-file work goes to the batch export of the same weights when the user
has it. Models name a batch_twin_id; SherpaOnnxEngine holds one
_ResidentRecognizer per export, transcribe() uses the batch one and
create_stream() keeps the streaming one. When the twin is not installed both point
at the same holder, so behaviour is unchanged. Measured through the gateway:
7398 ms → 212 ms, identical transcript, streaming_supported still true.

Two bugs found reviewing that fix, and fixed

Resolving the twin once in __init__ was wrong, because nothing on the download or
delete paths rebuilds the engine — only select_model, engine configure, and
deleting the active model do.

• A twin downloaded later was never used. Reproduced live: install the twin while
  the streaming model is selected and transcription stays at RTF 1.005. Since the
  model description now tells users to install it, this would have silently done
  nothing.
• A twin deleted later broke transcription. forget_if_active(twin_id) returns
  false for a non-active model, so the engine kept pointing at a removed directory.
  After the next unload it raised an unwrapped RuntimeError — an HTTP 500 — while
  /health still reported ready:

  twin present  -> infer=279ms
  twin deleted  -> RuntimeError: Load model from .../twin/encoder.int8.onnx
  

The twin is now resolved on each use, and a twin that cannot load falls back to the
selection, so an opportunistic speed-up can never fail a request the selection could
have served. Verified live, with no engine rebuild between steps:

streaming, no twin:        infer=7781ms rtf=1.047
twin appears:              infer= 212ms rtf=0.029
twin removed:              infer=7406ms rtf=0.996

unload() and model_is_resident still reach a twin that has since been uninstalled,
so idle offload can reclaim its weights.

Also

Relabelled the streaming model from "Fast · streaming" to "Low latency · high CPU"
and rewrote both Unified descriptions — the non-streaming entry's had been copy-pasted
and described the streaming variant.

Checks

592 tests pass (28 new), plus ruff check, ruff format --check,
flake8 --select=WPS,E999, and mypy — the full CI quality job. New tests cover the
twin wiring through EngineManager.select_model, a twin appearing and disappearing
without a rebuild, load-failure fallback, residency/unload across both exports, the
/health flag, and an assertion that Cohere is the only model flagged.

One judgement call worth flagging: warmup() now loads both exports when a twin is
in play (~1.3 GB instead of 663 MB), since an engine that reports itself warm and still
pays a cold load on the first dictation has warmed the wrong one. Easy to change if
you would rather warm only the batch export.

🤖 Generated with Claude Code

[netlify]

✅ Deploy Preview for voca-gateway canceled.

Name	Link
🔨 Latest commit	bca8d92
🔍 Latest deploy log	https://app.netlify.com/projects/voca-gateway/deploys/6a9cdf9656fd050008ec7ead

[greptile-apps]

Mr-Sunglasses has reached the 50-credit limit for trial accounts. To continue receiving code reviews, upgrade your plan.