Follow-up to #465: tdrz is already exposed in pywhispercpp, so the missing half is an output surface

[kacperpaczos]

#465 says this, about leaving TinyDiarize out of the model picker:

We intentionally do not expose TinyDiarize in this picker. It is a speaker-turn diarization model, not a plain dictation model, and it needs diarization-specific runtime/UI support before it would be useful in Vocalinux's text-injection flow.

I went to look at what the runtime half would involve, and it appears to be done already: pywhispercpp exposes tdrz — it shows up in pywhispercpp/model.py, pywhispercpp/constants.py and src/main.cpp, and whisper.cpp has tdrz_enable in whisper.h with the cli, stream and server examples using it. So reaching TinyDiarize from here looks like a catalog entry and a parameter rather than a binding change. Worth contrasting with Parakeet in #527, which is genuinely blocked on absadiki/pywhispercpp#172.

That leaves the other half of the sentence, and I think it is the real question — and it is not about diarization at all. A flow that types into whatever window has focus has nowhere to put a speaker label. [SPEAKER_TURN] markers injected into someone's document are noise. The blocker is not the model; it is that keystrokes are the only output surface.

What that surface would be

A recording mode: capture, transcribe, produce a document rather than keystrokes. That is a different product from dictation and I am not suggesting it be bolted onto the dictation loop.

It splits into four parts, only one of which is diarization:

1. Capture — Audio capture goes through PortAudio only: no PipeWire path, no system-audio source #751. _is_virtual_device() filters monitors out today, so system audio is a deny-list away for a mode where the user explicitly asks for it. Capturing one application's stream is a bigger job: PortAudio sees ALSA and Pulse devices, not PipeWire nodes.
2. Long-form transcription — the dictation path has a 2 s silence_timeout and nothing for an hour of audio. VocaGateway already does this part, with FFmpeg normalisation and several engines.
3. Speaker attribution — the ladder below.
4. Somewhere for the result to live — a transcript view and an export. This part does not exist anywhere in the family.

The speaker ladder, cheapest first

Two tracks, no model at all. Recording the monitor and the microphone separately means "me" and "them" are separated by construction. For a one-on-one call that is the entire feature — no model, no new dependency. Worth trying before anything else.

TinyDiarize. Speaker turns rather than identities, English only, since it is a fine-tune of small.en. Reachable today per above. It tells you the speaker changed; it will not tell you that the same person spoke twice.

Full diarization. Speaker identities across a recording, multilingual: segmentation, speaker embeddings, clustering. sherpa-onnx implements it — offline-speaker-diarization.py in its Python API examples, using pyannote-segmentation-3.0 plus a 3D-Speaker embedding model, both plain .onnx files from its releases. Two notes. onnxruntime is already an optional dependency here for Silero VAD, but the awkward part is the window stitching that sherpa implements in C++, not the models themselves. And this is the same library #724 is about, so the two would arrive together.

What I am actually asking

Is a recording surface something Vocalinux should have, or does it belong elsewhere — in the gateway, or in something separate that stands on it? Both answers are reasonable, and the answer decides whether any of the above is worth doing.

I have not measured TinyDiarize output on real recordings. Happy to run it and report what the markers look like in practice, since that tests the UI question specifically and costs nothing.

[github-actions]

Thanks @kacperpaczos. A maintainer will look into it.

---

Meanwhile, connect with us: