fix(vad): stop fragmenting live speech into sub-4s ASR requests, and filter hallucinations on every engine

[Hus-Mek]

Rebased onto devtest and scoped to the VAD fix (per review, 2026-08-29). This PR now
contains only the two VAD/segmentation changes below. The shared cross-provider
text_cleanup rewrite that was previously bundled here (hallucination filtering across
Whisper, Parakeet, and remote providers) has been removed from this branch and lands as
follow-up PR #747, so the VAD behavior can be validated without mixing in a
transcript-output policy change. The 0.3 confidence gate removed by #681 lives in
worker.rs/whisper_engine.rs and is untouched here, so this PR does not reintroduce it.

Updated after live validation (2026-09-01): the live path now uses 500 ms redemption
(the established Meetily Pro live policy); the batch paths keep 2000 ms. See defect 1.

What this fixes

Live VAD segmentation quality. Diagnosed from a real 26-minute Arabic meeting; both causes are
provider-agnostic, so local Whisper, Parakeet, and cloud providers are equally affected.

1. Live VAD redemption time was 400ms, fragmenting speech at mid-sentence breaths

pipeline.rs passed 400, while import.rs:58 and retranscription.rs:52 both define VAD_REDEMPTION_TIME_MS = 2000. Supporting evidence that 400 was not deliberate:

• the platform ternary was dead: if cfg!(target_os = "macos") { 400 } else { 400 }, under a comment claiming the two platforms differ
• vad.rs carried a comment asserting "Use full redemption_time from pipeline (2000ms)" - false for the live path
• vad.rs already contains a test named test_vad_400ms_vs_2000ms_segmentation whose docstring states that 400ms causes excessive fragmentation

Redemption time decides how long a silence must last before a speech segment closes, and every segment becomes its own ASR request. Measured on the real recording at 400ms:

metric	value
requests for 26 min of audio	322
median request length	3.49 s
requests under 3 s	42 %
requests that reached 25 s	1 of 321
boundaries in the 0.42-0.75 s range	152 of 321 (47 %)

That last row is the point: nearly half the segment boundaries were mid-sentence breaths that a longer redemption bridges.

Whisper is a fixed 30-second-window model. Below that it zero-pads the window and leans on its language-model prior, which was trained on web subtitles, so short clips come back as memorised boilerplate instead of speech. The VAD also pads every segment by 300 ms + 400 ms, a fixed cost that dominates short segments:

segment length	share that is padding silence	output is a single word
~1 s	60 %	81 %
~2 s	34 %	13 %
~3 s	22 %	6 %
~5 s	15 %	0 %
~7 s and over	9 %	0 %

Live-path policy (updated after review validation): an earlier revision of this PR aligned the live path to the batch value (2000 ms). Maintainer live testing with continuous system audio showed that is not viable for the live path: VAD kept one speech segment open for an extended period, no live transcript was emitted, and the accumulated-speech buffer crossed its warning threshold. The live path therefore now uses VAD_REDEMPTION_TIME_MS = 500, matching the established Meetily Pro live policy, while import.rs and retranscription.rs keep 2000 ms - batch has no latency requirement, so live and batch deliberately diverge. 500 ms addresses the fragmentation scope of this PR; it does not by itself guarantee bounded transcript delivery during continuous speech where no qualifying silence occurs - bounded live segments, forced-boundary correctness, and a maximum live-transcript-update interval are tracked separately in #756.

2. speech_start_sample double-counted the session position

The buggy live path was:

self.speech_start_sample = self.processed_samples + (timestamp_ms * 16000 / 1000);

Both operands are already session-absolute. processed_samples increments forever, and silero computes SpeechStart.timestamp_ms as processed_duration().saturating_sub(pre_speech_pad) - verified in silero-rs/src/lib.rs:302-305, emitted at 321-324. Summing them puts the start position at roughly 2x the truth, and the error grows with how late in the session the utterance begins.

The only reader is the force-end branch in flush(), so it escaped once per recording, on the final segment. On a 1539.7 s recording that segment was stored with audio_start_time = 3083.1 s (ratio 2.0007) and sorted to the end of the transcript. Every other segment tracked wall-clock at ratio ~1.01, which is why it looked like a one-off rather than drift. Relevant to #399.

devtest has since applied an equivalent literal fix to the SpeechStart path; this branch keeps the same correction expressed against the VAD_SAMPLE_RATE module constant, and retains the regression coverage below.

What changed

• pipeline.rs - named VAD_REDEMPTION_TIME_MS = 500 constant (live policy, per Meetily Pro), documented against the batch paths' 2000 ms; dead platform ternary removed; test_live_vad_redemption_matches_pro_policy locks the value
• vad.rs - session-absolute timestamp fix; VAD_SAMPLE_RATE promoted to a module constant; the false comment corrected; two regression tests added
• import.rs, retranscription.rs - unchanged at 2000 ms; stale doc comments (which described the live path as 400 ms) updated

Verification

Focused VAD tests, on the current branch:

$ cargo test --lib audio::vad

running 7 tests
test audio::vad::tests::test_flush_segment_timestamps_stay_within_audio_duration ... ok
test audio::vad::tests::test_speech_start_sample_never_exceeds_processed_samples ... ok
test audio::vad::tests::test_vad_continuous_processor_state_across_chunks ... ok
test audio::vad::tests::test_vad_cancellation ... ok
test audio::vad::tests::test_vad_large_file_progress ... ok
test audio::vad::tests::test_vad_chunked_vs_single_processing ... ok
test audio::vad::tests::test_vad_400ms_vs_2000ms_segmentation ... ok

test result: ok. 7 passed; 0 failed; 0 ignored; 0 measured; 208 filtered out; finished in 6.30s

$ cargo test --lib audio::pipeline

running 1 test
test audio::pipeline::tests::test_live_vad_redemption_matches_pro_policy ... ok

test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 214 filtered out; finished in 0.01s

The two regression tests requested in review:

• test_flush_segment_timestamps_stay_within_audio_duration - a final unfinished utterance satisfies start <= end <= input duration.
• test_speech_start_sample_never_exceeds_processed_samples - the VAD start position never exceeds processed audio. Asserted directly rather than via silero's segmentation of synthetic audio, because the force-end path only runs when recording stops mid-utterance and the error is only large enough to violate the invariant when speech begins late in the session. The fixture is therefore leading-silence-then-speech; with the bug present it reports start=41.67s vs processed=22.98s.

Both were verified in both directions - each bug reintroduced to confirm the test fails, then the fix restored to confirm it passes.

Full suite (cargo test --lib): 212 passed; the only failure is audio::device_detection::tests::test_calculate_buffer_timeout_bluetooth, which fails identically on unmodified devtest (confirmed by stashing). Unrelated to this PR.

Deliberately not included

• The overlapping segment pairs are not a bug. They are the arithmetic consequence of 700 ms of VAD padding against the redemption window; maximum overlap across all of them was exactly 0.28 s.
• A minimum-duration floor before dispatch. Once the timestamp fix lands, the shortest real segment measured was 0.97 s, so a 1 s floor is close to a no-op and risks dropping legitimate one-word replies.
• Bounded live segments under continuous speech. 500 ms redemption still requires a qualifying silence to close a segment; forced boundaries and a maximum live-transcript-update interval are tracked in fix(vad): bound live segments to guarantee transcript updates #756.
• vad.rs::resample_to_16k. Its filter size is sample_rate / (0.4 * sample_rate) - the rate algebraically cancels to a fixed 5-tap moving average with no real anti-aliasing at 3:1, which reads like a serious bug. I ported that exact algorithm and re-ran the full meeting against ffmpeg's polyphase resampler with chunking and language held constant: 2229 words vs 2207, median avg_logprob -0.293 vs -0.334 (the naive version scored marginally better), and both recovered the proper nouns. Whisper's mel front-end weights energy below ~4 kHz heavily, so aliasing folded in above 8 kHz barely reaches the features. Worth fixing for hygiene, but it is not a quality fix and I did not want to bundle it here.
• The cross-provider text_cleanup rewrite (hallucination filtering across all engines, repetition ratio measured on original text, multi-word boilerplate denylist). Removed from this branch; now under review separately as fix(transcription): shared cross-provider hallucination filtering (follow-up to #679) #747.

[Hus-Mek]

Pushed d5abf16 — a self-correction, and it interacts with #675, so flagging it rather than letting a reviewer find it.

What was wrong

The hallucination filter I added in dbb4e24 included a standalone-filler denylist that dropped "thank you", "okay", "thanks", "you", "bye" and Arabic شكرا (all orthographic variants) when they constituted a whole segment.

Whisper genuinely does hallucinate those on near-silent input — that is what made the list tempting, and the meeting I diagnosed had 14 standalone شكرا segments in 26 minutes. But nothing in the text distinguishes the two cases. شكرا decoded from 300 ms of silence and شكرا decoded from someone actually saying it are byte-identical.

So the list traded a cosmetic problem (a stray "thanks" in the transcript) for a semantic one (a participant's answer silently missing). And #675 is that exact failure already reported — "OK", "Yes", "Thanks" dropped by the byte-length confidence heuristic. My filter would have made a filed bug worse from a second, independent direction. Bad trade, now reverted.

What replaces it

Short-clip hallucination is handled where the evidence actually lives — the provider layer, gating on Whisper's own avg_logprob / compression_ratio from verbose_json. Those are acoustic-ish signals; the text is not. (That part is on the Groq provider in #610, since groq_provider.rs does not exist on devtest.)

Only unambiguous multi-word boilerplate is still filtered by text — اشتركوا في القناة, subtitler credits like ترجمة نانسي قنقر, thanks for watching, like and subscribe. No meeting contains those phrases, so there is no genuine reading to lose.

Degenerate repetition is still caught, now language-agnostically: the repetition ratio is measured on the original text rather than after collapsing. Measuring after collapsing hid it — a collapsed run leaves one word, and one word has a repetition ratio of zero. شكرا شكرا شكرا شكرا شكرا is still discarded, and the code no longer needs to know what the word means to do it.

Tests

The two inverted tests are replaced by keeps_short_acknowledgements_issue_675, asserting that OK, Yes, No, Sure, Got it, Thanks, Okay., Bye plus شكرا, شكراً, شكرا لك, نعم, طيب, لا and a tatweel-stretched شكـــرا all survive cleaning unchanged. 14 tests in the module, all passing.

Full suite unchanged at 202 passed, 1 failed — audio::device_detection::tests::test_calculate_buffer_timeout_bluetooth, which fails identically on unmodified devtest (verified by stashing). Not from this PR.

Note on #675

This PR no longer makes #675 worse, but it does not fix it either — the byte-length confidence heuristic in WhisperEngine::transcribe_audio_with_confidence and the 0.3 threshold in the worker are both untouched here. Happy to take that on separately if it is not already in progress; the proposed fix in #675 (emit on non-empty content, keep the confidence value as metadata only) reads correct to me and would compose cleanly with this PR.

[safvanatzack]

Thanks @Hus-Mek for the thoughtful work on this PR. The investigation into live VAD fragmentation, the 2-second redemption alignment, and the final-segment timestamp path is valuable; this is the strongest current OSS contribution in this area.

Our plan is to address the transcript-loss cluster in focused, independently reviewable pieces:

1. Preserve short valid transcripts: #681 already removes the 0.3 transcript-confidence gate on devtest.
2. Stabilize live VAD segmentation: use this PR for the 2-second redemption/hangover behavior so normal conversational pauses do not split one utterance into short ASR requests.
3. Correct end-of-recording VAD behavior: keep the absolute start-timestamp fix so flush() cannot emit a trailing segment beyond the audio duration—relevant to #399.
4. Review transcript cleanup independently: shared cleanup across Whisper, Parakeet, and remote providers changes output policy and should have its own focused PR.

Could you please rebase #679 onto the latest devtest and resolve the merge conflict?

For this rebased PR, please keep only the VAD-focused work:

• retain live VAD_REDEMPTION_TIME_MS = 2_000, aligned with import/retranscription;
• retain the absolute VAD start-timestamp correction;
• retain/add regression coverage proving:
  • a final unfinished utterance has start <= end <= input duration;
  • the VAD start position never exceeds processed audio.

Please move the shared cross-provider text_cleanup rewrite into a separate follow-up PR. It is useful work, but separating it lets us validate the VAD fix without mixing transcript-output policy changes across providers.

Please ensure the rebased branch integrates cleanly with #681 and does not reintroduce the old 0.3 confidence gate. Once rebased, run the focused Rust VAD tests and add the command/output to the PR description.

Thank you again. Splitting this work will make it substantially easier to review, test, and land the VAD improvements safely.

[Hus-Mek]

Thanks @safvanatzack — the four-way split makes sense, and scoping this PR to the VAD fix is the right call. I'll restructure it exactly as you've laid out.

Plan for the rebased #679:

• Rebase onto latest devtest and resolve the conflict. I can see Fix: stop dropping transcripts below confidence threshold (#675) #681 already landed there (2026-08-27), so the 0.3 confidence gate is gone on the base — I'll confirm the rebase doesn't drag the old gate back in from my branch's history.
• Keep only the VAD-focused work: live VAD_REDEMPTION_TIME_MS = 2_000 aligned with the import/retranscription path, and the absolute VAD start-timestamp correction so flush() can't emit a segment past the audio duration.
• Keep/add the regression coverage: a final unfinished utterance satisfies start <= end <= input duration, and the VAD start position never exceeds processed audio.
• Run the focused Rust VAD tests and add the command plus output to the PR description.

The shared cross-provider text_cleanup rewrite (repetition ratio on original text, multi-word boilerplate denylist) will move to its own follow-up PR so the output-policy change is reviewed on its own, independent of the VAD behavior. I'll open that once this one is clean.

[Hus-Mek]

Done — #679 is now rebased onto devtest and scoped to the VAD fix, and the text_cleanup work has moved to #747 as agreed.

• fix(vad): stop fragmenting live speech into sub-4s ASR requests, and filter hallucinations on every engine #679 (this PR): VAD-only. VAD_REDEMPTION_TIME_MS = 2000 aligned with the import/retranscription paths, the session-absolute speech_start_sample correction, and both regression tests. cargo test --lib audio::vad → 7 passed; command and output are in the updated description. The 0.3 gate removed by Fix: stop dropping transcripts below confidence threshold (#675) #681 is untouched.
• fix(transcription): shared cross-provider hallucination filtering (follow-up to #679) #747 (follow-up): the shared cross-provider hallucination filtering, reviewable on its own. It does not reintroduce the 0.3 gate.

Both branches integrate cleanly with the current devtest.

[safvanatzack]

@Hus-Mek I manually tested the rebased branch and found a live-transcription issue with the current 2000 ms redemption setting.

With continuous system audio from a podcast, VAD kept one speech segment open and no live transcript segment was emitted for an extended period. The buffer grew past the 62.5 s warning threshold and began producing repeated accumulation warnings.

For comparison only, I locally changed the live value to 800 ms and repeated the same test. It emitted transcript segments, and the resulting transcript had no material missing content compared with the source video. However, the segments were still large and the accumulated-buffer warning still occurred.

The timestamp correction and its regression coverage still look correct. I am sharing this manual-test finding now; we will complete internal testing and follow up separately on the appropriate live VAD policy/value.

[safvanatzack]

Thank you, @Hus-Mek, for following through on the rebase and for keeping this PR carefully focused. The VAD investigation, timestamp correction, and regression coverage are all appreciated.

One update from our live validation: 2000 ms is appropriate for import and retranscription, but it is not suitable for the live path. With continuous system audio, it kept a VAD segment open for an extended period, withheld live transcript emission, and crossed the accumulated-speech-buffer warning threshold.

Could you please make the following final adjustments to this PR?

• Set the live VAD_REDEMPTION_TIME_MS to 500 ms, matching the established Meetily Pro live policy.
• Keep import and retranscription at 2000 ms, since batch and live paths have different latency requirements.
• Retain the session-absolute speech_start_sample correction and its regression coverage for final-segment timestamp bounds.
• Keep the cross-provider text-cleanup work separate, as already agreed.
• Update the PR description so it no longer presents 2000 ms as the live-path policy.

The 500 ms change addresses the current live-fragmentation scope. It does not, by itself, guarantee bounded transcript delivery during continuous speech, where VAD may remain active without a qualifying silence. That work is now tracked separately in #756, covering bounded live segments, forced-boundary correctness, and a maximum live-transcript-update interval.

Thanks again for the thoughtful work and for working with us to keep these changes safe and independently reviewable.

[Hus-Mek]

Done — pushed 0ceee52 with the requested adjustments:

• Live path is now VAD_REDEMPTION_TIME_MS = 500 in pipeline.rs, matching the Meetily Pro live policy. The constant's doc comment records why live and batch deliberately diverge (continuous system audio keeping a segment open, withheld emission, the accumulated-buffer warning) and points to fix(vad): bound live segments to guarantee transcript updates #756 for bounded live delivery.
• Import and retranscription stay at 2000 ms. Their doc comments also had a stale "live pipeline (400ms)" reference, now corrected — no value change on either path.
• Added test_live_vad_redemption_matches_pro_policy locking the live value, alongside the existing test_vad_redemption_time_constant that locks batch at 2000.
• The session-absolute speech_start_sample correction and both timestamp regression tests are unchanged.
• The PR description no longer presents 2000 ms as the live-path policy; it now documents the 500/2000 split, your live-validation finding, and the fix(vad): bound live segments to guarantee transcript updates #756 follow-up scope.

Verification on the current branch: cargo test --lib audio::vad → 7 passed; cargo test --lib audio::pipeline → 1 passed; full cargo test --lib → 212 passed with only the pre-existing test_calculate_buffer_timeout_bluetooth failure that also fails on unmodified devtest. Output is in the updated description.

Agreed on the #756 framing — 500 ms fixes the fragmentation scope here, but a qualifying silence is still required to close a segment, so continuous speech needs the forced-boundary/max-update-interval work tracked there. Happy to pick that up if useful.

[safvanatzack]

Thank you, @Hus-Mek — I rechecked 0ceee52. The 500 ms live / 2000 ms batch split is now correct, and the focused tests pass.

One timestamp issue remains: flush() counts zero-padding in processed_samples. The existing 23-second fixture therefore reports an end of 23,010 ms. The test passes because it checks start <= duration and end >= start, but not end <= duration.

Adding the missing assertion reproduces the failure:

Segment 0 ends at 23010ms, beyond the 23000ms of audio supplied

Please preserve the real post-resampling endpoint while retaining padded VAD inference, then assert:

seg.end_timestamp_ms <= audio_duration_ms

Also, please adjust the two comments implying 500 ms bounds continuous-audio delivery; that remains #756.

The previous live-redemption blocker is resolved. These are the remaining focused corrections.