fix(transcription): shared cross-provider hallucination filtering (follow-up to #679) - #747
Open
Hus-Mek wants to merge 2 commits into
Open
Conversation
…ocal Whisper
`clean_repetitive_text` and `is_meaningless_output` were private associated
functions on `WhisperEngine`, reachable only from the local-Whisper path. The
Parakeet arm and the trait-based provider arm in `worker.rs` both stored
`result.text.trim()` verbatim, so Whisper's subtitle-corpus artifacts were
stripped on one engine and persisted on the others.
The denylist was also entirely English ("thank you for watching", "like and
subscribe", "applause"). Whisper emits the same class of artifact in whichever
language it decided it was hearing, so the filter silently applied to English
meetings and to nothing else. A real Arabic meeting stored these unfiltered:
اشتركوا في القناة "subscribe to the channel"
ترجمة نانسي قنقر a subtitler credit
شكرا 14 standalone occurrences, the Arabic analogue of
Whisper's English "Thank you." attractor on near-silent input
Extracts all five routines into `audio/transcription/text_cleanup.rs` and calls
it from all three engine arms. Adds the Arabic patterns alongside the English
ones, grouped by language so further languages are additive.
Behaviour changes beyond relocation:
- Filler such as "شكرا" is matched against the whole trimmed segment rather than
as a substring, so it is dropped when it IS the segment but kept inside a real
sentence. Trailing punctuation and Arabic tatweel are normalised first so
orthographic variants collapse to one form.
- `is_meaningless_output` is re-checked after repetition collapsing, because
"شكرا شكرا شكرا" only becomes recognisable as standalone filler once the
repetition has been removed. Caught by a test.
- `remove_word_repetitions` had identical `if`/`else` bodies (`repeat_count` is 1
in the else branch); collapsed to the single shared path.
14 unit tests cover both directions: the artifacts that must be removed, and
ordinary Arabic, code-switched Arabic/English and short genuine replies that must
survive.
…sue Zackriya-Solutions#675) The previous commit added a standalone-filler denylist ("thank you", "okay", "thanks", Arabic "شكرا" and variants) that dropped those words when they made up a whole segment. That was wrong, and it would have compounded a known bug. Whisper genuinely does hallucinate exactly those words on near-silent input, which is what made the list tempting. But they are also completely ordinary meeting speech, and NOTHING IN THE TEXT DISTINGUISHES THE TWO — "شكرا" decoded from 300ms of silence and "شكرا" decoded from someone saying it are byte-identical. So the denylist traded a cosmetic problem (a stray "thanks" in the transcript) for a semantic one (a participant's answer silently missing). Issue Zackriya-Solutions#675 is precisely that failure reported for "OK" / "Yes" / "Thanks" via a different mechanism, so adding a text-only filler list would have made a filed bug worse from a second direction. Short-clip hallucination belongs where the evidence actually is: the provider layer, gating on Whisper's own `avg_logprob` / `compression_ratio`. Only unambiguous multi-word boilerplate — "subscribe to the channel", subtitler credits, "thanks for watching" — is filtered by text, because no meeting contains those phrases. Degenerate repetition is still caught, now language-agnostically: the repetition ratio is measured on the ORIGINAL text rather than after collapsing. Collapsing first hid it, because a collapsed run leaves one word and one word has a repetition ratio of zero. "شكرا شكرا شكرا شكرا شكرا" is still discarded, without the code needing to know what the word means. Replaces the two inverted tests with `keeps_short_acknowledgements_issue_675`, covering OK / Yes / No / Sure / Got it / Thanks / Bye and the Arabic and tatweel-stretched equivalents. 14 tests pass.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follow-up to #679, split out per review so the VAD fix and this transcript-output policy change are reviewed independently.
What this fixes
Hallucination filtering only ran on local Whisper, and only knew English.
clean_repetitive_textandis_meaningless_outputwere private associated functions onWhisperEngine. The Parakeet arm and the provider (cloud) arm inworker.rsboth storedresult.text.trim()verbatim, so they got no filtering at all. The denylist was also entirely English ("thank you for watching","like and subscribe","applause").Net effect: an English meeting got cleaned and a non-English meeting did not. A real 26-minute Arabic recording stored these unfiltered:
اشتركوا في القناةترجمة نانسي قنقرشكراx14What changed
audio/transcription/text_cleanup.rs- the cleanup routines extracted into one module and called from all three engine arms (local Whisper, Parakeet, remote providers), with Arabic patterns added alongside the English ones and grouped by language so further languages are additive.whisper_engine.rs- the now-shared code removed; delegates to the new module. (This is why the diff is move-heavy: ~155 lines relocated out of here.)worker.rs- the Parakeet and provider arms now calltext_cleanup::clean_transcript_textinstead of storing raw text. No change to the confidence gate - the0.3threshold removed by Fix: stop dropping transcripts below confidence threshold (#675) #681 is not touched or reintroduced here.transcription/mod.rs- registers the new module.Behaviour, and the #675 correction
An earlier version of this work (in the original #679) included a standalone-filler denylist that dropped
"thank you","okay","thanks","you","bye"and Arabicشكراwhen they were a whole segment. That was reverted before this split: nothing in the text distinguishesشكراdecoded from 300 ms of silence fromشكراa participant actually said, so the list traded a cosmetic problem for a semantic one - and #675 is that exact failure already reported. Short-clip hallucination belongs in the provider layer gating on Whisper's ownavg_logprob/compression_ratio, not in a text denylist.What this PR filters by text is only unambiguous multi-word boilerplate (
اشتركوا في القناة, subtitler credits,thanks for watching,like and subscribe) - phrases no meeting contains, so there is no genuine reading to lose.Degenerate repetition is still caught language-agnostically: the repetition ratio is measured on the original text rather than after collapsing (measuring after collapsing hid it - a collapsed run leaves one word, ratio zero).
شكرا شكرا شكرا شكرا شكراis still discarded without the code needing to know what the word means.Verification
keeps_short_acknowledgements_issue_675asserts thatOK,Yes,No,Sure,Got it,Thanks,Okay.,Byeplusشكرا,شكراً,شكرا لك,نعم,طيب,لاand a tatweel-stretchedشكـــراall survive cleaning unchanged - i.e. this PR does not make #675 worse from a second direction.The 14 tests cover both directions deliberately: the artifacts that must be removed, and ordinary Arabic, code-switched Arabic/English, and short genuine replies that must survive.
Note on diff size
The diff is ~506 changed lines, but ~155 of the deletions are code relocated verbatim out of
whisper_engine.rsinto the new module; net new logic is small. Happy to split further if you'd prefer the extraction and the cross-provider wiring in separate commits, but they don't stand alone cleanly.