Handle empty alignments in find_alignment (companion to the CTranslate2 divide-by-zero fix for #1342) - #1460
Open
MAkcanca wants to merge 1 commit into
Open
Conversation
ctranslate2 returns an empty alignment for a window that has no frames left after the encoder-stride halving (num_frames < 2, e.g. a ~10ms final VAD chunk on audio where Whisper hallucinates past the clip end; see OpenNMT/CTranslate2#2065). find_alignment then crashed with IndexError at time_indices[jumps] because the boolean mask is padded to length 1 while the indexed array is empty. Return an empty word list for such segments instead: the segment simply carries no word timestamps. Fixes SYSTRAN#1342 (together with the ctranslate2 fix, which replaces the process-killing 0xC0000094 native crash with the empty alignment this change handles).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
find_alignmentcrashes withIndexError: boolean index did not match indexed arrayattime_indices[jumps]whenmodel.align()returns an empty alignment for a segment. This PR returns an empty word list for such segments instead — the segment simply carries no word timestamps, whichadd_word_timestampsalready handles.Why
This is the faster-whisper half of the fix for #1342 (process killed with
0xC0000094on Windows when word-level timestamps are enabled for specific audio files). Root cause and reproduction are detailed in #1342 (comment) and OpenNMT/CTranslate2#2065:align()withnum_frames=1(a ~10ms window).num_framesfor the encoder stride (1 → 0) and hits an integer division by zero in native code — killing the whole process, uncatchable from Python.align()returns an empty alignment for such windows — and this guard makes faster-whisper handle that gracefully.Verification
word_timestamps=True, vad_filter=True) that previously killed the process on every attempt now transcribes completely with both patches: 17 real segments with word timestamps, plus the trailing hallucinated segment (60.0→90.0s on 60.01s of audio) with no words.pytest tests/: 17 passed; the 2 failures (test_transcribe,test_multilingual_transcription— exact-transcript assertions) fail identically on a pristine checkout of master in the same environment, i.e. pre-existing and unrelated.black,isort,flake8all clean.AI assistance disclosure
I used an AI assistant (Claude) to help trace the crash and draft the patch; I directed the investigation and verified the behavior empirically (instrumented
align()capture, local patched CTranslate2 build, the real-world clip, and the test suite). I am responsible for the change.