QVAC-22367 fix(parakeet): window the offline encoder for long audio#101
Draft
Zbig9000 wants to merge 1 commit into
Draft
QVAC-22367 fix(parakeet): window the offline encoder for long audio#101Zbig9000 wants to merge 1 commit into
Zbig9000 wants to merge 1 commit into
Conversation
transcribe_samples() / transcribe_samples_stream() ran the conformer encoder over the whole input in a single graph. Self-attention is O(T_enc^2) in encoder frames, so multi-hour inputs allocated tens of GB of attention scores and OOMed the process (a ~90 min file needs ~100 GB and was SIGKILLed). The 0.3.3 ONNX long-form windowing was never re-implemented for the GGML batch path. When the input would exceed the per-window encoder-frame ceiling (auto: min(pos_emb_max_len, 3750)), slide the encoder over the audio in overlapping windows, trim the shared context at the interior seams, and concatenate the committed frames into the same [T_enc, d_model] buffer the decoder already consumes -- bounded peak memory, single decode, no mid-word cuts. Inputs that fit in one window keep the bit-identical single-pass path, so short/typical files are unaffected. New EngineOptions knobs long_form_window_frames / long_form_context_frames (0 = auto, <0 = disable) make the window tunable and testable. Tests: - test-long-form-windows: model-free planner unit test (centre coverage, no seam gaps/dupes, bounded window size incl. a 90-min case). Runs on every checkout. - test-long-form: fixture-gated windowed-vs-single-pass parity on CTC/TDT (auto-disabled when the GGUF/wav aren't staged).
Review StatusCurrent Status: ❌ PENDING Pending reviews: Needs 1 Management or Team Lead, and 1 more from Management, Team Lead, or Member. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
QVAC-22367 — Parakeet long-audio OOM fix (engine)
Problem
Engine::transcribe_samples/transcribe_samples_streamrun the FastConformerencoder over the entire input in a single graph. Self-attention is O(T_enc²), so
long audio builds enormous score tensors — a ~90 min file needs ~100 GB and the
process is SIGKILLed. The 0.3.3 ONNX long-form windowing was never re-implemented
for the GGML batch path.
Fix
When the input would exceed the per-window encoder-frame ceiling (auto:
min(pos_emb_max_len, 3750)), slide the encoder over the audio in overlappingwindows, trim the shared context at the interior seams, and concatenate the
committed frames into the same
[T_enc, d_model]buffer the decoder alreadyconsumes — bounded peak memory, single decode, no mid-word cuts. Inputs that fit
one window keep the bit-identical single-pass path, so short/typical files are
unaffected. New
EngineOptions.long_form_window_frames/long_form_context_frames(0 = auto, <0 = disable) make it tunable/testable.Tests
test-long-form-windows— model-free planner unit test (centre coverage, noseam gaps/duplicates, bounded window size incl. a 90-min case). Runs on every
checkout.
test-long-form— fixture-gated windowed-vs-single-pass parity on CTC/TDT.Built all 101 targets and
ctestgreen locally (CPU); the model-gated paritytest runs on CI where the GGUF fixtures are staged.
Base / reorg note (draft)
This branch is based on
ecac5bb7(the commit the shippedparakeet-cpp2026-07-13#2registry port pins) to keep the bugfix isolated and fullyvalidated against the currently-shipping tree.
masterhas since movedparakeet-cpp/→engines/parakeet/(reorg PR #95); the change needs replayingonto
engines/parakeet/(a mechanical relocation) before it can merge cleanly.Opening as draft pending that rebase / a decision on the preferred base.
Companion PRs
qvac-registry-vcpkg): QVAC-22367 bump[parakeet-cpp]: 2026-07-21#0 (long-audio windowed encoder) qvac-registry-vcpkg#254qvac): QVAC-22367 fix[transcription-parakeet]: bound long-audio memory via windowed encoder qvac#3379