fix: retry an ONNX decode that comes back empty

[Mr-Sunglasses]

Follow-up to #256. That fix was real but incomplete — recordings were still being dropped, and this is the actual mechanism.

What the padding fix missed

#256 assumed speech starting in sample zero was the trigger. It was a trigger, and padding killed it: 174 generated clips and 12 with synthetic room-noise floors all pass. But real recordings kept coming back empty.

A dump of one — a 4.0s clip this app dropped — shows why:

input	result
the recording as captured	""
the same samples ×1.001	"Hey, let's see if this sucks for eight seconds of audio."
the same samples ×0.999	""
speech shifted 100ms later	full sentence
speech shifted 300ms later	""
speech shifted 500ms later	full sentence

A 0.1% amplitude change is the difference between a perfect transcript and nothing at all. The decoders stop the moment they emit <|endoftext|> and for some inputs emit it as their very first token:

std::vector<int32_t> tokens = {max_token_id};   // if this is eos...
for (...) { if (tokens.back() == eos) break; } // ...the loop never runs
tokens.pop_back();                              // ...and this empties it

Which inputs land on that knife edge is not predictable from anything the app can measure — level, duration, DC offset, noise floor and language were all tested and none separate the failures. So no amount of tuning the padding fixes this. It is chaotic sensitivity in an upstream greedy decoder.

The fix

When a segment decodes to nothing, decode it again with the same silence distributed differently: all in front, all behind, then split unevenly.

Two properties keep this honest:

• Length-neutral. Every layout adds exactly as much silence as the first attempt, so a retry can never push a segment past the one-pass limit the segmenter exists to respect — the concern raised in review on fix: keep ONNX models from dropping whole recordings #256.
• Only after a failure. A decode that produced text is never retried, so the normal path costs nothing. A failing decode is the cheap case anyway (~0.13s for 4s of audio, since it stops on token one).

Verification

From the dumped recording I derived 20 inputs, each independently confirmed to decode to nothing on this model (gain-scaled and time-shifted variants — every one a genuine reproduction of the bug).

The ladder recovers all 20, including one (+2000ms lead-in) that no single reframing in the search recovered on its own. The 11 clips that already worked are unchanged.

swift test: 481 tests, 0 failures. New tests cover that a retry never changes the segment's length, that a successful decode is never retried, and that the ladder terminates.

Upstream

The real defect is in sherpa-onnx: offline-recognizer-canary-impl.h takes argmax of the first decoder logits unguarded, so when that argmax is <|endoftext|> the token loop never runs and tokens.pop_back() empties the result. Its reference script (scripts/nemo/canary/test_180m_flash.py) has the same structure, so the C++ is faithful to the reference — neither guards the first generated position, which is never a legitimate place for end-of-transcript when the audio contains speech.

Worth reporting upstream; this fix is a workaround at the only layer we control.

Note

The dumped-audio diagnostic that made this debuggable (writes the samples behind an empty decode to a WAV, opt-in via a hidden default) is deliberately not in this PR — one logical change per PR. Happy to send it separately if useful; it will pay for itself the next time this class of bug appears.

[netlify]

✅ Deploy Preview for voca-mac canceled.

Name	Link
🔨 Latest commit	eb1f2e6
🔍 Latest deploy log	https://app.netlify.com/projects/voca-mac/deploys/6a9bbc8599e3cd00087d82f1

[greptile-apps]

Greptile Summary

This PR retries empty sherpa-onnx segment decodes using several length-neutral silence layouts.

• Generalizes audio preparation to accept configurable leading and trailing silence.
• Adds a bounded recovery ladder that runs only after an empty or whitespace-only decode.
• Adds tests covering recovery, termination, successful-path behavior, and prepared-length invariants.

Confidence Score: 5/5

The PR appears safe to merge, with the bounded retry path preserving decoder-stream independence and existing segment-length constraints.

The changed path retries only empty results, creates a fresh native decoder stream for each attempt, terminates after three alternative layouts, and keeps every prepared retry the same length as the initial attempt.

Important Files Changed

Filename	Overview
Sources/VocaMac/Services/SherpaAudioPreparation.swift	Adds configurable, fixed-total silence layouts while preserving minimum waveform length and model-length budgeting.
Sources/VocaMac/Services/SherpaService.swift	Adds bounded retries for empty decodes using independently prepared waveforms and existing per-call decoder streams.
Tests/VocaMacTests/SherpaAudioPreparationTests.swift	Verifies that recovery layouts preserve prepared length, speech samples, and distinct framing.
Tests/VocaMacTests/SherpaServiceTests.swift	Verifies recovery sequencing, no retry after success, and termination after exhausting layouts.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[Prepare segment with balanced silence] --> B[Decode]
    B --> C{Transcript empty?}
    C -- No --> D[Return first result]
    C -- Yes --> E[Prepare next recovery layout]
    E --> F[Decode reframed segment]
    F --> G{Transcript recovered?}
    G -- Yes --> H[Return recovered result]
    G -- No, layouts remain --> E
    G -- No layouts remain --> I[Return original empty result]
    A --> J{Digital silence?}
    J -- Yes --> K[Skip segment]

Loading

_{Reviews (1): Last reviewed commit: "fix: retry an ONNX decode that comes bac..." | Re-trigger Greptile}