Skip to content

fix: harden ONNX transcription and update sherpa-onnx to 1.13.7 - #253

Merged
jatinkrmalik merged 2 commits into
VocaHQ:mainfrom
Mr-Sunglasses:fix/onnx-short-audio
Sep 5, 2026
Merged

fix: harden ONNX transcription and update sherpa-onnx to 1.13.7#253
jatinkrmalik merged 2 commits into
VocaHQ:mainfrom
Mr-Sunglasses:fix/onnx-short-audio

Conversation

@Mr-Sunglasses

@Mr-Sunglasses Mr-Sunglasses commented Sep 4, 2026

Copy link
Copy Markdown
Member

Problem

Short ONNX recordings went directly into native feature extraction and encoder subsampling. On the previous runtime, a one-sample Canary input aborted the process; tightly trimmed Moonshine Tiny clips produced repeated or incorrect words (YesYes, yes, yes, StopStar). The service also read model state without locking, could change recognizers between segments, and could reinstall an obsolete asynchronous load after unloading. Further regressions reproduced a 300 ms pause being ignored, continuous sound being split prematurely at 6.01 seconds instead of the 8-second limit, cancellation not reaching queued/running operations, and Korean segment boundaries losing spaces.

Changes

  • Pad non-silent segments shorter than one second with trailing zeros, including short final segments. Preserve original audio duration, reject empty/non-finite input, and skip digital silence.
  • Retain one recognizer for each complete request and serialize decoding per recognizer. Unload/replacement no longer invalidates active requests or blocks state access during inference. Reject cancelled or superseded model loads.
  • Require a meaningful energy drop when selecting pause boundaries; use the full segment limit when continuous sound has no pause. Preserve Korean spaces while retaining Chinese/Japanese joining behavior.
  • Forward cancellation through the shared operation queue and the ONNX decode worker. Stop before subsequent segments and discard cancelled results; keep teardown uncancellable and preserve queue ordering until active native work finishes.
  • Check cancellation before/during model hashing, before/after extraction, and before installation; preserve the existing rollback-safe install transaction.
  • Report stream/result allocation failures as errors instead of empty transcripts.
  • Update sherpa-onnx from the revision using the v1.13.4 framework to exact SPM release 1.13.7 (including its ONNX Runtime 1.28.1 dependency); verify the vendored C configuration builders against the matching release and refresh dependency documentation.
  • Add synthetic short-word fixtures, native integration regressions, and model-independent tests for audio preparation, pause selection, cancellation, queue recovery, error propagation, and Korean segment joining.

Validation

  • swift test: 462 tests, one unrelated skip, no failures. Native integration coverage ran locally with all five catalog models installed; CI skips those integrations when model weights are absent.
  • swift build -c release: passed.
  • Live CLI: short English words on Moonshine Tiny/Base, SenseVoice, and Canary; a 0.47-second Russian greeting on GigaAM; 8.7-second and 34.7-second recordings across the English models. Long recordings completed with all repeated passages retained; ordinary model spelling/punctuation differences remain.
  • git diff --check: passed.

Synthetic fixtures cover the reproduced failures; they do not establish accuracy for every microphone, language, or recording. No installed application or release was replaced. Remote CI/app-bundle verification remains separate from these local checks. Synchronous native inference/extraction cannot be interrupted mid-call; cancellation stops subsequent work after the current native operation returns.

@netlify

netlify Bot commented Sep 4, 2026

Copy link
Copy Markdown

Deploy Preview for voca-mac canceled.

Name Link
🔨 Latest commit 0ed1b3f
🔍 Latest deploy log https://app.netlify.com/projects/voca-mac/deploys/6a9b587d47e01a000872bb00

@github-actions github-actions Bot added app docs ci documentation Improvements or additions to documentation labels Sep 4, 2026
@greptile-apps

greptile-apps Bot commented Sep 4, 2026

Copy link
Copy Markdown

Greptile Summary

The PR conditions short ONNX audio before native decoding, strengthens recognizer and model-load lifecycle handling, and updates sherpa-onnx to 1.13.7.

  • Pads short non-silent segments while preserving reported recording duration and rejecting malformed audio.
  • Retains one recognizer per request, serializes decoding, and prevents obsolete model loads from being installed.
  • Improves cancellation and teardown behavior and reports native allocation failures explicitly.
  • Adds focused audio, serialization, lifecycle, and installed-model regression coverage.
  • Refreshes vendored dependency configuration and architecture documentation.

Confidence Score: 5/5

The PR appears safe to merge.

No blocking failure remains.

Important Files Changed

Filename Overview
Sources/VocaMac/Services/SherpaService.swift Retains recognizers for complete requests, serializes native decoding, propagates failures, and rejects cancelled or superseded loads.
Sources/VocaMac/Services/SherpaAudioPreparation.swift Adds malformed-audio validation, digital-silence suppression, and minimum-length padding for ONNX segments.
Sources/VocaMac/Services/LoadSerializer.swift Adds cancellation propagation while allowing teardown operations to retain their queue position.
Sources/VocaMac/Services/AudioSegmenter.swift Refines pause detection to avoid artificial early boundaries in continuous audio.
Sources/VocaMac/Services/ModelManager.swift Adds cancellation checks around download preparation, hashing, extraction, and installation.
Package.swift Pins sherpa-onnx to exact release 1.13.7 and excludes binary fixtures from test resources.
Tests/VocaMacTests/SherpaServiceTests.swift Adds regressions for short audio, malformed input, lifecycle invalidation, cancellation, and segment processing.

Sequence Diagram

sequenceDiagram
    participant Caller
    participant Router as TranscriptionRouter
    participant Sherpa as SherpaService
    participant Model as LoadedRecognizer
    participant Native as sherpa-onnx
    Caller->>Router: transcribe(audio)
    Router->>Sherpa: transcribe(audio)
    Sherpa->>Sherpa: validate and segment
    Sherpa->>Model: retain request snapshot
    loop Prepared non-silent segments
        Sherpa->>Model: acquire decode lock
        Model->>Native: create stream and decode
        Native-->>Model: result
        Model-->>Sherpa: transcript segment
    end
    Sherpa-->>Router: joined transcription
    Router-->>Caller: VocaTranscription
Loading

Reviews (2): Last reviewed commit: "fix: preserve ONNX pause boundaries and ..." | Re-trigger Greptile

@github-actions github-actions Bot removed the ci label Sep 4, 2026
@Mr-Sunglasses Mr-Sunglasses changed the title fix: handle short ONNX audio and update sherpa-onnx to 1.13.7 fix: harden ONNX transcription and update sherpa-onnx to 1.13.7 Sep 4, 2026
@jatinkrmalik

Copy link
Copy Markdown
Member

@Mr-Sunglasses thanks, this looks good. Merging.

@jatinkrmalik
jatinkrmalik merged commit f3d8f31 into VocaHQ:main Sep 5, 2026
10 checks passed
@Mr-Sunglasses
Mr-Sunglasses deleted the fix/onnx-short-audio branch September 5, 2026 04:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

app docs documentation Improvements or additions to documentation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants