fix: handle silent audio captures after stop - #178
Conversation
|
/build |
|
⏳ PR Build started for Build signed & notarized DMG... this usually takes 10–20 minutes. |
|
✅ PR Build ready!
📥 Install
SHA-256 checksum
|
|
Tested Results
Warmup-specific test (long take, then rapid shorts within the 3s idle window):
Confirmed we were on the PR binary ( TakeawayKeeping the engine warm helps longer / normal dictations (no ≥1.5s blanks in this session). It does not fix the silent-buffer case for very short takes — even immediately after a successful capture, inside the 3s window. So #176 is only partly addressed. Remaining failure mode looks more like “short recording still gets near-silent PCM” than “cold reacquire after idle release.” Happy to try another build if you want to dig into the short-clip path next (tap readiness / minimum capture / early buffers). |
|
One more data point / suggestion from comparing with VocaLinux (same machine class of usage — I run both daily). On Linux I basically never see this silent/blank short-dictation failure. The capture path there is different in ways that may be useful here:
So a possible follow-up to the warm-engine change: don’t treat |
Product take (after testing the warm-engine build)I appreciate #178 — keeping the engine warm for 3s is a reasonable mitigation for rapid re-records, and longer takes (≥~1.5s) looked solid on my machine. But after living with it for a bit, I don’t think this is the right long-term fix. Most of my dictations are cold starts. I’m not usually hammering the hotkey again within 3 seconds. The warm window only helps a narrow usage pattern, and our stress tests still showed sub‑~1.5s takes going blank even inside that window (right after a successful longer take). So we’re still fighting What I think the holistic fix isTreat “ready to speak” as a real state, and drive the existing cursor indicator off it:
That matches the real bug: the UI currently says “speak now” as soon as recording state flips, while early (or all) tap buffers can still be near-silent PCM. Whisper then “succeeds” on silence → empty text. Mic capture and Whisper are already separate; Whisper is fine. The lie is between UI and capture readiness. Compared to warm-engine: this helps every dictation (cold or warm), teaches the user when to talk, and pairs cleanly with “don’t transcribe silent buffers” (something VocaLinux already does via level/VAD before sending audio to the model). Warm-engine can still be a nice optimization so arming is usually instant — but it shouldn’t be the primary fix. Code pointers (no PR from me — just a map)Today the indicator goes red before/as the engine starts: // AppState.startRecording()
appStatus = .recording
isRecording = true
if showCursorIndicator {
cursorOverlay.show() // → phase = .recording (red) immediately
}
let didStartRecording = await startAudioEngine(...)Existing purple state is already there for post-stop transcription: // CursorOverlayManager
enum IndicatorPhase { case idle, recording, processing }
// recording = systemRed + mic.fill
// processing = purple #BF5AF2 + ellipsis.circle
func transitionToProcessing() { viewModel.phase = .processing }So visually, arming → ready is mostly: show Readiness signal already partially exists: audioEngine.onAudioLevel = { level in
self?.audioLevel = level
self?.cursorOverlay.updateAudioLevel(level)
}
On stop while still arming: in Also worth aligning: menu bar recording icon, if it turns red on Happy to retest a build that does arming→ready (+ silent-buffer drop). I think that’s the fix that matches how the app is actually used. |
New data — long takes can be fully blank (not just a slow start)After switching back to stock 0.7.0, I hit two back-to-back failures that change the picture:
If this were only “mic needs ~300ms to wake up,” a 19s hold would still contain ~18s of speech and Whisper would return most of the sentence. Instead decode finishes at silence-speed (~0.35s) and text is empty — so the buffer looks like near-silence for the entire key-hold, not “quiet intro + speech.” No errors around itLiterally nothing in the log except happy-path INFO: No Implication for #178 / readiness UI
Happy to grab a debug export or try a build that logs RMS/peak per recording if that helps. |
jatinkrmalik
changed the title
|
/build |
|
⏳ PR Build started for Build signed & notarized DMG... this usually takes 10–20 minutes. |
|
✅ PR Build ready!
📥 Install
SHA-256 checksum
|
|
Retested Results
Summary: ok=7 · whisper_blank=5 · TakeawayNormal/medium/long speech is fine on this build. The short blanks are unchanged from before: they still reach Whisper and come back empty at silence-speed. The new RMS gate never logged a drop, so those buffers are apparently above So this helps the “don’t transcribe obvious zeros” case in theory, but on my machine the failing shorts aren’t pure digital silence; they’re low-energy junk that still clears 0.0005. Possible next knobs:
Happy to grab another build that logs rms/peak for all stops. |
8ab6a77 to
c8774e5
Compare
2 participants
Summary
Scope note
Rebased onto main after #184 ("reject silent microphone captures") landed. #184 covers the pure-silence case with a user-visible error, which is better UX than the near-silent RMS gate in the earlier revision of this PR, so the gate is gone. What remains is the warm-engine lifecycle plus the engine reset on rejected captures.
Validation
Refs #176