fix(vad): stop fragmenting live speech and correct the flush timestamp

Two defects in the VAD path that together produced short, silence-heavy audio
clips and one corrupt segment per recording.

1. Live redemption time was 400ms

   `pipeline.rs` passed 400ms while `import.rs` and `retranscription.rs` both use
   2000ms, so live recording and offline re-transcription of the same audio
   segmented completely differently. The platform ternary was also dead
   (`if macos { 400 } else { 400 }`), and `vad.rs` carried a comment claiming the
   pipeline already passed 2000ms.

   Redemption time decides how long a silence must be before a speech segment is
   closed, and every segment becomes its own ASR request. At 400ms a 26-minute
   meeting was split into 322 requests with a median length of 3.5s. Whisper is a
   fixed 30-second-window model: below that it zero-pads the window and falls back
   on its language-model prior, which was trained on web subtitles, so short clips
   return memorised boilerplate ("subscribe to the channel", "thank you") rather
   than speech. On a real recording 47% of segment boundaries sat in the
   0.42-0.75s range, i.e. mid-sentence breaths a longer redemption simply bridges.

   Now uses a named constant kept equal to the offline paths. Costs up to ~1.6s of
   additional live-transcript latency at the end of each utterance.

2. `speech_start_sample` double-counted the session position

   It was computed as `processed_samples + timestamp_ms`, but silero's
   `timestamp_ms` is already session-absolute (`processed_duration()` minus
   `pre_speech_pad`), so the two absolute values were summed and the start position
   came out at roughly 2x the truth. The error grows with how late in the session
   the utterance begins.

   The only reader is the force-end branch in `flush()`, so it surfaced once per
   recording, on the final segment: a phantom row timestamped past the end of the
   audio, which sorted to the end of the stored transcript. Observed on a 1539.7s
   recording as a segment starting at 3083.1s.

Adds two regression tests, both verified to fail before the change and pass
after. They use a leading-silence-then-speech fixture because the force-end path
only runs when recording stops mid-utterance, and the error is only large enough
to violate the invariant when speech starts late.

Also promotes VAD_SAMPLE_RATE to a module constant, since every sample count and
timestamp in the module is expressed in it.

Author: Hus-Mek

frontend/src-tauri/src/audio/pipeline.rs

@@ -13,6 +13,15 @@ use super::recording_state::{AudioChunk, AudioError, RecordingState, DeviceType}
 use super::audio_processing::{audio_to_mono, LoudnessNormalizer, NoiseSuppressionProcessor, HighPassFilter};
 use super::vad::{ContinuousVadProcessor};
 
+/// How long a silence must last before the VAD closes a speech segment, and
+/// therefore how long the audio clips handed to the ASR engine are.
+///
+/// Kept equal to `import.rs::VAD_REDEMPTION_TIME_MS` and
+/// `retranscription.rs::VAD_REDEMPTION_TIME_MS` so that live recording and
+/// offline re-transcription of the same audio segment identically. Raising this
+/// trades live-transcript latency for longer, more accurate ASR requests.
+const VAD_REDEMPTION_TIME_MS: u32 = 2000;
+
 /// Ring buffer for synchronized audio mixing
 /// Accumulates samples from mic and system streams until we have aligned windows
 struct AudioMixerRingBuffer {
@@ -719,16 +728,34 @@ impl AudioPipeline {
         // For now, we log it for monitoring and potential optimization
         let _ = (mic_device_name, mic_device_kind, system_device_name, system_device_kind);
 
-        // Create VAD processor with balanced redemption time for speech accumulation
-        // The VAD processor now handles 48kHz->16kHz resampling internally
-        // This bridges natural pauses without excessive fragmentation
-        // For mac os core audio, 900ms, for windows 400ms seems good
-
-        let redemption_time = if cfg!(target_os = "macos") { 400 } else { 400 };
-
-        let vad_processor = match ContinuousVadProcessor::new(sample_rate, redemption_time) {
+        // Create VAD processor. The VAD processor handles 48kHz->16kHz resampling
+        // internally.
+        //
+        // Redemption time is how long a silence must last before the VAD closes a
+        // speech segment, so it decides how long the audio clips handed to the ASR
+        // engine are. Conversational speech pauses constantly for breath and
+        // mid-sentence thought, and every pause longer than this becomes a segment
+        // boundary and therefore a separate transcription request.
+        //
+        // This was 400ms, which fragmented a 26-minute meeting into 322 requests with
+        // a median length of 3.5s. Whisper is a fixed 30-second-window model: below
+        // that it zero-pads the window and leans on its language-model prior, which
+        // was trained on web subtitles, so short clips come back as memorised
+        // boilerplate ("subscribe to the channel", "thank you") instead of speech.
+        // Measured on a real recording, 47% of segment boundaries sat in the
+        // 0.42-0.75s range that a longer redemption simply bridges.
+        //
+        // 2000ms matches what the offline paths already use (see
+        // `import.rs::VAD_REDEMPTION_TIME_MS` and
+        // `retranscription.rs::VAD_REDEMPTION_TIME_MS`), so live and batch
+        // transcription now segment identically. It costs up to ~1.6s of extra
+        // live-transcript latency at the end of each utterance.
+        let vad_processor = match ContinuousVadProcessor::new(sample_rate, VAD_REDEMPTION_TIME_MS) {
             Ok(processor) => {
-                info!("VAD-driven pipeline: VAD segments will be sent directly to Whisper (no time-based accumulation)");
+                info!(
+                    "VAD-driven pipeline: segments dispatched per speech burst (redemption_time={}ms)",
+                    VAD_REDEMPTION_TIME_MS
+                );
                 processor
             }
             Err(e) => {

frontend/src-tauri/src/audio/vad.rs

@@ -4,6 +4,10 @@ use log::{debug, info, warn};
 use std::collections::VecDeque;
 use std::time::Duration;
 
+/// Silero VAD only operates at 16kHz; input is resampled to this rate, and every
+/// sample count and timestamp inside this module is expressed in it.
+const VAD_SAMPLE_RATE: u32 = 16000;
+
 /// Represents a complete speech segment detected by VAD
 #[derive(Debug, Clone)]
 pub struct SpeechSegment {
@@ -30,9 +34,6 @@ pub struct ContinuousVadProcessor {
 
 impl ContinuousVadProcessor {
     pub fn new(input_sample_rate: u32, redemption_time_ms: u32) -> Result<Self> {
-        // Silero VAD MUST use 16kHz - this is hardcoded requirement
-        const VAD_SAMPLE_RATE: u32 = 16000;
-
         // Use STRICT settings to prevent silence from reaching Whisper
         let mut config = VadConfig::default();
         config.sample_rate = VAD_SAMPLE_RATE as usize;
@@ -43,9 +44,10 @@ impl ContinuousVadProcessor {
         config.positive_speech_threshold = 0.50;  // Silero default - good for continuous speech
         config.negative_speech_threshold = 0.35;  // Silero default - allows natural pauses
 
-        // CRITICAL FIX: Removed redemption_time capping to support long continuous speech
-        // Previous: capped at 400ms, causing VAD to fragment 5-second speech into 40ms segments
-        // New: Use full redemption_time from pipeline (2000ms) to bridge natural pauses
+        // Use the caller's redemption_time uncapped, so long continuous speech stays
+        // in one segment. Callers pass 2000ms (see `pipeline.rs`, `import.rs` and
+        // `retranscription.rs`); shorter values fragment natural speech at every
+        // mid-sentence breath.
         config.redemption_time = Duration::from_millis(redemption_time_ms as u64);
         config.pre_speech_pad = Duration::from_millis(300);   // Pre-speech padding for context
         config.post_speech_pad = Duration::from_millis(400);  // Increased: more context at end
@@ -236,11 +238,16 @@ impl ContinuousVadProcessor {
                         self.last_logged_state = true;
                     }
                     self.in_speech = true;
-                    // Silero's timestamp_ms is already session-absolute (derived from its own
-                    // processed_duration), so it must NOT be offset by processed_samples —
-                    // adding them double-counts and yields start times past the end of the audio.
-                    // Convert ms to samples at the 16kHz VAD processing rate.
-                    self.speech_start_sample = timestamp_ms * 16000 / 1000;
+                    // `timestamp_ms` is ALREADY session-absolute: silero computes it as
+                    // `processed_duration() - pre_speech_pad`, where `processed_duration()`
+                    // is every sample the network has seen this session. Adding our own
+                    // session-absolute `processed_samples` to it double-counted the
+                    // position, producing a start timestamp of roughly 2x the true one.
+                    //
+                    // The only reader is the end-of-recording flush below, so the bug
+                    // surfaced once per recording, on the final segment — which landed at
+                    // ~2x the file duration and sorted to the end of the transcript.
+                    self.speech_start_sample = timestamp_ms * VAD_SAMPLE_RATE as usize / 1000;
                     self.current_speech.clear();
                 }
                 VadTransition::SpeechEnd { start_timestamp_ms, end_timestamp_ms, samples } => {
@@ -594,5 +601,94 @@ mod tests {
             assert!(duration_ms >= 200.0, "Segment {} too short: {:.0}ms", i, duration_ms);
         }
     }
-}
+    /// Leading silence, then speech that runs to the end of the buffer.
+    ///
+    /// This is the shape that matters for the flush path: an utterance that begins
+    /// late in a long session and is still in progress when recording stops.
+    fn generate_late_speech_audio(
+        silence_seconds: f32,
+        speech_seconds: f32,
+        sample_rate: u32,
+    ) -> Vec<f32> {
+        let silence_samples = (silence_seconds * sample_rate as f32) as usize;
+        let speech = generate_test_audio_with_speech(speech_seconds, sample_rate);
+
+        let mut samples = vec![0.0f32; silence_samples];
+        samples.extend_from_slice(&speech);
+        samples
+    }
+
+    /// `speech_start_sample` records where the current utterance began, so it can
+    /// never point past the number of samples the VAD has actually seen.
+    ///
+    /// It used to, because it was computed as `processed_samples + timestamp_ms` where
+    /// silero's `timestamp_ms` is ALREADY session-absolute
+    /// (`processed_duration() - pre_speech_pad`), which doubled the position. The only
+    /// reader is the force-end branch in `flush()`, so in production the corruption
+    /// escaped as one phantom segment per recording, timestamped past the end of the
+    /// audio. The error grows with how late the utterance starts, which is why it took
+    /// a long recording to surface.
+    #[test]
+    fn test_speech_start_sample_never_exceeds_processed_samples() {
+        // 20s of silence, then 3s of speech still running when the buffer ends.
+        let audio = generate_late_speech_audio(20.0, 3.0, 16000);
+
+        let mut processor =
+            ContinuousVadProcessor::new(16000, 2000).expect("Failed to create processor");
+        processor
+            .process_audio(&audio)
+            .expect("process_audio failed");
+
+        assert!(
+            processor.in_speech,
+            "expected to still be mid-speech at the end of the buffer; the invariant \
+             below would not be exercised otherwise"
+        );
 
+        assert!(
+            processor.speech_start_sample <= processor.processed_samples,
+            "speech_start_sample ({}) is past processed_samples ({}) - \
+             session-absolute timestamp double-count regression. \
+             In seconds: start={:.2}s vs processed={:.2}s",
+            processor.speech_start_sample,
+            processor.processed_samples,
+            processor.speech_start_sample as f64 / VAD_SAMPLE_RATE as f64,
+            processor.processed_samples as f64 / VAD_SAMPLE_RATE as f64,
+        );
+    }
+
+    /// Whatever `flush()` emits must also lie inside the audio that was supplied.
+    #[test]
+    fn test_flush_segment_timestamps_stay_within_audio_duration() {
+        let audio = generate_late_speech_audio(20.0, 3.0, 16000);
+        let audio_duration_ms = (audio.len() as f64 / 16000.0) * 1000.0;
+
+        let mut processor =
+            ContinuousVadProcessor::new(16000, 2000).expect("Failed to create processor");
+
+        let mut segments = processor
+            .process_audio(&audio)
+            .expect("process_audio failed");
+        let flushed = processor.flush().expect("flush failed");
+        assert!(
+            !flushed.is_empty(),
+            "flush() emitted nothing, so the force-end path under test never ran"
+        );
+        segments.extend(flushed);
+
+        for (i, seg) in segments.iter().enumerate() {
+            assert!(
+                seg.start_timestamp_ms <= audio_duration_ms,
+                "Segment {i} starts at {:.0}ms, beyond the {:.0}ms of audio supplied",
+                seg.start_timestamp_ms,
+                audio_duration_ms
+            );
+            assert!(
+                seg.end_timestamp_ms >= seg.start_timestamp_ms,
+                "Segment {i} ends before it starts: {:.0}ms -> {:.0}ms",
+                seg.start_timestamp_ms,
+                seg.end_timestamp_ms
+            );
+        }
+    }
+}