fix(audio): route WhisperKit capture via inputDeviceID (our fork), drop default-swap

Replaces the system-default capture-and-restore workaround for the WhisperKit
streaming path with the clean upstream mechanism: pin WhisperKit to OUR fork
(initcore0/argmax-oss-swift) at v1.0.0 + a single-file backport of upstream PR
argmaxinc/argmax-oss-swift#503, which threads an optional `inputDeviceID` through
`AudioStreamTranscriber` to `AudioProcessor.startRecordingLive(inputDeviceID:)`.

Why fork instead of pinning the upstream PR branch: that branch is diverged from
v1.0.0 (8 ahead / 1 behind, ~60 files incl. the CoreML AudioEncoder/TextDecoder/
FeatureExtractor paths the 1.0.0 pin + GPU-encoder choice deliberately hold
stable on macOS 26) and lives on a personal fork we don't control. Our fork
branch (openwhisp/v1.0.0-input-device) is v1.0.0 + ONLY the 4-line device patch,
pinned by immutable commit. When #503 lands in a tagged upstream release we bump
and delete the fork.

Engine changes:
- WhisperKitBridge.makeStreamHandle takes `inputDeviceID: AudioDeviceID?` and
  passes it to AudioStreamTranscriber(inputDeviceID:).
- WhisperKitStreamingEngine resolves the selected UID to an AudioDeviceID and
  passes it through — deleting the DefaultInputOverride engage/poll/restore dance
  and the isCapturing() helper. Unresolved (incl. a resolve→nil TOCTOU) stays a
  hard error, never a silent default capture.
- DefaultInputOverride remains in AudioInputRouter, now used ONLY by the legacy
  AVAudioRecorder path (which genuinely can only capture the system default).

Verified on live hardware against the actual forked WhisperKit lib (linked as the
app links it): AudioProcessor.startRecordingLive(inputDeviceID:) started the
engine bound to a non-default device (id 243) while the system default (id 110)
stayed UNCHANGED — direct per-device capture, no swap. Fresh-clone resolve pulls
the fork at the pinned commit and builds; ./build.sh green; swift test 435/435.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Author: initcore0

OpenWhisp/Services/WhisperKitBridge.swift

@@ -1,4 +1,5 @@
 import Foundation
+import CoreAudio   // AudioDeviceID (the input-device id threaded to AudioStreamTranscriber)
 
 /// Maps OpenWhisp's engine-facing language setting to WhisperKit decoding
 /// options. Mirrors `WhisperTask` (used for whisper.cpp): the shared
@@ -212,10 +213,16 @@ enum WhisperKitBridge {
     /// "auto" we leave it nil and let WhisperKit detect; because the engine only
     /// surfaces CONFIRMED text as live partials, the per-window detection on the
     /// unconfirmed tail doesn't cause visible flapping.
+    /// `inputDeviceID` (a CoreAudio `AudioDeviceID`) pins the input device the stream
+    /// captures from; nil = system default input. Threaded straight into
+    /// `AudioStreamTranscriber` (our WhisperKit fork backports upstream #503's
+    /// `inputDeviceID` passthrough), which forwards it to `startRecordingLive` — so
+    /// device routing needs NO system-default swap.
     static func makeStreamHandle(
         kit: WhisperKit,
         task: WhisperKitTaskMapper.Resolved,
         languageOverride: String?,
+        inputDeviceID: AudioDeviceID? = nil,
         onState: @escaping (WhisperKitStreamState) -> Void
     ) throws -> WhisperKitStreamHandle {
         guard let tokenizer = kit.tokenizer else {
@@ -249,6 +256,7 @@ enum WhisperKitBridge {
             audioProcessor: kit.audioProcessor,
             decodingOptions: options,
             useVAD: true,                 // skip silence — don't transcribe dead air
+            inputDeviceID: inputDeviceID, // nil = system default (fork backport of #503)
             stateChangeCallback: { _, new in
                 // Absolute-scale level for the silence VAD. `bufferEnergy` is
                 // RELATIVE to WhisperKit's rolling 2s silence floor — right for a
@@ -354,14 +362,6 @@ final class WhisperKitStreamHandle {
     func start() async throws { try await transcriber?.startStreamTranscription() }
     func stop() async { await transcriber?.stopStreamTranscription() }
 
-    /// True once WhisperKit's `AudioProcessor` has built and started its capture
-    /// engine — i.e. the input node is bound to the (currently-default) device.
-    /// The streaming engine polls this to know when it's safe to restore a
-    /// system-default-input override without losing the device binding.
-    func isCapturing() -> Bool {
-        (kit.audioProcessor as? AudioProcessor)?.audioEngine?.isRunning ?? false
-    }
-
     /// The full assembled transcript (confirmed + unconfirmed) as of the last state.
     func fullText() -> String {
         (latest?.fullText ?? "").trimmingCharacters(in: .whitespacesAndNewlines)

OpenWhisp/Services/WhisperKitStreamingEngine.swift

@@ -1,4 +1,5 @@
 import Foundation
+import CoreAudio   // AudioDeviceID (the input-device id threaded to WhisperKit)
 
 /// Experimental real-time WhisperKit engine (streaming partials).
 ///
@@ -22,14 +23,10 @@ final class WhisperKitStreamingEngine: StreamingTranscriptionEngine {
     /// (multilingual EN+RU) — the same model the file engine uses.
     private let modelName: String
 
-    /// Pinned input-device UID for the next session ("" = system default).
-    ///
-    /// Unlike Apple Speech, WhisperKit 1.0.0 owns the mic through
-    /// `AudioStreamTranscriber` → `AudioProcessor.startRecordingLive()`, which is
-    /// called with NO device (an internal API in a pinned remote dependency we can't
-    /// patch). WhisperKit's engine setup binds whatever the SYSTEM DEFAULT input is
-    /// at start. So the only way to route it to the selected device is to swap the
-    /// system default around stream start and restore it — see `deviceOverride`.
+    /// Pinned input-device UID for the next session ("" = system default). Resolved
+    /// to a CoreAudio device in `runStart` and passed straight into WhisperKit's
+    /// `AudioStreamTranscriber(inputDeviceID:)` (our fork backports upstream #503's
+    /// passthrough), so capture targets the device directly — no system-default swap.
     private var selectedDeviceID = ""
 
     func selectDevice(_ deviceID: String) {
@@ -49,11 +46,6 @@ final class WhisperKitStreamingEngine: StreamingTranscriptionEngine {
     // `handleState`. Touched only on the main actor (via the serialized chain).
     @MainActor private var transcriber: WhisperKitStreamHandle?
 
-    // Live system-default-input swap for the current session (nil when routing to the
-    // system default). Engaged just before the stream starts and restored once the
-    // engine is bound to the device (or on any teardown/error path). Main-actor only.
-    @MainActor private var deviceOverride: AudioInputRouter.DefaultInputOverride?
-
     // Last confirmed transcript we emitted as a partial — used so we only forward
     // forward-progress (monotonic confirmed text), avoiding paste churn from the
     // unconfirmed/hypothesis tail.
@@ -101,25 +93,27 @@ final class WhisperKitStreamingEngine: StreamingTranscriptionEngine {
     @MainActor
     private func runStart(task: WhisperKitTaskMapper.Resolved) async {
         do {
-            // Route to the selected input device. WhisperKit binds the SYSTEM DEFAULT
-            // input when its engine starts (no per-engine device seam in 1.0.0), so a
-            // pinned device requires swapping the default around stream start and
-            // restoring it once the engine is bound. An unresolved pinned device is a
-            // hard error — never silently capture the default.
-            //
-            // RESOLVE here (fail fast before the possibly-slow model load) but ENGAGE
-            // the swap only after ensureLoaded() below — the global default must not
-            // stay changed for the whole model load (cold start can be seconds), only
-            // for the brief window until the capture engine binds the device.
-            let deviceToRoute: AudioDevice?
+            // Resolve the selected input device. It's passed straight into
+            // AudioStreamTranscriber(inputDeviceID:) (our WhisperKit fork backports
+            // upstream #503), which forwards it to startRecordingLive — WhisperKit
+            // captures the chosen device directly, no system-default swap needed. An
+            // unresolved pinned device is a hard error, never a silent default capture.
+            let inputDeviceID: AudioDeviceID?
             switch AudioInputRoutingPolicy.decide(
                 microphoneID: selectedDeviceID,
                 deviceResolved: AudioInputRouter.canResolve(uid: selectedDeviceID)
             ) {
             case .systemDefault:
-                deviceToRoute = nil
+                inputDeviceID = nil
             case .useDevice(let uid):
-                deviceToRoute = AudioInputRouter.resolve(uid: uid)
+                // Resolve to the concrete device id. A nil here (device vanished
+                // between canResolve and now) is treated as unresolved — a hard error,
+                // NOT a silent nil that would fall back to the system default.
+                guard let id = AudioInputRouter.resolve(uid: uid)?.deviceID else {
+                    onError?(AudioInputRoutingPolicy.unresolvedMessage(uid: uid))
+                    return
+                }
+                inputDeviceID = id
             case .unresolved(let uid):
                 onError?(AudioInputRoutingPolicy.unresolvedMessage(uid: uid))
                 return
@@ -131,22 +125,15 @@ final class WhisperKitStreamingEngine: StreamingTranscriptionEngine {
             let handle = try WhisperKitBridge.makeStreamHandle(
                 kit: kit,
                 task: task,
-                languageOverride: nil
+                languageOverride: nil,
+                inputDeviceID: inputDeviceID
             ) { [weak self] newState in
                 Task { @MainActor in
                     guard let self, self.generation == myGeneration else { return }
                     self.handleState(newState)
                 }
             }
             transcriber = handle
-
-            // Engage the default swap NOW (model loaded; the stream is about to grab
-            // the mic). Restored once the engine binds the device (poll below) or on
-            // any teardown/error path.
-            if let device = deviceToRoute {
-                let override = AudioInputRouter.DefaultInputOverride()
-                if override.engage(device) { deviceOverride = override }
-            }
             // `start()` runs the realtime loop until stopped; it returns when the
             // stream ends. Don't block the chain on it (a stop must be able to run),
             // so drive it in a detached child whose lifetime the stop tears down.
@@ -161,54 +148,16 @@ final class WhisperKitStreamingEngine: StreamingTranscriptionEngine {
                 } catch {
                     NSLog("[WhisperKitStream] stream error: %@", error.localizedDescription)
                     guard let self, self.transcriber === handle, !self.didFinish else { return }
-                    self.restoreDeviceOverride()   // startup failed; undo the default swap
                     self.onError?("WhisperKit streaming failed: \(error.localizedDescription)")
                 }
             }
-            // Restore the system default promptly once WhisperKit's engine is bound to
-            // the (now-default) device: the AudioUnit holds its CurrentDevice after
-            // start, so the global default can go back with no effect on capture. This
-            // keeps the user's default changed for a sub-second window, not the whole
-            // session. Falls through to runStop's restore if the engine never binds.
-            if deviceOverride != nil {
-                restoreOverrideWhenCaptureLive(handle: handle, generation: myGeneration)
-            }
         } catch {
             NSLog("[WhisperKitStream] start error: %@", error.localizedDescription)
-            restoreDeviceOverride()
             guard !didFinish else { return }
             onError?("WhisperKit streaming failed: \(error.localizedDescription)")
         }
     }
 
-    /// Poll (bounded) for WhisperKit's capture engine to start, then restore the
-    /// system-default-input override. The device binding survives the restore. If the
-    /// engine never comes up within the budget the override is left for `runStop` to
-    /// restore, so the default is never stranded.
-    @MainActor
-    private func restoreOverrideWhenCaptureLive(handle: WhisperKitStreamHandle, generation: Int) {
-        Task { @MainActor [weak self] in
-            // ~2s budget at 50ms granularity: engine start is normally tens of ms;
-            // this only needs to outlast a slow cold start of the AVAudioEngine graph.
-            for _ in 0..<40 {
-                guard let self, self.generation == generation, self.transcriber === handle,
-                      !self.didFinish else { return }
-                if handle.isCapturing() {
-                    self.restoreDeviceOverride()
-                    return
-                }
-                try? await Task.sleep(nanoseconds: 50_000_000)
-            }
-        }
-    }
-
-    /// Restore the system default input if this session swapped it. Idempotent.
-    @MainActor
-    private func restoreDeviceOverride() {
-        deviceOverride?.restore()
-        deviceOverride = nil
-    }
-
     func stop(cancel: Bool) {
         // Synchronous main-actor enqueue (see start) so stop→start order is preserved.
         MainActor.assumeIsolated {
@@ -228,10 +177,6 @@ final class WhisperKitStreamingEngine: StreamingTranscriptionEngine {
         let handle = transcriber
         transcriber = nil
         didFinish = true
-        // Safety net: if the stream ended before the capture-live poll restored the
-        // system default (very short session, or the engine never bound), restore it
-        // now so the user's default input is never left changed. Idempotent.
-        restoreDeviceOverride()
         // Supersede the stream's generation so state callbacks it already
         // dispatched are dropped — they must not fire onPartial into whatever
         // session starts next. The final below is unaffected: it's computed

third_party/whisperkit-dep/Package.swift

@@ -12,15 +12,27 @@ let package = Package(
         .library(name: "WhisperKitDep", type: .static, targets: ["WhisperKitDep"])
     ],
     dependencies: [
-        // Pinned to the current 1.0.0 release. macOS 26 model loading is handled
-        // app-side via `modelFolder` + a GPU audio encoder (see docs/WHISPERKIT_PILOT.md),
-        // not by the WhisperKit version.
-        .package(url: "https://github.com/argmaxinc/WhisperKit.git", from: "1.0.0")
+        // Pinned to OUR fork of WhisperKit (argmaxinc renamed the repo to
+        // `argmax-oss-swift`), at the v1.0.0 release + a single-file backport of
+        // upstream PR #503 (inputDeviceID passthrough on AudioStreamTranscriber) so
+        // streaming/live capture can target a selected input device. We fork rather
+        // than pin the contributor's branch because that branch also carries ~60
+        // files of divergent macOS-26 CoreML/ANE churn; our branch is v1.0.0 + ONLY
+        // that patch. Pinned by exact commit (immutable) — bump to an upstream
+        // release once #503 lands there, then drop the fork. macOS 26 model loading
+        // is still handled app-side via `modelFolder` + a GPU audio encoder (see
+        // docs/WHISPERKIT_PILOT.md).
+        //
+        // Fork branch: openwhisp/v1.0.0-input-device (initcore0/argmax-oss-swift).
+        .package(
+            url: "https://github.com/initcore0/argmax-oss-swift.git",
+            revision: "7e5f648249fde3eeabab02250529f63f16476e91"
+        )
     ],
     targets: [
         .target(
             name: "WhisperKitDep",
-            dependencies: [.product(name: "WhisperKit", package: "WhisperKit")]
+            dependencies: [.product(name: "WhisperKit", package: "argmax-oss-swift")]
         )
     ]
 )