iOS runs ONNX Runtime while VocaMac gets the Apple-native engines (FluidAudio/ANE)

[kacperpaczos]

This came out of comparing engine stacks across the Voca clients. iOS is the one Apple platform not using Apple's accelerator path.

• VocaMac's Package.swift depends on WhisperKit, FluidAudio 0.15.5 and sherpa-onnx, and feat(web): put both phones on the OG card without frame chips #191 added Apple Speech on top.
• VocaPhone on iOS ships ios/ThirdParty/SherpaOnnx — a vendored onnxruntime.xcframework and sherpa-onnx.xcframework. There is no FluidAudio anywhere in this repo.

FluidAudio is macOS and iOS (Apache-2.0, CoreML, inference on the ANE), and its own showcase is mostly iPhone demos. Two things it brings that matter more on a phone than on a laptop:

• ANE instead of CPU. Lower power draw, and less pressure on the memory ceiling a keyboard extension has to live inside.
• Parakeet EOU 120M — streaming with end-of-utterance detection, English only, an order of magnitude smaller than the batch models. VoiceInk uses this SDK for the same reason.

One thing I could not determine from the file listing: coreml_provider_factory.h is present in the vendored ONNX Runtime headers, so the CoreML execution provider may already be reachable without adding any dependency at all. Whether it is actually enabled at runtime I cannot tell, and I do not have Apple hardware to check.

Not proposing to replace sherpa-onnx. The question is whether iOS should have an ANE path the way VocaMac does.

Related on the other platforms: VocaHQ/vocawin#39 (NPU on Windows) and #228 (execution provider on Android).

[github-actions]

Thanks @kacperpaczos. A maintainer will look into it.

---

Meanwhile, connect with us: