-
-
Notifications
You must be signed in to change notification settings - Fork 85
Open
Labels
enhancementNew feature or requestNew feature or request
Description
What
Enable Intel NPU hardware acceleration for local Whisper inference by building whisper.cpp with -DGGML_OPENVINO=ON.
Why
Modern Intel CPUs (Core Ultra series) have dedicated NPUs with up to 48 TOPS sitting idle. Offloading the Whisper encoder to the NPU would mean faster inference, lower CPU usage, and better battery life — all important for a local-first dictation app.
How
whisper.cpp already supports OpenVINO as a backend ([docs](https://github.com/ggml-org/whisper.cpp#openvino)). The main work would be:
- Build the whisper.cpp addon with
GGML_OPENVINO=ONfor the Windows x64 target - Auto-detect NPU availability at runtime and fall back to CPU if unavailable
- Generate/cache the OpenVINO encoder model on first launch (or bundle pre-converted models)
Environment
- Intel Core Ultra (Lunar Lake) with NPU, 32 GB RAM, Windows 11
- Amical v1.0.0
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
enhancementNew feature or requestNew feature or request