New models for foundry local with NPU support

[stiller-leser]

Hi there,

It's great to see finally some work being done on bringing models reliably / natively to Windows. I bought a Snapdragon Elite machine for the explicit purpose of running AI models locally. Pretty quickly I became frustrated with the hurdles. There's dangerous NPU support in llama.cpp (you have to disable secure boot) and the Olive team never responded to my issue: microsoft/Olive#2316

With BUILD coming up, is there any hope at all that some of the newer models will come to Foundry Local with NPU and not only CPU support? I really like my Snapdragon machine, but as things look like right now, I deeply regret not having bought a MacBook :(

Best regards,
stiller-leser

[baijumeswani]

We are working on getting some more models out. We aren't sure at this time if we can meet the //build timeframe for these npu models. But we definitely will be publishing more npu models soon.

[stiller-leser]

Hey,

Your answer is deeply appreciated. Awesome to hear that more NPU models are coming! I wonder if you could also work with the folks from llama.cpp to remove the Secure Boot obstacle (I am fully aware that this is the responsibility of another group)?

See ggml-org/llama.cpp#22187 for reference.

Best regards,
stiller-leser