Foundry/QNN NPU LLM crash on Surface Pro 11 Snapdragon X Elite

[ingcasucci]

Summary

Local NPU execution works with ONNX Runtime QNN on this machine, but Foundry Local NPU LLMs crash during text generation. The crash is reproducible with two different Foundry QNN NPU models:

• phi-3-mini-4k-instruct-qnn-npu:3
• qwen2.5-1.5b-instruct-qnn-npu:2

The failure is not a general NPU failure: a minimal ONNX model runs successfully on QNNExecutionProvider with device type NPU, and individual LLM ONNX components can run on NPU. The crash appears when the decoder context model is run with a prompt shorter than the fixed sliding window size of 64 tokens, and also when using onnxruntime_genai.Generator.append_tokens(...).

Environment

• Device: Surface Pro 11th Edition For Business, Model 2085
• CPU/SoC: Snapdragon X Elite, X1E80100
• OS: Windows 11 Enterprise ARM64, build 26200.8457
• RAM: 16 GB
• NPU device: Snapdragon(R) X Elite - X1E80100 - Qualcomm(R) Hexagon(TM) NPU
• NPU driver observed from PowerShell: 30.0.219.1000
• Foundry Local: Microsoft.FoundryLocal 0.8.119.102 arm64
• Python: official ARM64 Python 3.13
• Python packages:
  • onnxruntime==1.24.4
  • onnxruntime-qnn==2.1.1
  • onnxruntime-genai==0.13.2

What works

Minimal QNN NPU model works

Using a tiny ONNX QLinearMatMul model with:

• onnxruntime-qnn
• QNNExecutionProvider
• QnnHtp.dll
• selected ORT EP device type NPU

Result:

Selected device: npu
Creating InferenceSession...
Session created.
Running inference...
Output: [[128 128 128]]
Minimal QNN test completed.

This shows the NPU and the QNN plugin can execute at least a simple ONNX graph.

Foundry model ONNX components can load and partly run on NPU

For phi-3-mini-4k-instruct-qnn-npu:3:

• embeddings.onnx runs on NPU
• lm_head.onnx runs on NPU
• context_ctx.onnx runs on NPU when total_seq_len=64
• iterator_ctx.onnx runs on NPU after a 64-token context prefill

For qwen2.5-1.5b-instruct-qnn-npu:2:

• context_ctx.onnx runs on NPU when total_seq_len=64

What fails

ORT GenAI Generator crashes

For both models, using onnxruntime_genai.Generator crashes during:

generator.append_tokens(input_tokens)

The Python faulthandler output shows:

Windows fatal exception: access violation

Current thread:
  File "...test_ort_genai_direct.py", line 171 in main

where line 171 is:

generator.append_tokens(input_tokens)

Windows Event Viewer reports crashes such as:

Application: python.exe
Faulting module: ntdll.dll
Exception code: 0xc0000409

Foundry Local service/CLI crashes show the same class of failure when calling chat completions against NPU models.

Direct context ONNX run crashes for short prompt length

For phi-3-mini-4k-instruct-qnn-npu:3:

• context_ctx.onnx with total_seq_len=64: succeeds
• context_ctx.onnx with total_seq_len=4: crashes with access violation

For qwen2.5-1.5b-instruct-qnn-npu:2:

• context_ctx.onnx with total_seq_len=64: succeeds
• context_ctx.onnx with total_seq_len=30: crashes with access violation

Example faulthandler output for Qwen:

Windows fatal exception: access violation

Current thread:
  File "...onnxruntime_inference_collection.py", line 296 in run
  File "<stdin>", line 32 in <module>

This points to a native crash inside InferenceSession.run(...).

Reproduction Notes

The models both use:

"sliding_window": {
    "window_size": 64,
    "pad_value": 0,
    "alignment": "left",
    "slide_key_value_cache": false
}

The crash occurs when the prompt/prefill length is shorter than the 64-token window, while a full 64-token context run succeeds.

Expected Behavior

Foundry Local / ORT GenAI should be able to run NPU LLM generation for ordinary short prompts, e.g. "ok" or "Rispondi solo: ok", without crashing the hosting process.

Actual Behavior

The hosting process crashes with native access violation / 0xc0000409 during the first generation path:

• Foundry Local: service process terminates during /v1/chat/completions
• ORT GenAI direct: generator.append_tokens(...) crashes
• ONNX Runtime direct: context_ctx.onnx crashes when total_seq_len < 64

Local Repro Artifacts

Scripts used locally:

• test_qnn_ep_minimal.py
• test_ort_genai_direct.py

Relevant local logs:

• ort-genai-qwen15-faulthandler.log
• qwen15-context30-faulthandler.log
• ort-genai-faulthandler.log

context-short-faulthandler.log
ort-genai-faulthandler.log
ort-genai-qwen15-faulthandler.log
qwen15-context30-faulthandler.log
test_ort_genai_direct.py
test_qnn_ep_minimal.py

[prathikr]

Hello, the version you are using (0.8.119.102) is very old and no longer supported. Can you please confirm if this issue is still present using v1.1.0? I just tried running chat-completion on my Snapdragon Elite X for model: phi-3-mini-4k-instruct-qnn-npu:3 and it is working.

Please note, if you are using CLI, there is currently no support for that in v0.9.0+ but it is in our roadmap (no exact timetable yet). You can feel free to leverage any of our SDKs to integrate Foundry Local v0.9.0+ into your applications as that is recommended over shipping production apps using CLI.

[ingcasucci]

Hello, the version you are using (0.8.119.102) is very old and no longer supported. Can you please confirm if this issue is still present using v1.1.0? I just tried running chat-completion on my Snapdragon Elite X for model: phi-3-mini-4k-instruct-qnn-npu:3 and it is working.

Please note, if you are using CLI, there is currently no support for that in v0.9.0+ but it is in our roadmap (no exact timetable yet). You can feel free to leverage any of our SDKs to integrate Foundry Local v0.9.0+ into your applications as that is recommended over shipping production apps using CLI.

Hello @prathikr , thanks for your feedback.

First, I have to say that the installation on Windows does not install the v1.1.0 (at least, not in my region, Italy)

PS C:\Users\XXX> foundry --version
0.8.119
PS C:\Users\XXX> winget upgrade Microsoft.FoundryLocal
Non sono stati trovati aggiornamenti disponibili.
Non sono disponibili versioni più recenti del pacchetto dalle origini configurate.
PS C:\Users\XXX

foundry-sdk-110-npu-faulthandler-redacted.log
foundry-sdk-110-npu-foundry-core-redacted.log
foundry-sdk-110-version-evidence-redacted.txt

foundry --version
0.8.119

I moved away from the old CLI path and reproduced this with the official Python WinML SDK/runtime 1.1.0.

CPU execution works in the same SDK environment. NPU execution still crashes.

Environment:

• foundry-local-sdk-winml==1.1.0
• foundry-local-core-winml==1.1.0
• onnxruntime-core==1.23.2.3
• onnxruntime-genai-core==0.13.2
• Native ARM64 Python 3.13.13
• Surface Pro 11 for Business, Snapdragon X Elite X1E80100
• Windows 11 ARM64 build 26200.8457
• NPU driver: 30.0.219.1000
• QNN EP package: MicrosoftCorporationII.WinML.Qualcomm.QNN.EP.1.8, version 1.8.30.0

Observed behavior:

• QNNExecutionProvider registers successfully.
• phi-3-mini-4k-instruct-qnn-npu:3 downloads successfully.
• The model loads successfully.
• The process crashes during the first chat completion call.

Foundry log shows:
Model loaded successfully: phi-3-mini-4k-instruct-qnn-npu:3
Command: ModelLoad Status: Success

Python faulthandler shows:
Windows fatal exception: access violation
foundry_local_sdk/openai/chat_client.py, line 215, in complete_chat

Windows Event Viewer reports:

• Application: python.exe
• Faulting module: ntdll.dll
• Exception code: 0xc0000409
• Event name: BEX64

The runtime also prints:
Error in cpuinfo: Unknown chip model name 'Snapdragon(R) X 12-core X1E80100 @ 3.40 GHz'.
Please add new Windows on Arm SoC/chip support to arm/windows/init.c!

I attached redacted logs and version evidence from the SDK 1.1.0 run. Since you mentioned this works on your Snapdragon Elite X, can you share your exact Snapdragon SKU, Windows build, NPU driver version, QNN EP package/version, Python version, and SDK/runtime versions?

foundry-sdk-110-npu-faulthandler-redacted.log
foundry-sdk-110-npu-foundry-core-redacted.log
foundry-sdk-110-version-evidence-redacted.txt

[prathikr]

Hi @ingcasucci, you will not be able to upgrade CLI via winget to v1.1.0 as we do not have CLI supported for v0.9.0+ yet. Support for that is scheduled for next month.

The runtime print you see is harmless, we are working to remove it but it shouldn't affect execution.

Snapdragon SKU: Snapdragon(R) X Elite - X1E80100 - Qualcomm(R) Hexagon(TM) NPU
Windows OS Build: 26200.8457
NPU driver version: 30.0.219.1000
QNN EP package/version: KB5096135
Python Version: 3.12.10
SDK version: 1.1.0