Application is getting crashed while creating session for the onnxruntime-qnn with QnnCpu backend option.

[AviIIT]

Describe the issue

I attempted to execute the MobileNet model using the ONNX Runtime with the QNN execution provider on a CPU, specifying 'CPUExecutionProvider' as the execution provider option. However, the InferenceSession frequently crashes during its creation when running the same script.

To reproduce

I tried to execute the mobilenet model using below script:

import onnxruntime as ort
import numpy as np
import time
import io_utils

execution_provider_option = {"backend_path": "QnnCpu.dll"}

#Create ONNX Runtime session.
onnx_model_path = "./mobilenet_v2.onnx"

session = ort.InferenceSession(onnx_model_path,
                              providers=["QNNExecutionProvider"],
                              provider_options=[execution_provider_option])

#Step2: Input/Output handling, Generate raw input
#github repo for below artifact: https://github.com/quic/wos-ai/tree/main/Artifacts
img_path = "https://raw.githubusercontent.com/quic/wos-ai/refs/heads/main/Artifacts/kitten.jpg"
raw_img = io_utils.preprocess(img_path)


#Model input and output names
outputs = session.get_outputs()[0].name
inputs = session.get_inputs()[0].name

#Step3: Model inferencing using preprocessed input.
start_time = time.time()
for i in range(10):
   prediction = session.run([outputs], {inputs: raw_img})
end_time = time.time()
execution_time = ((end_time-start_time) * 1000)/10

#Step4: Output postprocessing
io_utils.postprocess(prediction)
print("Execution Time: ", execution_time, "ms")

This script and necessary setup instructions can be found here- https://docs.qualcomm.com/bundle/publicresource/topics/80-62010-1/ort-qnn-ep.html?product=1601111740057789

Urgency

It is very urgent.

Platform

Windows

OS Version

11

ONNX Runtime Installation

Released Package

ONNX Runtime Version or Commit ID

onnxruntime-qnn 1.21.0

ONNX Runtime API

Python

Architecture

ARM64

Execution Provider

QNN

Execution Provider Library Version

QNN 2.31.0.250130

[xadupre]

Is there any error message?

[AviIIT]

No, It is not showing any error. Tried debugging and found out that on the InferenceSession creation step, it is simply crashing.

[xadupre]

• @adrianlizarraga

[AviIIT]

Hello @xadupre
could you please let me know if there is any update on this?

[jywu-msft]

are you using amd64/x64 python or arm64 python? There's a current known issue with QNN EP on amd64/arm64 being investigated that you may be running into.

[jywu-msft]

there's a known issue with x64 onnxruntime-qnn 1.21.0 package that's being investigated.
can you pip install onnxruntime-qnn==1.20.2 ?

[AviIIT]

Yes, I am using the python amd64 version.
I tried with the onnxruntime-qnn==1.20.2, still it has the same issue of crashing.

[jywu-msft]

which python version are you using?
I installed miniconda , using python 3.12 and
pip install onnxruntime-qnn==1.20.2 works for me. (1.21.0 does not)
alternatively arm64 python and arm64 onnxruntime-qnn 1.21.0 should work