ort.InferenceSession fails silently

[DavidLuong98]

Describe the issue

onnxruntime-qnn :1.20.2

I am using python to profile the models. I have a sample code below :

provider_option = {
    'backend_path': 'QnnHtp.dll',
    'htp_performance_mode': 'sustained_high_performance',
    'htp_graph_finalization_optimization_mode': '3',
    'soc_model': '60',
    'htp_arch': '73'
}

try:
    return ort.InferenceSession(
        self.model_path,
        sess_options=sess_options,
        providers=['QNNExecutionProvider'],
        provider_options=[provider_option],
    )
except Exception as e:
    print("Error creating InferenceSession:", e)

ort.InferenceSession(..) fails silently without any explanation. It does not hit the exception either.

When running this with onnxruntime_perf_test.exe the console prints out :

:This is an invalid model. Type Error: Type 'tensor(double)' of input parameter ((..._output_0_quantized) of operator (DequantizeLinear) in node ((..._output_0_DequantizeLinear) is invalid.

Expected behavior is to have this throw the error above.

To reproduce

I am not able to share the model. I have a CNN model which I created dummy weights. From there, I ran the ORT code snippet above.

Urgency

Not urgent

Platform

Windows

OS Version

24H2

ONNX Runtime Installation

Built from Source

ONNX Runtime Version or Commit ID

1.20.2

ONNX Runtime API

Python

Architecture

ARM64

Execution Provider

Other / Unknown

Execution Provider Library Version

QNN

[tianleiwu]

The error means the model is invalid. DequantizeLinear input shall not be double. See https://onnx.ai/onnx/operators/onnx__DequantizeLinear.html

You can add a node Cast(to=float) to convert the input to float.

[DavidLuong98]

The error means the model is invalid. DequantizeLinear input shall not be double. See https://onnx.ai/onnx/operators/onnx__DequantizeLinear.html

You can add a node Cast(to=float) to convert the input to float.

Yes, i agree that the model is invalid. But should ort.InferenceSession() throw an error instead of failing silently??

[tianleiwu]

Yes, i agree that the model is invalid. But should ort.InferenceSession() throw an error instead of failing silently??

@HectorSVC, could you try reproduce it and see whether Qnn EP can handle invalid model properly.