Getting Errors during ONNX translation #652
Description
System Info
Hey there!
Tried installing a 1 year old project and the container won't start anymore. Getting it on multiple models.
Can somebody explain me why/whats going on?
Running on
- Linux
- x86
- Intel(R) Core(TM) Ultra 7 255U
Infinity is the latest cpu image.
The ONNX file model_quantized_optimized.onnx is not a regular name used in optimum.onnxruntime, the ORTModel might not behave as expected.
2025-12-23 22:51:12.139294021 [E:onnxruntime:, inference_session.cc:2280 operator()] Exception during initialization: /onnxruntime/onnxruntime/core/providers/openvino/ov_interface.cc:79 std::shared_ptr<ov::Model> onnxruntime::openvino_ep::OVCore::ReadModel(std::string&&, const std::string&) [OpenVINO-EP] [OpenVINO-EP] Exception while Reading network: Check 'false' failed at src/frontends/onnx/frontend/src/frontend.cpp:164:
FrontEnd API failed with GeneralFailure:
Errors during ONNX translation:
Check '(element_type_bias == ov::element::f32)' failed at src/frontends/onnx/frontend/src/op/com.microsoft/dynamic_quantize_matmul.cpp:73:
While validating ONNX node '<Node(DynamicQuantizeMatMul): /encoder/layer.0/attention/output/dense/MatMul_quant>':
Unsupported input bias type, accepted FP32 but got: dynamic
Check '(element_type_bias == ov::element::f32)' failed at src/frontends/onnx/frontend/src/op/com.microsoft/dynamic_quantize_matmul.cpp:73:
While validating ONNX node '<Node(DynamicQuantizeMatMul): /encoder/layer.0/intermediate/dense/MatMul_quant>':
Unsupported input bias type, accepted FP32 but got: dynamic
Check '(element_type_bias == ov::element::f32)' failed at src/frontends/onnx/frontend/src/op/com.microsoft/dynamic_quantize_matmul.cpp:73:
While validating ONNX node '<Node(DynamicQuantizeMatMul): /encoder/layer.0/output/dense/MatMul_quant>':
Unsupported input bias type, accepted FP32 but got: dynamic
Information
- Docker + cli
- pip + cli
- pip + usage of Python interface
Tasks
- An officially supported CLI command
- My own modifications
Reproduction
I get it when trying to start a container with following:
infinity:
image: michaelf34/infinity:latest-cpu
command: >
v2
--engine optimum
--model-id michaelfeil/bge-small-en-v1.5
--port ${PORT:-7997}
ports:
- "${PORT:-7997}:${PORT:-7997}"
volumes:
- "sail-infinity:/app/.cache"
networks:
- app-network