Problems with OpenVino Export

[phillipkusinski]

Hi guys,

im currently working on a anomaly detection application for my bachelorthesis. Im having trouble exporting my trained EfficientAD model to openvino. In the following you can see my export script and the generated failure message.

I'm using Python 3.11.9 and installed all the imports with pip install "anomalib[openvino]", pip install requests, pip install onnxscript

Code:
from anomalib.models import EfficientAd
from anomalib.engine import Engine

engine = Engine()

ckpt_path=r"C:\Users\ukusp003\Desktop\bachelorarbeit\5_Code\image_learning\image_anomaly_detection\anomalib_framework\checkpoints\EfficientAD_dataset_L_NO_SSR_1DB_20251127-152914\EfficientAD_dataset_L_NO_SSR_1DB_20251127-152914_epoch=29.ckpt"
model=EfficientAd.load_from_checkpoint(ckpt_path)

exported_paths=engine.export(model=model,
export_type="openvino",
export_root="./exported_models")

print(f"Export finsihed: {exported_paths}")

Errormsg:

ValueError Traceback (most recent call last)
File c:\Users\ukusp003\Desktop\bachelorarbeit\5_Code.venv_ANOMALIB\Lib\site-packages\torch\onnx_internal\exporter_compat.py:93, in export_compat(model, args, f, kwargs, export_params, verbose, input_names, output_names, opset_version, custom_translation_table, dynamic_axes, dynamic_shapes, keep_initializers_as_inputs, external_data, report, optimize, verify, profile, dump_exported_program, artifacts_dir, fallback, legacy_export_kwargs)
91 try:
92 dynamic_shapes, args, kwargs = (
---> 93 _dynamic_shapes.from_dynamic_axes_to_dynamic_shapes(
94 model,
95 args,
96 kwargs,
97 dynamic_axes=dynamic_axes,
98 input_names=input_names,
99 output_names=set(output_names or ()),
100 )
101 )
102 except Exception as e:

File c:\Users\ukusp003\Desktop\bachelorarbeit\5_Code.venv_ANOMALIB\Lib\site-packages\torch\onnx_internal\exporter_dynamic_shapes.py:95, in from_dynamic_axes_to_dynamic_shapes(model, args, kwargs, dynamic_axes, output_names, input_names)
94 # We need tree structure to represent dynamic_shapes
---> 95 dynamic_shapes = _unflatten_dynamic_shapes_with_inputs_tree(inputs, dynamic_shapes)
97 # Since the dynamic_shapes are now in the order of the model parameters,
98 # we need to convert args and kwargs to the order of the model parameters.

File c:\Users\ukusp003\Desktop\bachelorarbeit\5_Code.venv_ANOMALIB\Lib\site-packages\torch\onnx_internal\exporter_dynamic_shapes.py:307, in _unflatten_dynamic_shapes_with_inputs_tree(inputs, dynamic_shapes)
306 _, tree_structure = _pytree.tree_flatten(inputs)
--> 307 return _pytree.tree_unflatten(dynamic_shapes.values(), tree_structure)
...
110 _dynamic_shapes.convert_str_to_export_dim(dynamic_shapes)
111 )
113 if opset_version < _constants.TORCHLIB_OPSET:

RuntimeError: # Failed to convert 'dynamic_axes' to 'dynamic_shapes'. Please provide 'dynamic_shapes' directly. Refer to the documentation for 'torch.export.export' for more information on dynamic shapes.

[rajeshgangireddy]

Hi @phillipkusinski
This is because in torch.onnx.export(), dynamo is changed to True by default in torch 2.9.
Can you switch to Torch 2.8 and see if it solves?

I have a fix for this in #3155 - here.

[phillipkusinski]

I will try this and give you Feedback. Thank you in advance!

[phillipkusinski]

I tried it now with torch 2.8 and it worked. Thank you. Regarding to the inference topic I have another question. What is the common practice when working with this in an industrial environment?
Is it better to just load the saved images as an PredictDataset and inference with engine.predict() or is it better to export it to OpenVino and use the OpenVino inferencer?

I also got different results after predicting with Openvino and with the common engine.predict method. The anomalys where detect way stronger in the heatmap of the OV prediction in comparison to the other.

Image generated from openvino (look at the heatmap maximas)

Image generated from engine (look at the ....)

What is the reason for different Inference result when predicted with the Same Image and the Same Model checkpoint?