Skip to content

fix bug onnx export with nms. Onnx operator WHERE requires inputs of same type  #988

@edelah

Description

@edelah

Prerequisite

🐞 Describe the bug

There is a type mismatch in the inputs of /Where_1 node in onnx model graph model. The model can not be loaded.

Traceback (most recent call last):
File "/home/unknown/code/chessvideos/tests/test_onnx_raw.py", line 14, in
session = onnxruntime.InferenceSession(
File "/home/unknown/miniconda3/lib/python3.10/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py", line 347, in init
self._create_inference_session(providers, provider_options, disabled_optimizers)
File "/home/unknown/miniconda3/lib/python3.10/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py", line 384, in _create_inference_session
sess = C.InferenceSession(session_options, self._model_path, True, self._read_config_from_model)
onnxruntime.capi.onnxruntime_pybind11_state.Fail: [ONNXRuntimeError] : 1 : FAIL : Load model from /home/unknown/code/chessvideos/models/global/rtmdet_tiny_nms.onnx failed:Type Error: Type parameter (T) of Optype (Where) bound to different types (tensor(int64) and tensor(float) in node (/Where_1).

Environment

torch 2.1.0
cuda 12.1
mmengine>=0.6.0
mmcv 2.1.0
mmdet 3.3.0

Additional information

  • code used to export the model to onnx:

!python /content/mmyolo/projects/easydeploy/tools/export_onnx.py
/content/mmyolo/work_dirs/custom/20240413_200608/vis_data/config.py
/content/epoch_10.pth
--work-dir work_dirs/rtmdet
--img-size 640 640
--batch 1
--device cpu
--simplify
--pre-topk 1000
--keep-topk 100
--iou-threshold 0.80
--score-threshold 0.20

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions