RTDETRv2 to ONNX error

[Pawel842]

I try to convert the RTDETRv2 to onnx

paddle2onnx --model_dir rtdetrv2_r18vd_dsp_3x_coco \
            --model_filename model.pdmodel \
            --params_filename model.pdiparams \
            --opset_version 16 \
            --save_file rtdetr.onnx

(also tried with opset versions 13, 15,16,17,18) and
but I got following error

Aborted (core dumped)
root@a951431fc0df:/opt/ml/output/Combined_dataset_v3_with_test/model# paddle2onnx --model_dir rtdetrv2_r18vd_dsp_3x_coco             --model_filename model.pdmodel             --params_filename model.pdiparams             --opset_version 19             --save_file ppyoloes.onnx
/usr/local/lib/python3.12/dist-packages/paddle/base/framework.py:803: UserWarning: You are using GPU version Paddle, but your CUDA device is not set properly. CPU device will be used by default.
  warnings.warn(
2026-01-13 09:56:54 [WARNING]   The .pdmodel file is deprecated in paddlepaddle 3.0 and will be removed in the future. Try to convert from .pdmodel file to json file.
I0113 09:56:54.956501   117 program_interpreter.cc:242] New Executor is Running.
[Paddle2ONNX] Start parsing the Paddle model file...
[Paddle2ONNX]
[ERROR][Paddle2ONNX] Due to the unsupported operators, the conversion is aborted.
--------------------------------------
C++ Traceback (most recent call last):
--------------------------------------
0   paddle2onnx::Export(char const*, char const*, char**, int*, int, bool, bool, bool, bool, bool, paddle2onnx::CustomOp*, int, char const*, char**, int*, char const*, bool*, bool, char**, int)
----------------------
Error Message Summary:
----------------------
FatalError: Process abort signal is detected by the operating system.
  [TimeInfo: *** Aborted at 1768298215 (unix time) try "date -d @1768298215" if you are using GNU date ***]
  [SignalInfo: *** SIGABRT (@0x75) received by PID 117 (TID 0x7f5d383a4300) from PID 117 ***]
Aborted (core dumped)
root@a951431fc0df:/opt/ml/output/Combined_dataset_v3_with_test/model#

Informations (please complete the following information):

• Inference engine for deployment: nvcr.io/nvidia/paddlepaddle:24.02-py3
• Why convert to onnx：Use on different platform
• Paddle2ONNX Version: .2.1.0 (but also tested on 2.0.1 and 1.3.1
• Email/Wechat/Phone:

Could you help me solve this issue plz?

[alex-dinh]

Can you share the download link to the model files for RTDETRv2?

[Pawel842]

Can you share the download link to the model files for RTDETRv2?

@alex-dinh Sure.
All training output files are here
https://drive.google.com/file/d/1D5m4lGtGQ1ZM0bLzcD7FsG25oXDmQrku/view?usp=sharing

[alex-dinh]

I tried running the conversion with some extra logging, and looks like this model is not supported yet. The mapping for the tensor operation broadcast_tensors needs to be implemented in order for this model to be converted.

[Paddle2ONNX] Start parsing the Paddle model file...
[Paddle2ONNX] There are some ops not supported yet, including broadcast_tensors,
[ERROR][Paddle2ONNX] Due to the unsupported operators, the conversion is aborted.

[alex-dinh]

@Pawel842 I made some changes and was able to export the model as ONNX, but I'm not sure if it works. Can you test it out?

Download link: https://huggingface.co/alex-dinh/RTDETRv2-ONNX/resolve/main/rtdetrv2_r18vd_dsp_3x_coco.onnx?download=true

[Pawel842]

Thank you very much!! @alex-dinh.
I tried to load the model but I got following error

    raise RuntimeError(f"Failed to load ONNX model: {e}") from e
RuntimeError: Failed to load ONNX model: [ONNXRuntimeError] : 1 : FAIL : Node (Reshape.84) Op (Reshape) [ShapeInferenceError] Invalid Target shape product of 0. Product cannot be 0 in combination with -1

But i'm not sure if it's related to paddlepaddle export or onnx export :/

Could you share what changes are required to convert model to onnx?

btw.
I will be OOO for the next week, if any more data will be needed from my side I will response right after comeback.

[alex-dinh]

Hi @Pawel842, I'm wondering why you are trying to convert this model from Paddle to onnx? Especially since this model is not originally a Paddle model. I'd recommend trying the RT-DETR onnx models from this HF repo: https://huggingface.co/onnx-community/rtdetr_v2_r18vd-ONNX/tree/main

But i'm not sure if it's related to paddlepaddle export or onnx export :/

I'm not sure either but it may not even be necessary to use Paddle2ONNX for your use case, I think you'll have an easier time converting directly from Pytorch directly if a specific model you need hasn't been released as ONNX yet. See here: https://huggingface.co/PekingU/rtdetr_r18vd

[Pawel842]

Sure, that looks like a way to go, I will try and update the thread.