[Bug] Yoloxpose's onnx model reasoning effect is poor, and multiple bounding boxes appear in single-player scenes

[da13132]

Checklist

• I have searched related issues but cannot get the expected help.
• 2. I have read the FAQ documentation but cannot get the expected help.
• 3. The bug has not been fixed in the latest version.

Describe the bug

I try to deploy yoloposex using mmdeploy. When I convert yoloxpose-tiny and yoloxpose-small to the onnx model, the visualization results of the onnx model and pytorch are poor, and multiple bounding boxes appear in the single-person scene, while the visualization results of pytorch inference directly in mmpose are normal.This does not seem to be a problem with onnx itself, I hope to get your help.

During the conversion process, there are many prompts like this one: TracerWarning: Converting a tensor to a Python integer might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will btreated as a constant in the future. This means that the trace might not generalize to other inputs!

This is the visualization result of yoloposex-tiny in mmpose

This is the visualization result of yoloxpose-tiny after mmdeploy uses onnx export, including onnx visualization and pytorch visualization
（Visualization of onnx）

（Visualization of pytorch）

After using the onnx model of yoloxpose-small, the visualization result is slightly better

Reproduction

python tools/deploy.py /home/keli/download/mmdeploy/configs/mmpose/pose-detection_yolox-pose_onnxruntime_dynamic.py /home/keli/download/pose/projects/yolox-pose/configs/yolox-pose_s_8xb32-300e_coco.py /home/keli/download/pose/pthfile/yoloxpose/yolox-pose_s_8xb32-300e_coco-9f5e3924_20230321.pth demo/resources/human-pose.jpg --work-dir yoloxpose_onnx/yoloxpose_small --device cpu --show --dump-info

#pose-detection_yolox-pose_onnxruntime_dynamic.py
base = ['./pose-detection_static.py', '../base/backends/onnxruntime.py']
onnx_config = dict(
output_names=['dets', 'keypoints'],
dynamic_axes={
'input': {
0: 'batch',
},
'dets': {
0: 'batch',
},
'keypoints': {
0: 'batch'
}
})
codebase_config = dict(
post_processing=dict(
score_threshold=0.01,
iou_threshold=0.5,
max_output_boxes_per_class=200,
pre_top_k=5000,
keep_top_k=100,
background_label_id=-1,
))

Environment

08/28 21:01:09 - mmengine - INFO - **********Environmental information**********
08/28 21:01:10 - mmengine - INFO - sys.platform: linux
08/28 21:01:10 - mmengine - INFO - Python: 3.9.17 (main, Jul  5 2023, 20:41:20) [GCC 11.2.0]
08/28 21:01:10 - mmengine - INFO - CUDA available: True
08/28 21:01:10 - mmengine - INFO - numpy_random_seed: 2147483648
08/28 21:01:10 - mmengine - INFO - GPU 0: NVIDIA GeForce RTX 3090
08/28 21:01:10 - mmengine - INFO - CUDA_HOME: /usr/local/cuda-11.6
08/28 21:01:10 - mmengine - INFO - NVCC: Cuda compilation tools, release 11.6, V11.6.124
08/28 21:01:10 - mmengine - INFO - GCC: gcc (Ubuntu 9.4.0-1ubuntu1~20.04.1) 9.4.0
08/28 21:01:10 - mmengine - INFO - PyTorch: 1.13.1+cu116
08/28 21:01:10 - mmengine - INFO - PyTorch compiling details: PyTorch built with:
  - GCC 9.3
  - C++ Version: 201402
  - Intel(R) Math Kernel Library Version 2020.0.0 Product Build 20191122 for Intel(R) 64 architecture applications
  - Intel(R) MKL-DNN v2.6.0 (Git Hash 52b5f107dd9cf10910aaa19cb47f3abf9b349815)
  - OpenMP 201511 (a.k.a. OpenMP 4.5)
  - LAPACK is enabled (usually provided by MKL)
  - NNPACK is enabled
  - CPU capability usage: AVX2
  - CUDA Runtime 11.6
  - NVCC architecture flags: -gencode;arch=compute_37,code=sm_37;-gencode;arch=compute_50,code=sm_50;-gencode;arch=compute_60,code=sm_60;-gencode;arch=compute_70,code=sm_70;-gencode;arch=compute_75,code=sm_75;-gencode;arch=compute_80,code=sm_80;-gencode;arch=compute_86,code=sm_86
  - CuDNN 8.5  (built against CUDA 11.7)
    - Built with CuDNN 8.3.2
  - Magma 2.6.1
  - Build settings: BLAS_INFO=mkl, BUILD_TYPE=Release, CUDA_VERSION=11.6, CUDNN_VERSION=8.3.2, CXX_COMPILER=/opt/rh/devtoolset-9/root/usr/bin/c++, CXX_FLAGS= -fabi-version=11 -Wno-deprecated -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -fopenmp -DNDEBUG -DUSE_KINETO -DUSE_FBGEMM -DUSE_QNNPACK -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -DSYMBOLICATE_MOBILE_DEBUG_HANDLE -DEDGE_PROFILER_USE_KINETO -O2 -fPIC -Wno-narrowing -Wall -Wextra -Werror=return-type -Werror=non-virtual-dtor -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wunused-local-typedefs -Wno-unused-parameter -Wno-unused-function -Wno-unused-result -Wno-strict-overflow -Wno-strict-aliasing -Wno-error=deprecated-declarations -Wno-stringop-overflow -Wno-psabi -Wno-error=pedantic -Wno-error=redundant-decls -Wno-error=old-style-cast -fdiagnostics-color=always -faligned-new -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Werror=cast-function-type -Wno-stringop-overflow, LAPACK_INFO=mkl, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, TORCH_VERSION=1.13.1, USE_CUDA=ON, USE_CUDNN=ON, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_MKL=ON, USE_MKLDNN=ON, USE_MPI=OFF, USE_NCCL=ON, USE_NNPACK=ON, USE_OPENMP=ON, USE_ROCM=OFF, 

08/28 21:01:10 - mmengine - INFO - TorchVision: 0.14.1+cu116
08/28 21:01:10 - mmengine - INFO - OpenCV: 4.8.0
08/28 21:01:10 - mmengine - INFO - MMEngine: 0.8.2
08/28 21:01:10 - mmengine - INFO - MMCV: 2.0.1
08/28 21:01:10 - mmengine - INFO - MMCV Compiler: GCC 9.3
08/28 21:01:10 - mmengine - INFO - MMCV CUDA Compiler: 11.6
08/28 21:01:10 - mmengine - INFO - MMDeploy: 1.2.0+
08/28 21:01:10 - mmengine - INFO - 

08/28 21:01:10 - mmengine - INFO - **********Backend information**********
08/28 21:01:10 - mmengine - INFO - tensorrt:    8.5.3.1
08/28 21:01:10 - mmengine - INFO - tensorrt custom ops: Available
08/28 21:01:10 - mmengine - INFO - ONNXRuntime: 1.8.1
08/28 21:01:10 - mmengine - INFO - ONNXRuntime-gpu:     None
08/28 21:01:10 - mmengine - INFO - ONNXRuntime custom ops:      Available
08/28 21:01:10 - mmengine - INFO - pplnn:       None
08/28 21:01:10 - mmengine - INFO - ncnn:        1.0.20230808
08/28 21:01:10 - mmengine - INFO - ncnn custom ops:     Available
08/28 21:01:10 - mmengine - INFO - snpe:        None
08/28 21:01:10 - mmengine - INFO - openvino:    None
08/28 21:01:10 - mmengine - INFO - torchscript: 1.13.1+cu116
08/28 21:01:10 - mmengine - INFO - torchscript custom ops:      NotAvailable
08/28 21:01:10 - mmengine - INFO - rknn-toolkit:        None
08/28 21:01:10 - mmengine - INFO - rknn-toolkit2:       None
08/28 21:01:10 - mmengine - INFO - ascend:      None
08/28 21:01:10 - mmengine - INFO - coreml:      None
08/28 21:01:10 - mmengine - INFO - tvm: None
08/28 21:01:10 - mmengine - INFO - vacc:        None
08/28 21:01:10 - mmengine - INFO - 

08/28 21:01:10 - mmengine - INFO - **********Codebase information**********
08/28 21:01:10 - mmengine - INFO - mmdet:       3.0.0
08/28 21:01:10 - mmengine - INFO - mmseg:       None
08/28 21:01:10 - mmengine - INFO - mmpretrain:  1.0.0rc8
08/28 21:01:10 - mmengine - INFO - mmocr:       None
08/28 21:01:10 - mmengine - INFO - mmagic:      None
08/28 21:01:10 - mmengine - INFO - mmdet3d:     None
08/28 21:01:10 - mmengine - INFO - mmpose:      1.0.0
08/28 21:01:10 - mmengine - INFO - mmrotate:    None
08/28 21:01:10 - mmengine - INFO - mmaction:    None
08/28 21:01:10 - mmengine - INFO - mmrazor:     None
08/28 21:01:10 - mmengine - INFO - mmyolo:      0.6.0

Error traceback

08/28 20:49:18 - mmengine - WARNING - Failed to search registry with scope "mmpose" in the "Codebases" registry tree. As a workaround, the current "Codebases" registry in "mmdeploy" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "mmpose" is a correct scope, or whether the registry is initialized.
08/28 20:49:18 - mmengine - WARNING - Failed to search registry with scope "mmpose" in the "mmpose_tasks" registry tree. As a workaround, the current "mmpose_tasks" registry in "mmdeploy" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "mmpose" is a correct scope, or whether the registry is initialized.
08/28 20:49:18 - mmengine - WARNING - Failed to search registry with scope "mmyolo" in the "Codebases" registry tree. As a workaround, the current "Codebases" registry in "mmdeploy" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "mmyolo" is a correct scope, or whether the registry is initialized.
/home/keli/下载/pose/mmpose/utils/setup_env.py:79: UserWarning: The current default scope "mmyolo" is not "mmpose", `register_all_modules` will force the currentdefault scope to be "mmpose". If this is not expected, please set `init_default_scope=Fse`.
  warnings.warn('The current default scope '
08/28 20:49:19 - mmengine - INFO - Start pipeline mmdeploy.apis.pytorch2onnx.torch2onnx in subprocess
08/28 20:49:20 - mmengine - WARNING - Failed to search registry with scope "mmpose" in the "Codebases" registry tree. As a workaround, the current "Codebases" registry in "mmdeploy" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "mmpose" is a correct scope, or whether the registry is initialized.
08/28 20:49:20 - mmengine - WARNING - Failed to search registry with scope "mmpose" in the "mmpose_tasks" registry tree. As a workaround, the current "mmpose_tasks" registry in "mmdeploy" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "mmpose" is a correct scope, or whether the registry is initialized.
/home/keli/下载/pose/mmpose/datasets/datasets/utils.py:102: UserWarning: The metainfo config file "configs/_base_/datasets/coco.py" does not exist. A matched config file "/home/keli/下载/pose/mmpose/.mim/configs/_base_/datasets/coco.py" will be usestead.
  warnings.warn(
Loads checkpoint by local backend from path: /home/keli/下载/pose/pthfile/yoloxpose/yolox-pose_s_8xb32-300e_coco-9f5e3924_20230321.pth
The model and loaded state dict do not match exactly

missing keys in source state_dict: bbox_head.assigner_oks_calculator.sigmas

08/28 20:49:20 - mmengine - WARNING - "FileClient" will be deprecated in future. Please use io functions in https://mmengine.readthedocs.io/en/latest/api/fileio.html#file-io
08/28 20:49:20 - mmengine - WARNING - "HardDiskBackend" is the alias of "LocalBackend" and the former will be deprecated in future.
08/28 20:49:20 - mmengine - WARNING - DeprecationWarning: get_onnx_config will be deprecated in the future. 
08/28 20:49:20 - mmengine - INFO - Export PyTorch model to ONNX: yoloxpose_onnx/yoloxpose_small/end2end.onnx.
/home/keli/下载/mmdeploy/mmdeploy/codebase/mmdet/models/detectors/single_stage.py:80: TracerWarning: Iterating over a tensor might cause the trace to be incorrect. Passing a tensor of different shape won't change the number of iterations executed (d might lead to errors or silently give incorrect results).
  img_shape = [int(val) for val in img_shape]
/home/keli/下载/mmdeploy/mmdeploy/codebase/mmdet/models/detectors/single_stage.py:80: TracerWarning: Converting a tensor to a Python integer might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will btreated as a constant in the future. This means that the trace might not generalize to other inputs!
  img_shape = [int(val) for val in img_shape]
/home/keli/下载/mmdeploy/mmdeploy/core/optimizers/function_marker.py:160: TracerWarning: Converting a tensor to a Python integer might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as constant in the future. This means that the trace might not generalize to other inputs!
  ys_shape = tuple(int(s) for s in ys.shape)
/home/keli/anaconda3/envs/mmdeploy/lib/python3.9/site-packages/torch/functional.py:504: UserWarning: torch.meshgrid: in an upcoming release, it will be required to pass the indexing argument. (Triggered internally at ../aten/src/ATen/native/TensorShape.cpp:3190.)
  return _VF.meshgrid(tensors, **kwargs)  # type: ignore[attr-defined]
/home/keli/下载/mmdeploy/mmdeploy/mmcv/ops/nms.py:285: TracerWarning: torch.tensor results are registered as constants in the trace. You can safely ignore this warning if you use this function to create tensors out of constant variables that would  the same every time you call this function. In any other case, this might cause the trace to be incorrect.
  iou_threshold = torch.tensor([iou_threshold], dtype=torch.float32)
/home/keli/下载/mmdeploy/mmdeploy/mmcv/ops/nms.py:286: TracerWarning: torch.tensor results are registered as constants in the trace. You can safely ignore this warning if you use this function to create tensors out of constant variables that would  the same every time you call this function. In any other case, this might cause the trace to be incorrect.
  score_threshold = torch.tensor([score_threshold], dtype=torch.float32)
/home/keli/下载/mmdeploy/mmdeploy/pytorch/functions/topk.py:28: TracerWarning: torch.tensor results are registered as constants in the trace. You can safely ignore this warning if you use this function to create tensors out of constant variables th would be the same every time you call this function. In any other case, this might cause the trace to be incorrect.
  k = torch.tensor(k, device=input.device, dtype=torch.long)
/home/keli/下载/mmdeploy/mmdeploy/mmcv/ops/nms.py:44: TracerWarning: Converting a tensor to a Python float might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the fute. This means that the trace might not generalize to other inputs!
  score_threshold = float(score_threshold)
/home/keli/下载/mmdeploy/mmdeploy/mmcv/ops/nms.py:45: TracerWarning: Converting a tensor to a Python float might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the fute. This means that the trace might not generalize to other inputs!
  iou_threshold = float(iou_threshold)
/home/keli/anaconda3/envs/mmdeploy/lib/python3.9/site-packages/mmcv/ops/nms.py:123: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  assert boxes.size(1) == 4
/home/keli/anaconda3/envs/mmdeploy/lib/python3.9/site-packages/mmcv/ops/nms.py:124: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  assert boxes.size(0) == scores.size(0)
/home/keli/anaconda3/envs/mmdeploy/lib/python3.9/site-packages/torch/onnx/_internal/jit_utils.py:258: UserWarning: The shape inference of prim::Constant type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function. (Triggered internally at ../torch/csrc/jit/passes/onnx/shape_type_inference.cpp:1884.)
  _C._jit_pass_onnx_node_shape_type_inference(node, params_dict, opset_version)
/home/keli/anaconda3/envs/mmdeploy/lib/python3.9/site-packages/torch/onnx/symbolic_opset9.py:5408: UserWarning: Exporting aten::index operator of advanced indexing in opset 14 is achieved by combination of multiple ONNX operators, including Reshape, Transpose, Concat, and Gather. If indices include negative values, the exported graph will produce incorrect results.
  warnings.warn(
/home/keli/下载/mmdeploy/mmdeploy/mmcv/ops/nms.py:86: FutureWarning: 'torch.onnx._patch_torch._graph_op' is deprecated in version 1.13 and will be removed in version 1.14. Please note 'g.op()' is to be removed from torch.Graph. Please open a GitHubssue if you need this functionality..
  max_output_boxes_per_class = g.op(
/home/keli/下载/mmdeploy/mmdeploy/mmcv/ops/nms.py:100: FutureWarning: 'torch.onnx._patch_torch._graph_op' is deprecated in version 1.13 and will be removed in version 1.14. Please note 'g.op()' is to be removed from torch.Graph. Please open a GitHuissue if you need this functionality..
  return g.op('NonMaxSuppression', boxes, scores,
/home/keli/anaconda3/envs/mmdeploy/lib/python3.9/site-packages/torch/onnx/utils.py:687: UserWarning: The shape inference of prim::Constant type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function. (Triggered internally at ../torch/csrc/jit/passes/onnx/shape_type_inference.cpp:1884.)
  _C._jit_pass_onnx_graph_shape_type_inference(
/home/keli/anaconda3/envs/mmdeploy/lib/python3.9/site-packages/torch/onnx/utils.py:1178: UserWarning: The shape inference of prim::Constant type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function. (Triggered internally at ../torch/csrc/jit/passes/onnx/shape_type_inference.cpp:1884.)
  _C._jit_pass_onnx_graph_shape_type_inference(
08/28 20:49:23 - mmengine - INFO - Execute onnx optimize passes.
08/28 20:49:23 - mmengine - INFO - Finish pipeline mmdeploy.apis.pytorch2onnx.torch2onnx
08/28 20:49:24 - mmengine - INFO - Start pipeline mmdeploy.apis.utils.utils.to_backend in main process
08/28 20:49:24 - mmengine - INFO - Finish pipeline mmdeploy.apis.utils.utils.to_backend
08/28 20:49:24 - mmengine - INFO - visualize onnxruntime model start.
08/28 20:49:26 - mmengine - WARNING - Failed to search registry with scope "mmpose" in the "Codebases" registry tree. As a workaround, the current "Codebases" registry in "mmdeploy" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "mmpose" is a correct scope, or whether the registry is initialized.
08/28 20:49:26 - mmengine - WARNING - Failed to search registry with scope "mmpose" in the "mmpose_tasks" registry tree. As a workaround, the current "mmpose_tasks" registry in "mmdeploy" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "mmpose" is a correct scope, or whether the registry is initialized.
08/28 20:49:26 - mmengine - WARNING - Failed to search registry with scope "mmyolo" in the "backend_segmentors" registry tree. As a workaround, the current "backend_segmentors" registry in "mmdeploy" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "mmyolo" is a correct scope, or whether the registry is initialized.
08/28 20:49:26 - mmengine - INFO - Successfully loaded onnxruntime custom ops from /home/keli/下载/mmdeploy/mmdeploy/lib/libmmdeploy_onnxruntime_ops.so
/home/keli/下载/pose/mmpose/datasets/datasets/utils.py:102: UserWarning: The metainfo config file "configs/_base_/datasets/coco.py" does not exist. A matched config file "/home/keli/下载/pose/mmpose/.mim/configs/_base_/datasets/coco.py" will be usestead.
  warnings.warn(
08/28 20:49:26 - mmengine - WARNING - "FileClient" will be deprecated in future. Please use io functions in https://mmengine.readthedocs.io/en/latest/api/fileio.html#file-io
08/28 20:49:26 - mmengine - WARNING - "HardDiskBackend" is the alias of "LocalBackend" and the former will be deprecated in future.
/home/keli/下载/mmdeploy/mmdeploy/codebase/mmpose/deploy/pose_detection_model.py:195: UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.clone().detach().requires_grad_(True), ratherhan tensor.new_tensor(sourceTensor).
  keypoints /= keypoints.new_tensor(scale_factor).reshape(1, 1, 2)
/home/keli/下载/mmdeploy/mmdeploy/codebase/mmpose/deploy/pose_detection_model.py:196: UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.clone().detach().requires_grad_(True), ratherhan tensor.new_tensor(sourceTensor).
  bboxes /= keypoints.new_tensor(scale_factor).repeat(1, 2)
08/28 20:49:26 - mmengine - WARNING - render and display result skipped for headless device, exception no display name and no $DISPLAY environment variable
08/28 20:49:27 - mmengine - INFO - visualize onnxruntime model success.
08/28 20:49:27 - mmengine - INFO - visualize pytorch model start.
08/28 20:49:29 - mmengine - WARNING - Failed to search registry with scope "mmpose" in the "Codebases" registry tree. As a workaround, the current "Codebases" registry in "mmdeploy" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "mmpose" is a correct scope, or whether the registry is initialized.
08/28 20:49:29 - mmengine - WARNING - Failed to search registry with scope "mmpose" in the "mmpose_tasks" registry tree. As a workaround, the current "mmpose_tasks" registry in "mmdeploy" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "mmpose" is a correct scope, or whether the registry is initialized.
/home/keli/下载/pose/mmpose/datasets/datasets/utils.py:102: UserWarning: The metainfo config file "configs/_base_/datasets/coco.py" does not exist. A matched config file "/home/keli/下载/pose/mmpose/.mim/configs/_base_/datasets/coco.py" will be usestead.
  warnings.warn(
Loads checkpoint by local backend from path: /home/keli/下载/pose/pthfile/yoloxpose/yolox-pose_s_8xb32-300e_coco-9f5e3924_20230321.pth
The model and loaded state dict do not match exactly

missing keys in source state_dict: bbox_head.assigner_oks_calculator.sigmas

08/28 20:49:29 - mmengine - WARNING - "FileClient" will be deprecated in future. Please use io functions in https://mmengine.readthedocs.io/en/latest/api/fileio.html#file-io
08/28 20:49:29 - mmengine - WARNING - "HardDiskBackend" is the alias of "LocalBackend" and the former will be deprecated in future.
/home/keli/anaconda3/envs/mmdeploy/lib/python3.9/site-packages/torch/functional.py:504: UserWarning: torch.meshgrid: in an upcoming release, it will be required to pass the indexing argument. (Triggered internally at ../aten/src/ATen/native/TensorShape.cpp:3190.)
  return _VF.meshgrid(tensors, **kwargs)  # type: ignore[attr-defined]
08/28 20:49:29 - mmengine - WARNING - render and display result skipped for headless device, exception no display name and no $DISPLAY environment variable
08/28 20:49:30 - mmengine - INFO - visualize pytorch model success.
08/28 20:49:30 - mmengine - INFO - All process success.

[RunningLeon]

hi, the visualization results are aligned between pytorch model and onnxruntime, which means the conversion is good. You can also compare the evaluation results if possible. You can set a threshold and filter out boxes and keypoints.

[da13132]

hi, the visualization results are aligned between pytorch model and onnxruntime, which means the conversion is good. You can also compare the evaluation results if possible. You can set a threshold and filter out boxes and keypoints.

In fact, I simply modified the code about configuration file data preprocessing in mmdeploy/codebase/mmpose/deploy/pose_detection.py
I shall

 test_pipeline.append(
             dict(
                 type='Normalize',
                 mean=data_preprocessor.mean,
                 std=data_preprocessor.std,
                 to_rgb=data_preprocessor.bgr_to_rgb))

change into

if 'mean' and 'std' and 'bgr_to_rgb' in data_preprocessor:
         test_pipeline.append(
             dict(
                 type='Normalize',
                 mean=data_preprocessor.mean,
                 std=data_preprocessor.std,
                 to_rgb=data_preprocessor.bgr_to_rgb))
     else:
         test_pipeline.append(
             dict(
                 type='None',
                 mean=None,
                 std=None,
                 to_rgb=None))

In this way, yoloxpose can be deployed normally. If the code here is not modified by default, yoloxpose will report an error when converting the onnx format

Traceback (most recent call last):
   File "/home/keli/下载/mmdeploy/tools/deploy.py", line 335, in <module>
     main()
   File "/home/keli/下载/mmdeploy/tools/deploy.py", line 129, in main
     export2SDK(
   File "/home/keli/下载/mmdeploy/mmdeploy/backend/sdk/export_info.py", line 353, in export2SDK
     pipeline_info = get_pipeline(deploy_cfg, model_cfg, work_dir, device)
   File "/home/keli/下载/mmdeploy/mmdeploy/backend/sdk/export_info.py", line 286, in get_pipeline
     preprocess = get_preprocess(deploy_cfg, model_cfg, device=device)
   File "/home/keli/下载/mmdeploy/mmdeploy/backend/sdk/export_info.py", line 197, in get_preprocess
     transforms = task_processor.get_preprocess()
   File "/home/keli/下载/mmdeploy/mmdeploy/codebase/mmpose/deploy/pose_detection.py", line 342, in get_preprocess
     model_cfg = process_model_config(self.model_cfg, [''], input_shape)
   File "/home/keli/下载/mmdeploy/mmdeploy/codebase/mmpose/deploy/pose_detection.py", line 60, in process_model_config
     mean=data_preprocessor.mean,
   File "/home/keli/anaconda3/envs/mmdeploy/lib/python3.9/site-packages/mmengine/config/config.py", line 106, in __getattr__
     raise AttributeError(f"'{self.__class__.__name__}' object has no "
AttributeError: 'ConfigDict' object has no attribute 'mean'

This shows that the data_preprocessor in the yolox configuration file does not seem to be supported by the deployment of mmpose. I am thinking whether the lack of this preprocessing leads to the offset of the onnx inference results.

[RunningLeon]

@da13132 Hi，yolox-pose is supported in sdk by PR #2240, which is not mered yet. so when you run tools/deploy.py, you should not include argument --dump-info.

[da13132]

@da13132 Hi，yolox-pose is supported in sdk by PRhttps://github.com//pull/2240, which is not mered yet. so when you run tools/deploy.py, you should not include argument --dump-info.

Thank you for your reply. When I remove the command --dump-info, the visual inference results have not changed, and multiple bounding boxes still appear at the same time.

[RunningLeon]

hi, you need to set threshold as mentioned before.

hi, the visualization results are aligned between pytorch model and onnxruntime, which means the conversion is good. You can also compare the evaluation results if possible. You can set a threshold and filter out boxes and keypoints.

This is onnxruntime result on my machine with same command for small model and the evaluation results are aligned with pytorch and onnxruntime.

[da13132]

hi, you need to set threshold as mentioned before.

hi, the visualization results are aligned between pytorch model and onnxruntime, which means the conversion is good. You can also compare the evaluation results if possible. You can set a threshold and filter out boxes and keypoints.

This is onnxruntime result on my machine with same command for small model and the evaluation results are aligned with pytorch and onnxruntime.

Thanks for your reply, I tried modifying the threshold to be close to the training configuration

codebase_config = dict(
     post_processing=dict(
         score_threshold=0.001,
         iou_threshold=0.65,
         max_output_boxes_per_class=300,
         pre_top_k=5000,
         keep_top_k=100,
         background_label_id=-1,
     ))

This still didn't change, when I tried max_output_boxes_per_class=1, only the most accurate bounding box was left

[da13132]

hi, you need to set threshold as mentioned before.

hi, the visualization results are aligned between pytorch model and onnxruntime, which means the conversion is good. You can also compare the evaluation results if possible. You can set a threshold and filter out boxes and keypoints.

This is onnxruntime result on my machine with same command for small model and the evaluation results are aligned with pytorch and onnxruntime.

When I tried testing using a multiplayer scenario, yolopose-s and yolopose-t showed similar results. It seems codebase_config =

dict(
     post_processing=dict(
         score_threshold=0.01,
         iou_threshold=0.65,
         max_output_boxes_per_class=100,
         pre_top_k=5000,
         keep_top_k=100,
         background_label_id=-1,
     ))

The values ​​of score_threshold and iou_threshold have minimal impact on the results. I tried to expand them, but the test results did not change.

yolopose-t

yolopose-s

[RunningLeon]

hi, I mean threshhold for bboxes or keypoints, jut like the following codes( could change to bbox_scores > 0.3 # for example)

mmdeploy/mmdeploy/codebase/mmpose/deploy/pose_detection_model.py

Line 185 in 123b9fc

inds = bbox_scores > 0.0

hi, the visualization results are aligned between pytorch model and onnxruntime, which means the conversion is good. You can also compare the evaluation results if possible. You can set a threshold and filter out boxes and keypoints.

[da13132]

hi, I mean threshhold for bboxes or keypoints, jut like the following codes( could change to bbox_scores > 0.3 # for example)

mmdeploy/mmdeploy/codebase/mmpose/deploy/pose_detection_model.py

Line 185 in 123b9fc

inds = bbox_scores > 0.0

hi, the visualization results are aligned between pytorch model and onnxruntime, which means the conversion is good. You can also compare the evaluation results if possible. You can set a threshold and filter out boxes and keypoints.

Thanks for your help, this is useful

[github-actions]

This issue is marked as stale because it has been marked as invalid or awaiting response for 7 days without any further response. It will be closed in 5 days if the stale label is not removed or if there is no further response.