[Bug] jetson上安装mmdeploy的意义是？运行需要mmdet，mmdet需要mmengine，但安装只能到0.4.0，源码编译又有MMLogger的问题

[vxgu86]

Checklist

• I have searched related issues but cannot get the expected help.
• 2. I have read the FAQ documentation but cannot get the expected help.
• 3. The bug has not been fixed in the latest version.

Describe the bug

想请问下 jetson上安装mmdeploy的意义是什么，是可以在jetson上转换模型吗？
在jetson上安装mmdeploy后
运行需要mmdet，mmdet需要mmengine，但安装只能到0.4.0，源码编译又有MMLogger的错误。

Reproduction

就是 mmdeploy.py的指令

Environment

按jetson上部署的指令安装的
MMLogger提示错误

python ./tools/deploy.py     configs/mmdet/detection/detection_tensorrt_dynamic-64x64-608x608.py     /home/user/mmdeploy_ws/deploypth/configs/rtmdet/rtmdet_l_8xb32-300e_coco.py     /home/user/mmdeploy_ws/deploypth/epoch_300.pth     /home/user/mmdeploy_ws/deploypth/mil_sea_renchuan_kr_TV_014952_20230403131233319_visible.JPG     --work-dir ../deploypth     --device cuda:0     --show
Traceback (most recent call last):
  File "./tools/deploy.py", line 335, in <module>
    main()
  File "./tools/deploy.py", line 108, in main
    logger.setLevel(log_level)
  File "/home/user/mmdeploy_ws/mmengine/mmengine/logging/logger.py", line 323, in setLevel
    logger._cache.clear()
AttributeError: 'MMLogger' object has no attribute '_cache'

Error traceback

No response

[RunningLeon]

hi, pls check if this PR is helpful to you: #2587

[yinfan98]

Hi, The reason you need to build this package is that you can utilize it for model transformation as well as inference using its API.

[yinfan98]

You mentioned

mmdet requires mmengine, but installs only up to 0.4.0

But I follow the guide and it install the mmengine 0.8.0.
I wonder if you compiled the mmdeploy converter? If I remember correctly it installs mmengine at follow step.

Install Model Converter

# build TensorRT custom operators
mkdir -p build && cd build
cmake .. -DMMDEPLOY_TARGET_BACKENDS="trt"
make -j$(nproc) && make install

# install model converter
cd ${MMDEPLOY_DIR}
pip install -v -e .
# "-v" means verbose, or more output
# "-e" means installing a project in editable mode,
# thus any local modifications made to the code will take effect without re-installation.

[ShihaoHan19980609]

我把这行代码注销掉了，不影响正常运行

[613B]

Hi.

If you don't need inference result for test image, it is easily to convert model on your training machine, get the onnx model, and deploy TensorRT model from onnx model on your jetson by trtexec. This way does not require mmdeploy to be installed on Jetson.

Thus, the steps are below.

On your host traning machine (that mmdeploy installed).

python ./tools/deploy.py     configs/mmdet/detection/detection_tensorrt_dynamic-64x64-608x608.py     /home/user/mmdeploy_ws/deploypth/configs/rtmdet/rtmdet_l_8xb32-300e_coco.py     /home/user/mmdeploy_ws/deploypth/epoch_300.pth     /home/user/mmdeploy_ws/deploypth/mil_sea_renchuan_kr_TV_014952_20230403131233319_visible.JPG     --work-dir ../deploypth     --device cuda:0     --show

and, you'll get ${work-dir}/end2end.onnx. (I recommend using tools/torch2onnx.py instead of tools/deploy.py to avoid the process deploying ${work-dir}/end2end.engine)

Next, place work-dir/end2end.onnx to your jetson.

On your jetson

trtexec --onnx=end2end.onnx --saveEngine=end2end.engine

Additional options may be needed for dynamic shapes, quantization and so on.

This way will be much easier. But I'm not sure if it fits your use case.

想请问下 jetson上安装mmdeploy的意义是什么

In my opinion, it helps us to avoid complex configuration about calibration when int8 quantization is needed. Also, it's nice that we can get images infer on jetson immediately.

I hope this information may help you. And I would appreciate it if you could point out any mistakes.

[github-actions]

This issue is marked as stale because it has been marked as invalid or awaiting response for 7 days without any further response. It will be closed in 5 days if the stale label is not removed or if there is no further response.

[github-actions]

This issue is closed because it has been stale for 5 days. Please open a new issue if you have similar issues or you have any new updates now.

[vxgu86]

Hi.

If you don't need inference result for test image, it is easily to convert model on your training machine, get the onnx model, and deploy TensorRT model from onnx model on your jetson by trtexec. This way does not require mmdeploy to be installed on Jetson.

Thus, the steps are below.

On your host traning machine (that mmdeploy installed).

python ./tools/deploy.py     configs/mmdet/detection/detection_tensorrt_dynamic-64x64-608x608.py     /home/user/mmdeploy_ws/deploypth/configs/rtmdet/rtmdet_l_8xb32-300e_coco.py     /home/user/mmdeploy_ws/deploypth/epoch_300.pth     /home/user/mmdeploy_ws/deploypth/mil_sea_renchuan_kr_TV_014952_20230403131233319_visible.JPG     --work-dir ../deploypth     --device cuda:0     --show

and, you'll get ${work-dir}/end2end.onnx. (I recommend using tools/torch2onnx.py instead of tools/deploy.py to avoid the process deploying ${work-dir}/end2end.engine)

Next, place work-dir/end2end.onnx to your jetson.

On your jetson

trtexec --onnx=end2end.onnx --saveEngine=end2end.engine

Additional options may be needed for dynamic shapes, quantization and so on.

This way will be much easier. But I'm not sure if it fits your use case.

想请问下 jetson上安装mmdeploy的意义是什么

In my opinion, it helps us to avoid complex configuration about calibration when int8 quantization is needed. Also, it's nice that we can get images infer on jetson immediately.

I hope this information may help you. And I would appreciate it if you could point out any mistakes.

@613B Hi, thanks for your reply and sorry for my late
I tried your method with
/usr/src/tensorrt/bin/trtexec --onnx=/home/end2end.onnx --saveEngine=/home/work_dir/end.engine
but get error as follows

`[01/04/2024-01:13:15] [W] [TRT] onnx2trt_utils.cpp:366: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
[01/04/2024-01:13:17] [I] [TRT] No importer registered for op: TRTBatchedNMS. Attempting to import as plugin.
[01/04/2024-01:13:17] [I] [TRT] Searching for plugin: TRTBatchedNMS, plugin_version: 1, plugin_namespace:
[01/04/2024-01:13:17] [E] [TRT] ModelImporter.cpp:773: While parsing node number 626 [TRTBatchedNMS -> "/TRTBatchedNMS_output_0"]:
[01/04/2024-01:13:17] [E] [TRT] ModelImporter.cpp:774: --- Begin node ---
[01/04/2024-01:13:17] [E] [TRT] ModelImporter.cpp:775: input: "/Unsqueeze_11_output_0"
input: "/Sigmoid_output_0"
output: "/TRTBatchedNMS_output_0"
output: "/TRTBatchedNMS_output_1"
name: "/TRTBatchedNMS"
op_type: "TRTBatchedNMS"
attribute {
name: "background_label_id"
i: -1
type: INT
}
attribute {
name: "clip_boxes"
i: 0
type: INT
}
attribute {
name: "iou_threshold"
f: 0.65
type: FLOAT
}
attribute {
name: "is_normalized"
i: 0
type: INT
}
attribute {
name: "keep_topk"
i: 200
type: INT
}
attribute {
name: "num_classes"
i: 24
type: INT
}
attribute {
name: "return_index"
i: 0
type: INT
}
attribute {
name: "score_threshold"
f: 0.001
type: FLOAT
}
attribute {
name: "topk"
i: 5000
type: INT
}
domain: "mmdeploy"

[01/04/2024-01:13:17] [E] [TRT] ModelImporter.cpp:776: --- End node ---
[01/04/2024-01:13:17] [E] [TRT] ModelImporter.cpp:779: ERROR: builtin_op_importers.cpp:4870 In function importFallbackPluginImporter:
[8] Assertion failed: creator && "Plugin not found, are the plugin name, version, and namespace correct?"
[01/04/2024-01:13:17] [E] Failed to parse onnx file
[01/04/2024-01:13:17] [I] Finish parsing network model
[01/04/2024-01:13:17] [E] Parsing model failed
[01/04/2024-01:13:17] [E] Failed to create engine from model.
[01/04/2024-01:13:17] [E] Engine set up failed
`
I rechecked the operations for TensorRT custom operators as follows:

build TensorRT custom operators

mkdir -p build && cd build
cmake .. -DMMDEPLOY_TARGET_BACKENDS="trt"
make -j$(nproc) && make install

what is the problem?

[HEIseYOUmolc]

bro ，do you have more efficient ways in jetson nano？I have try the onnxruntime and tensorrt to deploy my model ，but always have problem………

onnxruntime

[2024-01-06 19:15:19.539] [mmdeploy] [info] [model.cpp:35] [DirectoryModel] Load model: "/home/nvidia/文档/mmdeploy_models/rtdetr1"
[2024-01-06 19:15:19.804] [mmdeploy] [error] [resize.cpp:84] unsupported interpolation method: bicubic
[2024-01-06 19:15:19.804] [mmdeploy] [error] [task.cpp:99] error parsing config: {
"context": {
"device": "",
"model": "",
"stream": ""
},
"input": [
"img"
],
"module": "Transform",
"name": "Preprocess",
"output": [
"prep_output"
],
"transforms": [
{
"backend_args": null,
"type": "LoadImageFromFile"
},
{
"interpolation": "bicubic",
"keep_ratio": false,
"size": [
640,
640
],
"type": "Resize"
},
{
"mean": [
0,
0,
0
],
"std": [
255,
255,
255
],
"to_rgb": true,
"type": "Normalize"
},
{
"size_divisor": 32,
"type": "Pad"
},
{
"type": "DefaultFormatBundle"
},
{
"keys": [
"img"
],
"meta_keys": [
"flip",
"pad_shape",
"pad_param",
"ori_filename",
"scale_factor",
"img_id",
"img_path",
"img_norm_cfg",
"img_shape",
"valid_ratio",
"ori_shape",
"filename",
"flip_direction"
],
"type": "Collect"
}
],
"type": "Task"
}
Segmentation fault (core dumped)

TensorRT

mmdeploy) nvidia@nvidia-desktop:~/文档/mmdeploy/build/bin$ ./object_detection cuda /home/nvidia/文档/mmdeploy_models/rt-detr /home/nvidia/图片/resources/test.jpg
[2024-01-06 19:13:35.692] [mmdeploy] [info] [model.cpp:35] [DirectoryModel] Load model: "/home/nvidia/文档/mmdeploy_models/rt-detr"
[2024-01-06 19:13:36.402] [mmdeploy] [error] [resize.cpp:84] unsupported interpolation method: bicubic
[2024-01-06 19:13:36.402] [mmdeploy] [error] [task.cpp:99] error parsing config: {
"context": {
"device": "",
"model": "",
"stream": ""
},
"input": [
"img"
],
"module": "Transform",
"name": "Preprocess",
"output": [
"prep_output"
],
"transforms": [
{
"backend_args": null,
"type": "LoadImageFromFile"
},
{
"interpolation": "bicubic",
"keep_ratio": false,
"size": [
640,
640
],
"type": "Resize"
},
{
"mean": [
0,
0,
0
],
"std": [
255,
255,
255
],
"to_rgb": true,
"type": "Normalize"
},
{
"size_divisor": 32,
"type": "Pad"
},
{
"type": "DefaultFormatBundle"
},
{
"keys": [
"img"
],
"meta_keys": [
"ori_shape",
"valid_ratio",
"img_norm_cfg",
"pad_param",
"img_path",
"img_id",
"img_shape",
"filename",
"scale_factor",
"flip_direction",
"ori_filename",
"flip",
"pad_shape"
],
"type": "Collect"
}
],
"type": "Task"
}
[2024-01-06 19:13:41.757] [mmdeploy] [error] [trt_net.cpp:28] TRTNet: 6: The engine plan file is not compatible with this version of TensorRT, expecting library version 8.2.1 got 8.2.3, please rebuild.
[2024-01-06 19:13:41.804] [mmdeploy] [error] [trt_net.cpp:28] TRTNet: 4: [runtime.cpp::deserializeCudaEngine::50] Error Code 4: Internal Error (Engine deserialization failed.)
[2024-01-06 19:13:41.804] [mmdeploy] [error] [trt_net.cpp:75] failed to deserialize TRT CUDA engine
[2024-01-06 19:13:41.829] [mmdeploy] [error] [net_module.cpp:54] Failed to create Net backend: tensorrt, config: {
"context": {
"device": "",
"model": "",
"stream": ""
},
"input": [
"prep_output"
],
"input_map": {
"img": "input"
},
"is_batched": false,
"module": "Net",
"name": "rtdetr",
"output": [
"infer_output"
],
"output_map": {},
"type": "Task"
}
[2024-01-06 19:13:41.829] [mmdeploy] [error] [task.cpp:99] error parsing config: {
"context": {
"device": "",
"model": "",
"stream": ""
},
"input": [
"prep_output"
],
"input_map": {
"img": "input"
},
"is_batched": false,
"module": "Net",
"name": "rtdetr",
"output": [
"infer_output"
],
"output_map": {},
"type": "Task"
}
Segmentation fault (core dumped)