Skip to content

Commit a2e4209

Browse files
brianzheng206brianzheng206
committed
nuke backend manager, clean up params
1 parent d4e4b58 commit a2e4209

File tree

10 files changed

+361
-799
lines changed

10 files changed

+361
-799
lines changed

‎deep_object_detection/CMakeLists.txt‎

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -38,7 +38,6 @@ find_package(rcl_interfaces REQUIRED)
3838
find_package(OpenCV REQUIRED COMPONENTS core imgproc)
3939

4040
add_library(deep_object_detection_lib SHARED
41-
src/backend_manager.cpp
4241
src/generic_postprocessor.cpp
4342
src/image_preprocessor.cpp
4443
src/deep_object_detection_node.cpp

‎deep_object_detection/README.md‎

Lines changed: 96 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -9,10 +9,81 @@ The `deep_object_detection` package provides:
99
- Automatic output format detection and adaptation
1010
- Multi-camera support via MultiImage messages with flexible batching
1111
- Configurable preprocessing and postprocessing pipelines
12-
- Explicit execution provider selection (TensorRT, CUDA, CPU) with fail-fast behavior
12+
- Plugin-based backend architecture (CPU, CUDA, TensorRT)
1313
- Full ROS 2 lifecycle node support
1414

15-
## Package Contents
15+
```
16+
┌────────────────────────────────────────────────────────────────────────────────────┐
17+
│ ROS 2 Topics │
18+
│ │
19+
│ ┌──────────────────────────────────┐ ┌──────────────────────────────┐ │
20+
│ │ MultiImage Topic │ │ Detection2DArray Topic │ │
21+
│ │ (compressed images) │ └──────────────────────────────┘ │
22+
│ └───────────────┬──────────────────┘ ┌──────────────────────────────┐ │
23+
│ │ │ ImageMarker Topic │ │
24+
│ │ └──────────────────────────────┘ │
25+
└──────────────────┼───────────────────────────────────────────────┬────────────────┘
26+
│ │
27+
▼ ▼
28+
┌────────────────────────────────────────────────────────────────────────────────────┐
29+
│ DeepObjectDetectionNode (rclcpp_lifecycle::LifecycleNode) │
30+
│ │
31+
│ ┌──────────────────────────────────────────────────────────────────────────────┐ │
32+
│ │ Processing Pipeline │ │
33+
│ │ │ │
34+
│ │ ┌──────────────────────────┐ ┌──────────────────────────┐ │ │
35+
│ │ │ ImagePreprocessor │ │ Backend Plugin │ │ │
36+
│ │ │ • decode images │ ────▶│ • plugin loader │ │ │
37+
│ │ │ • resize │ │ • memory allocation │ │ │
38+
│ │ │ • normalize │ │ • inference execution │ │ │
39+
│ │ │ • pack tensor │ └─────────────┬────────────┘ │ │
40+
│ │ └──────────────────────────┘ │ │ │
41+
│ │ ▼ │ │
42+
│ │ ┌────────────────────────────────────────────────────────────────────────┐ │ │
43+
│ │ │ GenericPostprocessor │ │ │
44+
│ │ │ • decode outputs │ │ │
45+
│ │ │ • NMS filtering │ │ │
46+
│ │ │ • coordinate transforms │ │ │
47+
│ │ └────────────────────────────────────────────────────────────────────────┘ │ │
48+
│ └──────────────────────────────────────────────────────────────────────────────┘ │
49+
└──────────────────────────────────────────────────────────────────┬─────────────────┘
50+
│
51+
▼
52+
┌────────────────────────────────────────────────────────────────────────────────────┐
53+
│ Backend Plugins (pluginlib) │
54+
│ │
55+
│ ┌──────────────────────────────────────────────────────────────────────────────┐ │
56+
│ │ ClassLoader<DeepBackendPlugin> │ │
57+
│ │ │ │
58+
│ │ ┌──────────────────────────┐ ┌──────────────────────────┐ │ │
59+
│ │ │ onnxruntime_cpu │ │ onnxruntime_gpu │ │ │
60+
│ │ │ (CPU Backend) │ │ (CUDA / TensorRT) │ │ │
61+
│ │ └─────────────┬────────────┘ └─────────────┬────────────┘ │ │
62+
│ │ └───────────────┬──────────────────┘ │ │
63+
│ │ ▼ │ │
64+
│ │ ┌────────────────────────────────────────────────────────────────────────┐ │ │
65+
│ │ │ BackendMemoryAllocator │ │ │
66+
│ │ │ (CPU/GPU memory management) │ │ │
67+
│ │ └────────────────────────────────────────────────────────────────────────┘ │ │
68+
│ │ ┌────────────────────────────────────────────────────────────────────────┐ │ │
69+
│ │ │ BackendInferenceExecutor │ │ │
70+
│ │ │ (ONNX Runtime inference) │ │ │
71+
│ │ └────────────────────────────────────────────────────────────────────────┘ │ │
72+
│ └──────────────────────────────────────────────────────────────────────────────┘ │
73+
└──────────────────────────────────────────────────────────────────┬─────────────────┘
74+
│
75+
▼
76+
┌────────────────────────────────────────────────────────────────────────────────────┐
77+
│ Model & Configuration │
78+
│ │
79+
│ ┌──────────────────────────┐ ┌──────────────────────────────────────────┐ │
80+
│ │ ONNX Model File (.onnx) │ │ YAML Config │ │
81+
│ └──────────────────────────┘ │ • preprocessing │ │
82+
│ │ • postprocessing │ │
83+
│ │ • execution provider │ │
84+
│ └──────────────────────────────────────────┘ │
85+
└────────────────────────────────────────────────────────────────────────────────────┘
86+
```
1687

1788
### Nodes
1889

@@ -185,14 +256,17 @@ deep_object_detection_node:
185256
output_detections_topic: "/detections"
186257
```
187258

188-
### Execution Provider Selection
259+
### Backend Plugin Selection
189260

190-
The node supports explicit provider selection with fail-fast behavior. If the specified provider is unavailable or fails to initialize, the node will immediately throw an error (no silent fallbacks).
261+
The node uses a plugin-based architecture for backend selection. You must specify which backend plugin to use via the `Backend.plugin` parameter.
191262

192-
**Available Providers:**
193-
- `tensorrt` - TensorRT execution provider (requires CUDA and TensorRT)
194-
- `cuda` - CUDA execution provider (requires CUDA)
195-
- `cpu` - CPU execution provider (always available)
263+
**Available Plugins:**
264+
- `onnxruntime_cpu` - CPU backend (always available)
265+
- `onnxruntime_gpu` - GPU backend supporting CUDA and TensorRT (requires CUDA)
266+
267+
For GPU plugin, you can specify the execution provider via `Backend.execution_provider`:
268+
- `cuda` - CUDA execution provider
269+
- `tensorrt` - TensorRT execution provider (requires TensorRT)
196270

197271
## Expected Model Format
198272

@@ -210,6 +284,15 @@ The node automatically detects and adapts to various ONNX model output formats:
210284

211285
## Parameters
212286

287+
**Important:** All string parameters must be specified in **lowercase**. The node performs direct string comparisons and does not normalize case. For example:
288+
- `model.bbox_format` must be `"cxcywh"`, `"xyxy"`, or `"xywh"` (not `"CXCYWH"`)
289+
- `preprocessing.normalization_type` must be `"imagenet"`, `"scale_0_1"`, `"custom"`, or `"none"`
290+
- `preprocessing.resize_method` must be `"letterbox"`, `"resize"`, `"crop"`, or `"pad"`
291+
- `postprocessing.score_activation` must be `"sigmoid"`, `"softmax"`, or `"none"`
292+
- `postprocessing.class_score_mode` must be `"all_classes"` or `"single_confidence"`
293+
- `Backend.plugin` must be `"onnxruntime_cpu"` or `"onnxruntime_gpu"`
294+
- `Backend.execution_provider` must be `"cuda"` or `"tensorrt"` (for GPU plugin)
295+
213296
### Required Parameters
214297
- **`model_path`** (string): Absolute path to ONNX model file (e.g., `/workspaces/deep_ros/yolov8m.onnx`).
215298
- **`input_topic`** (string): MultiImage topic name to subscribe to.
@@ -223,9 +306,11 @@ The node automatically detects and adapts to various ONNX model output formats:
223306
- **`preprocessing.resize_method`** (string, default: "letterbox"): Image resizing method.
224307
- **`postprocessing.score_threshold`** (float, default: 0.25): Minimum confidence score.
225308
- **`postprocessing.nms_iou_threshold`** (float, default: 0.45): IoU threshold for NMS.
226-
- **`preferred_provider`** (string, default: "tensorrt"): Execution provider (tensorrt, cuda, or cpu).
227-
- **`min_batch_size`** (int, default: 1): Minimum images before processing.
228-
- **`max_batch_size`** (int, default: 3): Maximum images per batch.
309+
- **`Backend.plugin`** (string, required): Backend plugin name (`"onnxruntime_cpu"` or `"onnxruntime_gpu"`).
310+
- **`Backend.execution_provider`** (string, default: "tensorrt"): Execution provider for GPU plugin (`"cuda"` or `"tensorrt"`). Only used with `onnxruntime_gpu` plugin.
311+
- **`Backend.device_id`** (int, default: 0): GPU device ID (for CUDA/TensorRT).
312+
- **`Backend.trt_engine_cache_enable`** (bool, default: false): Enable TensorRT engine caching.
313+
- **`Backend.trt_engine_cache_path`** (string, default: "/tmp/deep_ros_ort_trt_cache"): TensorRT engine cache directory.
229314

230315
See `config/generic_model_params.yaml` for a complete parameter reference.
231316

‎deep_object_detection/config/generic_model_params.yaml‎

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -30,7 +30,6 @@ deep_object_detection_node:
3030
class_score_mode: "all_classes"
3131
class_score_start_idx: -1
3232
class_score_count: -1
33-
coordinate_space: "preprocessed"
3433

3534
layout:
3635
auto_detect: true
@@ -42,8 +41,9 @@ deep_object_detection_node:
4241
score_idx: 4
4342
class_idx: 5
4443

45-
preferred_provider: "tensorrt"
46-
device_id: 0
47-
warmup_tensor_shapes: true
48-
enable_trt_engine_cache: true
49-
trt_engine_cache_path: "/tmp/deep_ros_ort_trt_cache"
44+
Backend:
45+
plugin: "onnxruntime_gpu" # Options: "onnxruntime_cpu" or "onnxruntime_gpu"
46+
execution_provider: "tensorrt" # For onnxruntime_gpu: "cuda" or "tensorrt"
47+
device_id: 0
48+
trt_engine_cache_enable: true
49+
trt_engine_cache_path: "/tmp/deep_ros_ort_trt_cache"

0 commit comments

Comments
 (0)