feat(BEVFusion): add gpu-based image undistortion to the bevfusion node

[KSeangTan]

Description

This PR introduces a new feature to BEVFusion to run gpu-based image undistortion in the node, which is the same implementation in autoware_camera_streampetr.

This PR also slightly refactors and reorganizes the code structure as the prerequisite to separate the BEVFusion node according to the task type, for example, BEVFusion_camera, BEVFusion_camera_lidar and BEVFusion_lidar. Note that the refactoring will be done in another PR.

Specifically, it makes the following main changes:

• Add parameter: run_image_undistortion to bevfusion_camera_lidar.param.yaml and perception/autoware_bevfusion/config/bevfusion_lidar.param.yaml to enable/disable image_undistortion. [1], [2]
• Add parameter: flip_image_channels to perception/autoware_bevfusion/config/ml_package_bevfusion_camera_lidar.param.yaml and perception/autoware_bevfusion/config/ml_package_bevfusion_lidar.param.yaml to flip image channels in preprocessing. [3], [4]
• Add camera_data.hpp and camera_data.cpp as a data structure to preprocess/save image data in cpu/gpu. [5], [6]
• Add camera_metrices.hpp and camera_matrices.cpp to precompute camera undistortion. [7], [8]
• Add camera_preprocessing.hpp and camera_preprocessing.cpp to handle everything related to image preprocessing kernels [9], [10]
• Update initializeSensorFusionSubscribers to construct camera_matrices and camera_data [11]
• Construct image_preprocessing_params in bevfusion_node [12]
• Replace image_msgs_ and camera_info_msgs_ by camera_data_ptrs_ and camera_matrices_ptrs_, respectively. [13]
• Update imageCallback in bevfusion_node to update camera_data_ptrs_ instead of image_msgs_. [14]
• Update cameraInfoCallback to update camera_data_ptrs_ instead of camera_info_msgs_. [15]
• Replace image_msgs by camera_data_ptrs in bevfusion_trt to accepth the new inputs [16], [17]

How was this PR tested?

• Tested with the following example internally and locally.

Examples

Before

After

ROS Parameter Changes

Additions and removals

Change type	Parameter Name	Type	Default Value	Description
Add	run_image_undistortion	bool	false	Set true to run the image undistortion in the node.
Add	flip_image_channels	bool	true	Set true to flip image channels in the preprocessing, for example, BGR -> RGB.

[github-actions]

Thank you for contributing to the Autoware project!

🚧 If your pull request is in progress, switch it to draft mode.

Please ensure:

• You've checked our contribution guidelines.
• Your PR follows our pull request guidelines.
• All required CI checks pass before marking the PR ready for review.

[Copilot]

Pull request overview

Adds a camera image undistortion + preprocessing path to the BEVFusion camera-lidar pipeline by introducing per-camera GPU preprocessing objects and wiring them through the node and TensorRT runner.

Changes:

• Introduce CameraData / CameraMatrices and a CUDA camera preprocessing implementation (remap/undistort + resize/crop + optional channel flip).
• Refactor BEVFusionNode and BEVFusionTRT to pass/use CameraData instead of raw image/camera-info buffers.
• Add new ROS parameters (run_image_undistortion, flip_image_channels) and update launch/config wiring accordingly; remove the old ROI resize kernel from the generic preprocess module.

Reviewed changes

Copilot reviewed 22 out of 22 changed files in this pull request and generated 6 comments.

Show a summary per file

File	Description
perception/autoware_bevfusion/src/bevfusion_node_utils.cpp	Builds per-camera objects and updates intrinsics/mask handling to use CameraData.
perception/autoware_bevfusion/src/bevfusion_node.cpp	Declares new preprocessing params and routes image/camera_info callbacks into CameraData.
perception/autoware_bevfusion/lib/preprocess/preprocess_kernel.cu	Removes the old image ROI resize kernel from the generic preprocess CUDA module.
perception/autoware_bevfusion/lib/camera_matrices.cpp	Adds a CameraMatrices implementation (but duplicates the lib/camera/ version).
perception/autoware_bevfusion/lib/camera/camera_preprocess_kernel.cu	Adds CUDA remap (undistortion) and fused resize/crop kernel used per camera.
perception/autoware_bevfusion/lib/camera/camera_matrices.cpp	Implements camera intrinsics/distortion handling and undistortion map upload to GPU.
perception/autoware_bevfusion/lib/camera/camera_data.cpp	Implements per-camera message storage and GPU preprocessing orchestration.
perception/autoware_bevfusion/lib/bevfusion_trt.cpp	Refactors image preprocessing to use CameraData (per-camera streams/buffers).
perception/autoware_bevfusion/launch/bevfusion.launch.xml	Renames launch args for camera topics and keeps remaps consistent.
perception/autoware_bevfusion/include/autoware/bevfusion/preprocess/preprocess_kernel.hpp	Removes the old ROI resize API from PreprocessCuda.
perception/autoware_bevfusion/include/autoware/bevfusion/camera/camera_preprocess.hpp	Declares the new camera preprocessing CUDA API.
perception/autoware_bevfusion/include/autoware/bevfusion/camera/camera_matrices.hpp	Declares CameraMatrices and device-side undistortion maps.
perception/autoware_bevfusion/include/autoware/bevfusion/camera/camera_data.hpp	Declares CameraData and ImagePreProcessingParams.
perception/autoware_bevfusion/include/autoware/bevfusion/bevfusion_trt.hpp	Updates TRT interfaces to accept CameraData instead of raw image vectors.
perception/autoware_bevfusion/include/autoware/bevfusion/bevfusion_node.hpp	Stores per-camera objects and updates subscriber initialization signature.
perception/autoware_bevfusion/config/ml_package_bevfusion_lidar.param.yaml	Adds flip_image_channels parameter for lidar-only package config.
perception/autoware_bevfusion/config/ml_package_bevfusion_camera_lidar.param.yaml	Adds flip_image_channels parameter for camera-lidar package config.
perception/autoware_bevfusion/config/bevfusion_lidar.param.yaml	Adds run_image_undistortion toggle (false) for lidar-only config.
perception/autoware_bevfusion/config/bevfusion_camera_lidar.param.yaml	Adds run_image_undistortion toggle (true) for camera-lidar config.
perception/autoware_bevfusion/CMakeLists.txt	Wires new camera sources into the build.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[codecov]

Codecov Report

❌ Patch coverage is 0% with 164 lines in your changes missing coverage. Please review.
✅ Project coverage is 18.92%. Comparing base (d799ddd) to head (c5074e7).

Files with missing lines	Patch %	Lines
...tion/autoware_bevfusion/lib/camera/camera_data.cpp	0.00%	86 Missing ⚠️
.../autoware_bevfusion/lib/camera/camera_matrices.cpp	0.00%	30 Missing ⚠️
...e_bevfusion/lib/camera/camera_preprocess_kernel.cu	0.00%	20 Missing ⚠️
...rception/autoware_bevfusion/src/bevfusion_node.cpp	0.00%	11 Missing ⚠️
...on/autoware_bevfusion/src/bevfusion_node_utils.cpp	0.00%	10 Missing ⚠️
...erception/autoware_bevfusion/lib/bevfusion_trt.cpp	0.00%	7 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main   #12279      +/-   ##
==========================================
+ Coverage   18.49%   18.92%   +0.42%     
==========================================
  Files        1855     1893      +38     
  Lines      129017   130102    +1085     
  Branches    45872    48352    +2480     
==========================================
+ Hits        23864    24619     +755     
- Misses      85382    86241     +859     
+ Partials    19771    19242     -529     

Flag	Coverage Δ		*Carryforward flag
daily	21.01% <ø> (ø)		Carriedforward from 2fd3e30
daily-cuda	18.57% <ø> (+<0.01%)	⬆️	Carriedforward from 2fd3e30
daily-humble-amd64-cuda	18.47% <ø> (+<0.01%)	⬆️	Carriedforward from 2fd3e30
daily-humble-amd64-nocuda	20.88% <ø> (ø)		Carriedforward from 2fd3e30
differential	2.14% <ø> (?)
differential-cuda	0.00% <0.00%> (?)
total-cuda	18.57% <ø> (+<0.01%)	⬆️	Carriedforward from 2fd3e30
total-humble-cuda	18.45% <ø> (+0.10%)	⬆️	Carriedforward from 2fd3e30

*This pull request uses carry forward flags. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.