lucas/onnx_gpu_backend

[lucasreljic]

Adding deep_ort_gpu_backend_plugin to allow for inference with onnxruntime with GPU-based execution providers. Uses the onnxruntime_gpu_vendor package.

• Tested with object_detection branch on CUDA v12.2
• Currently, only tested with CUDA as the execution provider; through further debugging, TensorRT might be possible
• Passes in inputs under CPU memory just like deep_ort_backend_plugin, since onnxruntime converts to GPU memory anyway. There are methods for passing in GPU memory, which could be explored in the future when ROS Nitros is implemented.

[Edwardius]

Thanks for working on this! Some comments.

[Edwardius]

@lucasreljic tests pass. Works with tensorrt and Cuda