Support CUDA and ROCM mainly on Linux #697
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This commit introduces support for the ROCm execution provider in ONNX Runtime. It adds new UI options for selecting ROCm as the inference device, updates constants, and conditionally compiles ROCm-related code paths. CUDA and ROCm options are now only shown if the corresponding ONNX Runtime support is available.
Introduced the 'GPUROCM' key with appropriate translations in all supported locale .ini files to support ROCM inference device labeling in the UI.
Replaces HAVE_ONNXRUNTIME_CUDA and HAVE_ONNXRUNTIME_ROCM with HAVE_ONNXRUNTIME_CUDA_EP and HAVE_ONNXRUNTIME_ROCM_EP to match updated macro definitions. Ensures correct conditional compilation for CUDA and ROCm execution providers.
This commit removes all code, localization strings, and documentation references related to TensorRT support. The affected files include source code, constants, and multiple locale files, reflecting the discontinuation of TensorRT as an inference device option.
Collaborator
Author
|
@royshil please review |
sobalap
pushed a commit
to sobalap/obs-backgroundremoval
that referenced
this pull request
Jan 7, 2026
* Update CMakeLists.txt * Add ROCm support for ONNX Runtime GPU execution This commit introduces support for the ROCm execution provider in ONNX Runtime. It adds new UI options for selecting ROCm as the inference device, updates constants, and conditionally compiles ROCm-related code paths. CUDA and ROCm options are now only shown if the corresponding ONNX Runtime support is available. * Add GPU-ROCM translation to all locale files Introduced the 'GPUROCM' key with appropriate translations in all supported locale .ini files to support ROCM inference device labeling in the UI. * Update enhance-filter.cpp * Update CMakeLists.txt * Update CMakeLists.txt * Update ONNX Runtime GPU provider macro names Replaces HAVE_ONNXRUNTIME_CUDA and HAVE_ONNXRUNTIME_ROCM with HAVE_ONNXRUNTIME_CUDA_EP and HAVE_ONNXRUNTIME_ROCM_EP to match updated macro definitions. Ensures correct conditional compilation for CUDA and ROCm execution providers. * Remove TensorRT support and references This commit removes all code, localization strings, and documentation references related to TensorRT support. The affected files include source code, constants, and multiple locale files, reflecting the discontinuation of TensorRT as an inference device option.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This pull request adds support for the ONNX Runtime ROCM execution provider (for AMD GPUs) and improves detection and configuration of GPU acceleration options. It also removes the previous TensorRT (NVIDIA) support and updates both the codebase and user interface to reflect these changes. Additionally, documentation and localization files are updated to match the new GPU support options.
Users on linux who need GPU have to build our plugin from source, and this policy ensures that a correct GPU framework is present on their system.
Build system and GPU provider detection:
CMakeLists.txtto detect ONNX Runtime CUDA and ROCM execution providers at build time, definingHAVE_ONNXRUNTIME_CUDA_EPandHAVE_ONNXRUNTIME_ROCM_EPaccordingly. This enables conditional compilation of GPU support depending on available providers.DISABLE_ONNXRUNTIME_GPUinCMakeLists.txt.Codebase and execution provider changes:
Documentation updates:
README.mdto clarify GPU support: Windows will support WinML, Mac supports CoreML, and Linux supports CUDA and ROCM via ONNX Runtime. Removed outdated references to TensorRT.Localization/UI updates:
Constants:
USEGPU_TENSORRTconstant toUSEGPU_ROCMinsrc/consts.h.