Skip to content
Draft
54 changes: 35 additions & 19 deletions docs/build/eps.md
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,7 @@ The onnxruntime code will look for the provider shared libraries in the same loc
{: .no_toc }

* Install [CUDA](https://developer.nvidia.com/cuda-toolkit) and [cuDNN](https://developer.nvidia.com/cudnn)
* The CUDA execution provider for ONNX Runtime is built and tested with CUDA 11.8, 12.2 and cuDNN 8.9. Check [here](../execution-providers/CUDA-ExecutionProvider.md#requirements) for more version information.
* The CUDA execution provider for ONNX Runtime is built and tested with CUDA 12.x and cuDNN 9. Check [here](../execution-providers/CUDA-ExecutionProvider.md#requirements) for more version information.
* The path to the CUDA installation must be provided via the CUDA_HOME environment variable, or the `--cuda_home` parameter. The installation directory should contain `bin`, `include` and `lib` sub-directories.
* The path to the CUDA `bin` directory must be added to the PATH environment variable so that `nvcc` is found.
* The path to the cuDNN installation must be provided via the CUDNN_HOME environment variable, or `--cudnn_home` parameter. In Windows, the installation directory should contain `bin`, `include` and `lib` sub-directories.
Expand Down Expand Up @@ -110,7 +110,7 @@ See more information on the TensorRT Execution Provider [here](../execution-prov

* Follow [instructions for CUDA execution provider](#cuda) to install CUDA and cuDNN, and setup environment variables.
* Follow [instructions for installing TensorRT](https://docs.nvidia.com/deeplearning/tensorrt/latest/installing-tensorrt/installing.html)
* The TensorRT execution provider for ONNX Runtime is built and tested with TensorRT 10.8.
* The TensorRT execution provider for ONNX Runtime is built and tested with TensorRT 10.9.
* The path to TensorRT installation must be provided via the `--tensorrt_home` parameter.
* ONNX Runtime uses [TensorRT built-in parser](https://developer.nvidia.com/tensorrt/download) from `tensorrt_home` by default.
* To use open-sourced [onnx-tensorrt](https://github.com/onnx/onnx-tensorrt/tree/main) parser instead, add `--use_tensorrt_oss_parser` parameter in build commands below.
Expand All @@ -123,14 +123,15 @@ See more information on the TensorRT Execution Provider [here](../execution-prov
* i.e It's version-matched if assigning `tensorrt_home` with path to TensorRT-10.9 built-in binaries and onnx-tensorrt [10.9-GA branch](https://github.com/onnx/onnx-tensorrt/tree/release/10.9-GA) specified in [cmake/deps.txt](https://github.com/microsoft/onnxruntime/blob/main/cmake/deps.txt).


### **[Note to ORT 1.21.0 open-sourced parser users]**
### **[Note to ORT 1.21/1.22 open-sourced parser users]**

* ORT 1.21.0 links against onnx-tensorrt 10.8-GA, which requires upcoming onnx 1.18.
* Here's a temporarily fix to preview on onnx-tensorrt 10.8-GA (or newer) when building ORT 1.21.0:
* ORT 1.21/1.22 link against onnx-tensorrt 10.8-GA/10.9-GA, which requires newly released onnx 1.18.
* Here's a temporarily fix to preview on onnx-tensorrt 10.8-GA/10.9-GA when building ORT 1.21/1.22:
* Replace the [onnx line in cmake/deps.txt](https://github.com/microsoft/onnxruntime/blob/rel-1.21.0/cmake/deps.txt#L38)
with `onnx;https://github.com/onnx/onnx/archive/f22a2ad78c9b8f3bd2bb402bfce2b0079570ecb6.zip;324a781c31e30306e30baff0ed7fe347b10f8e3c`
with `onnx;https://github.com/onnx/onnx/archive/e709452ef2bbc1d113faf678c24e6d3467696e83.zip;c0b9f6c29029e13dea46b7419f3813f4c2ca7db8`
* Download [this](https://github.com/microsoft/onnxruntime/blob/7b2733a526c12b5ef4475edd47fd9997ebc2b2c6/cmake/patches/onnx/onnx.patch) as raw file and save file to [cmake/patches/onnx/onnx.patch](https://github.com/microsoft/onnxruntime/blob/rel-1.21.0/cmake/patches/onnx/onnx.patch) (do not copy/paste from browser, as it might alter line break type)
* Build ORT 1.21.0 with trt-related flags above (including `--use_tensorrt_oss_parser`)
* Build ORT with trt-related flags above (including `--use_tensorrt_oss_parser`)
* The [onnx 1.18](https://github.com/onnx/onnx/releases/tag/v1.18.0) is supported by latest ORT main branch. Please checkout main branch and build ORT-TRT with `--use_tensorrt_oss_parser` to enable OSS parser with full onnx 1.18 support.

### Build Instructions
{: .no_toc }
Expand Down Expand Up @@ -234,6 +235,21 @@ These instructions are for the latest [JetPack SDK](https://developer.nvidia.com

* For a portion of Jetson devices like the Xavier series, higher power mode involves more cores (up to 6) to compute but it consumes more resource when building ONNX Runtime. Set `--parallel 1` in the build command if OOM happens and system is hanging.

## TensorRT-RTX

See more information on the NV TensorRT RTX Execution Provider [here](../execution-providers/TensorRTRTX-ExecutionProvider.md).

### Prerequisites
{: .no_toc }

* Follow [instructions for CUDA execution provider](#cuda) to install CUDA and setup environment variables.
* Intall TensorRT for RTX from nvidia.com (TODO: add link when available)

### Build Instructions
{: .no_toc }
`build.bat --config Release --parallel 32 --build_dir _build --build_shared_lib --use_nv_tensorrt_rtx --tensorrt_home "C:\dev\TensorRT-RTX-1.1.0.3" --cuda_home "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.9" --cmake_generator "Visual Studio 17 2022" --use_vcpkg`
Replace the --tensorrt_home and --cuda_home with correct paths to CUDA and TensorRT-RTX installations.

## oneDNN

See more information on oneDNN (formerly DNNL) [here](../execution-providers/oneDNN-ExecutionProvider.md).
Expand Down Expand Up @@ -276,15 +292,15 @@ See more information on the OpenVINO™ Execution Provider [here](../execution-p
{: .no_toc }

1. Install the OpenVINO™ offline/online installer from Intel<sup>®</sup> Distribution of OpenVINO™<sup>TM</sup> Toolkit **Release 2024.3** for the appropriate OS and target hardware:
* [Windows - CPU, GPU, NPU](https://www.intel.com/content/www/us/en/developer/tools/openvino-toolkit/download.html?PACKAGE=OPENVINO_BASE&VERSION=v_2024_3_0&OP_SYSTEM=WINDOWS&DISTRIBUTION=ARCHIVE).
* [Linux - CPU, GPU, NPU](https://www.intel.com/content/www/us/en/developer/tools/openvino-toolkit/download.html?PACKAGE=OPENVINO_BASE&VERSION=v_2024_3_0&OP_SYSTEM=LINUX&DISTRIBUTION=ARCHIVE)
* [Windows - CPU, GPU, NPU](https://www.intel.com/content/www/us/en/developer/tools/openvino-toolkit/download.html?PACKAGE=OPENVINO_BASE&VERSION=v_2025_3_0&OP_SYSTEM=WINDOWS&DISTRIBUTION=ARCHIVE).
* [Linux - CPU, GPU, NPU](https://www.intel.com/content/www/us/en/developer/tools/openvino-toolkit/download.html?PACKAGE=OPENVINO_BASE&VERSION=v_2025_3_0&OP_SYSTEM=LINUX&DISTRIBUTION=ARCHIVE)

Follow [documentation](https://docs.openvino.ai/2024/home.html) for detailed instructions.
Follow [documentation](https://docs.openvino.ai/2025/index.html) for detailed instructions.

*2024.5 is the current recommended OpenVINO™ version. [OpenVINO™ 2024.5](https://docs.openvino.ai/2024/index.html) is minimal OpenVINO™ version requirement.*
*2025.3 is the current recommended OpenVINO™ version. [OpenVINO™ 2025.0](https://docs.openvino.ai/2025/index.html) is minimal OpenVINO™ version requirement.*

2. Configure the target hardware with specific follow on instructions:
* To configure Intel<sup>®</sup> Processor Graphics(GPU) please follow these instructions: [Windows](https://docs.openvino.ai/2024/get-started/configurations/configurations-intel-gpu.html#windows), [Linux](https://docs.openvino.ai/2024/get-started/configurations/configurations-intel-gpu.html#linux)
* To configure Intel<sup>®</sup> Processor Graphics(GPU) please follow these instructions: [Windows](https://docs.openvino.ai/2025/get-started/install-openvino/configurations/configurations-intel-gpu.html#windows), [Linux](https://docs.openvino.ai/2025/get-started/install-openvino/configurations/configurations-intel-gpu.html#linux)


3. Initialize the OpenVINO™ environment by running the setupvars script as shown below. This is a required step:
Expand All @@ -296,7 +312,7 @@ See more information on the OpenVINO™ Execution Provider [here](../execution-p
```
$ source <openvino_install_directory>/setupvars.sh
```
**Note:** If you are using a dockerfile to use OpenVINO™ Execution Provider, sourcing OpenVINO™ won't be possible within the dockerfile. You would have to explicitly set the LD_LIBRARY_PATH to point to OpenVINO™ libraries location. Refer our [dockerfile](https://github.com/microsoft/onnxruntime/blob/main/dockerfiles/Dockerfile.openvino).


### Build Instructions
{: .no_toc }
Expand All @@ -319,7 +335,7 @@ See more information on the OpenVINO™ Execution Provider [here](../execution-p
* `--use_openvino` builds the OpenVINO™ Execution Provider in ONNX Runtime.
* `<hardware_option>`: Specifies the default hardware target for building OpenVINO™ Execution Provider. This can be overriden dynamically at runtime with another option (refer to [OpenVINO™-ExecutionProvider](../execution-providers/OpenVINO-ExecutionProvider.md#summary-of-options) for more details on dynamic device selection). Below are the options for different Intel target devices.

Refer to [Intel GPU device naming convention](https://docs.openvino.ai/2024/openvino-workflow/running-inference/inference-devices-and-modes/gpu-device.html#device-naming-convention) for specifying the correct hardware target in cases where both integrated and discrete GPU's co-exist.
Refer to [Intel GPU device naming convention](https://docs.openvino.ai/2025/openvino-workflow/running-inference/inference-devices-and-modes/gpu-device.html#device-naming-convention) for specifying the correct hardware target in cases where both integrated and discrete GPU's co-exist.

| Hardware Option | Target Device |
| --------------- | ------------------------|
Expand Down Expand Up @@ -351,8 +367,8 @@ Example's: HETERO:GPU,CPU or AUTO:GPU,CPU or MULTI:GPU,CPU
* To enable this feature during build time. Use `--use_openvino ` `<hardware_option>_NO_PARTITION`

```
Usage: --use_openvino CPU_FP32_NO_PARTITION or --use_openvino GPU_FP32_NO_PARTITION or
--use_openvino GPU_FP16_NO_PARTITION
Usage: --use_openvino CPU_NO_PARTITION or --use_openvino GPU_NO_PARTITION or
--use_openvino GPU_NO_PARTITION
```

For more information on OpenVINO™ Execution Provider&#39;s ONNX Layer support, Topology support, and Intel hardware enabled, please refer to the document [OpenVINO™-ExecutionProvider](../execution-providers/OpenVINO-ExecutionProvider.md)
Expand Down Expand Up @@ -595,7 +611,7 @@ e.g.
### Linux
{: .no_toc }

Currently Linux support is only enabled for AMD Adapable SoCs. Please refer to the guidance [here](../execution-providers/Vitis-AI-ExecutionProvider.md#amd-adaptable-soc-installation) for SoC targets.
Currently Linux support is only enabled for AMD Adapable SoCs. Please refer to the guidance [here](../execution-providers/Vitis-AI-ExecutionProvider.md#installation-for-amd-adaptable-socs) for SoC targets.

---

Expand Down Expand Up @@ -624,7 +640,7 @@ Dockerfile instructions are available [here](https://github.com/microsoft/onnxru

#### Build Phython Wheel

`./build.sh --config Release --build --build_wheel --parallel --use_migraphx --migraphx_home /opt/rocm`
`./build.sh --config Release --build_wheel --parallel --use_migraphx --migraphx_home /opt/rocm`

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you update your branch to latest gh-pages so that only relevant openvino changes are reflecting for review


Then the python wheels(*.whl) could be found at ```./build/Linux/Release/dist```.

Expand Down Expand Up @@ -653,7 +669,7 @@ Dockerfile instructions are available [here](https://github.com/microsoft/onnxru

#### Build Phython Wheel

`./build.sh --config Release --build --build_wheel --parallel --use_rocm --rocm_home /opt/rocm`
`./build.sh --config Release --build_wheel --parallel --use_rocm --rocm_home /opt/rocm`

Then the python wheels(*.whl) could be found at ```./build/Linux/Release/dist```.

Expand Down
Loading
Loading