Skip to content

[MIGraphx][Documentation] Update Documentation for ROCm 6.4 and additional MIGraphX EP variables #24663

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
tianleiwu merged 12 commits into from
May 8, 2025
Merged

[MIGraphx][Documentation] Update Documentation for ROCm 6.4 and additional MIGraphX EP variables #24663

Changes from 5 commits
Commits
Show all changes
12 commits
Select commit Hold shift + click to select a range
6548c20
Initial add of env option flags
Jan 28, 2025
70b9caf
Clean up formatting start provider options variables
Jan 28, 2025
f79eac9
Update rocm matrix for 6.4, update session and env variables used for…
May 6, 2025
7dfc080
Fix spacing of python versions
May 6, 2025
9e40fc8
Fix spacing for entry
May 6, 2025
f236d0e
Flip table for onnxruntime support matrix
May 7, 2025
9b21eb6
Update based on reviewer comments
May 7, 2025
b1e46da
fixup! Update based on reviewer comments
May 7, 2025
90186fb
Add dockerhub link, reorder where build from source goes and add tabl…
May 7, 2025
4466b40
fixup! Add dockerhub link, reorder where build from source goes and a…
May 7, 2025
91d158a
Update based on TianLeiWu's Comments
May 8, 2025
cfce053
Update doc with further comments
May 8, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
97 changes: 75 additions & 22 deletions docs/execution-providers/MIGraphX-ExecutionProvider.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,26 +33,27 @@ Below is the matrix of supported ROCm versions corresponding to Ubuntu builds.
Links for prebuild Python Wheels (.whl) are linked below corresponding to python versions for the host OS based on Ubuntu support.


|ONNX Runtime Version|MIGraphX ROCm Release| Repo.Radeon link|
|---|---|---|
|main|5.4||
|1.14|5.4||
|1.13|5.4||
|1.13|5.3.2||
|1.12|5.2.3||
|1.12|5.2||
|1.13|5.3||
|1.14|5.4||
|1.15|5.4.x||
|1.16|6.0.0||
|1.16|6.0.2| [Python 3.10](https://repo.radeon.com/rocm/manylinux/rocm-rel-6.0.2/onnxruntime_rocm-inference-1.17.0-cp310-cp310-linux_x86_64.whl)|
|1.17|6.1| [Python 3.8](https://repo.radeon.com/rocm/manylinux/rocm-rel-6.1/onnxruntime_rocm-inference-1.17.0-cp38-cp38-linux_x86_64.whl) [Python 3.10](https://repo.radeon.com/rocm/manylinux/rocm-rel-6.1/onnxruntime_rocm-inference-1.17.0-cp310-cp310-linux_x86_64.whl)|
|1.17|6.1.3| [Python 3.10](https://repo.radeon.com/rocm/manylinux/rocm-rel-6.1.3/onnxruntime_rocm-1.17.0-cp310-cp310-linux_x86_64.whl)|
|1.18|6.2| [Python 3.8](https://repo.radeon.com/rocm/manylinux/rocm-rel-6.2/onnxruntime_rocm-1.18.0-cp38-cp38-linux_x86_64.whl) [Python 3.10](https://repo.radeon.com/rocm/manylinux/rocm-rel-6.2/onnxruntime_rocm-1.18.0-cp310-cp310-linux_x86_64.whl)|
|1.18|6.2.3| [Python 3.10](https://repo.radeon.com/rocm/manylinux/rocm-rel-6.2.3/onnxruntime_rocm-1.18.0-cp310-cp310-linux_x86_64.whl)|
|1.18|6.2.4| [Python 3.10](https://repo.radeon.com/rocm/manylinux/rocm-rel-6.2.4/onnxruntime_rocm-1.18.0-cp310-cp310-linux_x86_64.whl)|
|1.19|6.3| [Python 3.10](https://repo.radeon.com/rocm/manylinux/rocm-rel-6.3/onnxruntime_rocm-1.19.0-cp310-cp310-linux_x86_64.whl) [Python 3.12](https://repo.radeon.com/rocm/manylinux/rocm-rel-6.3/onnxruntime_rocm-1.19.0-cp312-cp312-linux_x86_64.whl)|
|1.19|6.3.1| [Python 3.10](https://repo.radeon.com/rocm/manylinux/rocm-rel-6.3.1/onnxruntime_rocm-1.19.0-cp310-cp310-linux_x86_64.whl) [Python 3.12](https://repo.radeon.com/rocm/manylinux/rocm-rel-6.3.1/onnxruntime_rocm-1.19.0-cp312-cp312-linux_x86_64.whl)|
|ONNX Runtime Version|MIGraphX ROCm Release| Python 3.8 | Python 3.10 | Python 3.12 |
|---|---|---|---|---|
|main|5.4||||
|1.14|5.4||||
|1.13|5.4||||
|1.13|5.3.2||||
|1.12|5.2.3||||
|1.12|5.2||||
|1.13|5.3||||
|1.14|5.4||||
|1.15|5.4.x||||
|1.16|6.0.0||||
|1.16|6.0.2||[3.10](https://repo.radeon.com/rocm/manylinux/rocm-rel-6.0.2/onnxruntime_rocm-inference-1.17.0-cp310-cp310-linux_x86_64.whl)||
|1.17|6.1| [3.8](https://repo.radeon.com/rocm/manylinux/rocm-rel-6.1/onnxruntime_rocm-inference-1.17.0-cp38-cp38-linux_x86_64.whl)| [3.10](https://repo.radeon.com/rocm/manylinux/rocm-rel-6.1/onnxruntime_rocm-inference-1.17.0-cp310-cp310-linux_x86_64.whl)||
|1.17|6.1.3|| [3.10](https://repo.radeon.com/rocm/manylinux/rocm-rel-6.1.3/onnxruntime_rocm-1.17.0-cp310-cp310-linux_x86_64.whl)||
|1.18|6.2| [3.8](https://repo.radeon.com/rocm/manylinux/rocm-rel-6.2/onnxruntime_rocm-1.18.0-cp38-cp38-linux_x86_64.whl)| [3.10](https://repo.radeon.com/rocm/manylinux/rocm-rel-6.2/onnxruntime_rocm-1.18.0-cp310-cp310-linux_x86_64.whl)||
|1.18|6.2.3|| [3.10](https://repo.radeon.com/rocm/manylinux/rocm-rel-6.2.3/onnxruntime_rocm-1.18.0-cp310-cp310-linux_x86_64.whl)||
|1.18|6.2.4|| [3.10](https://repo.radeon.com/rocm/manylinux/rocm-rel-6.2.4/onnxruntime_rocm-1.18.0-cp310-cp310-linux_x86_64.whl)||
|1.19|6.3|| [3.10](https://repo.radeon.com/rocm/manylinux/rocm-rel-6.3/onnxruntime_rocm-1.19.0-cp310-cp310-linux_x86_64.whl) | [3.12](https://repo.radeon.com/rocm/manylinux/rocm-rel-6.3/onnxruntime_rocm-1.19.0-cp312-cp312-linux_x86_64.whl)|
|1.19|6.3.1|| [3.10](https://repo.radeon.com/rocm/manylinux/rocm-rel-6.3.1/onnxruntime_rocm-1.19.0-cp310-cp310-linux_x86_64.whl) | [3.12](https://repo.radeon.com/rocm/manylinux/rocm-rel-6.3.1/onnxruntime_rocm-1.19.0-cp312-cp312-linux_x86_64.whl)|
|1.21|6.4.0|| [3.10](https://repo.radeon.com/rocm/manylinux/rocm-rel-6.4/onnxruntime_rocm-1.21.0-cp310-cp310-linux_x86_64.whl) | [3.12](https://repo.radeon.com/rocm/manylinux/rocm-rel-6.4/onnxruntime_rocm-1.21.0-cp312-cp312-linux_x86_64.whl)|

## Build
For build instructions, please see the [BUILD page](../build/eps.md#amd-migraphx).
Expand Down Expand Up @@ -84,8 +85,60 @@ You can check [here](https://github.com/scxiao/ort_test/tree/master/python/run_o
model on either the CPU or MIGraphX Execution Provider.


## Configuration Options
MIGraphX providers an environment variable ORT_MIGRAPHX_FP16_ENABLE to enable the FP16 mode.
## Session Variables

These flags can be invoked via the Provider Options struct [link] when creating an Onnxruntime Session Object and invoking the MIGraphXExecutionProvider

Items are added as a python dictionary when invoking the migraphx execution provider

|Session Option Flag | Values | Description|
|---|---|---|
| device_id | INT | Selecit the device ID specified for the session run (default will be device 0) |
| migraphx_fp16_enable | 1 or 0 | Enable FP16 quantization mode via the MIGraphX API of the input model|
| migraphx_int8_enable | 1 or 0 | Enable int8 static quantization mode of the input model via the MIGraphX API.\n Requires calibration table path vars to be set.|
| migraphx_int8_calibration_table_name | <absolute path to calibration table> | Path to a set of input calibration data for int8 static model quantization |
| migraphx_int8_use_native_calibration_table | 1 or 0 | Use a calibration table from Nvidia native int8 format or json dumped format|
| migraphx_save_compiled_model | 1 or 0 | Enable saving a model as an MIGraphX (.mxr) format after compile |
| migraphx_save_compiled_path | <string> - Path to write .mxr file (default is ./compiled_model.mxr) | Path where the MIGraphX compiled model is stored. |
| migraphx_load_compiled_model | 1 or 0 | Enable loading a model as an MIGraphX (.mxr) format after compile |
| migraphx_load_compiled_path | <string> - Path to read .mxr file (default is ./compiled_model.mxr) | Path where the MIGraphX compiled model is stored. |
| migraphx_exhaustive_tune | 1 or 0 (default 0) | Enable exhaustive tuning of parameters as part of compilation via the MIGraphX API. Adds additional compile time for a potential perf boost.|
| migraphx_mem_limit | INT | Set the memory limit used for memory arena. Default uses ORTs default_memory_arena_cfg value. |
| migraphx_external_alloc | Address | Address of external memory allocator function used for this EP. Useful for reading in larger models weights. |
| migraphx_external_free | Address | Address of external memory deallocator function used for this EP. Useful for unloadng what was allocated with the migraphx_external_alloc input |
| migraphx_external_empty_cache | Address | Address of external memory cache used for this model. Useful for caching results of externally allocated models. |


## Environment Variaables

Envirinoment variables can be invoked on a global level. These are typically used via

```
export ORT_MIGRAPHX_XXXXX = <value>
```

or invoked before running a binary as

```
ORT_MIGRAPHX_XXXX=<value> python3 example_script.py
```

This will start an inference session and superceed inputs invoked via Session()

|Environment Option Flag | Values | Description|
|---|---|---|
| ORT_MIGRAPHX_DUMP_MODEL_OPS | 1 or 0 | Enable dumping of model operators during parsing |
| ORT_MIGRAPHX_FP16_ENABLE | 1 or 0 | Enable FP16 quantization mode via the MIGraphX API of the input model|
| ORT_MIGRAPHX_INT8_ENABLE | 1 or 0 | Enable int8 static quantization mode of the input model via the MIGraphX API.\n Requires calibration table path vars to be set.|
| ORT_MIGRAPHX_INT8_CALIBRATION_TABLE_NAME | <absolute path to calibration table> | Path to a set of input calibration data for int8 static model quantization |
| ORT_MIGRAPHX_INT8_USE_NATIVE_CALIBRATION_TABLE | 1 or 0 | Use a calibration table from Nvidia native int8 format or |
| ORT_MIGRAPHX_SAVE_COMPILED_MODEL | 1 or 0 | Enable saving a model as an MIGraphX (.mxr) format after compile |
| ORT_MIGRAPHX_SAVE_COMPILED_PATH | <string> - Path to write .mxr file (default is ./compiled_model.mxr) | Path where the MIGraphX compiled model is stored. |
| ORT_MIGRAPHX_LOAD_COMPILED_MODEL | 1 or 0 | Enable loading a model as an MIGraphX (.mxr) format after compile |
| ORT_MIGRAPHX_LOAD_COMPILED_PATH | <string> - Path to read .mxr file (default is ./compiled_model.mxr) | Path where the MIGraphX compiled model is stored. |
| ORT_MIGRAPHX_EXHAUSTIVE_TUNE | 1 or 0 (default 0) | Enable exhaustive tuning of parameters as part of compilation via the MIGraphX API. Adds additional compile time for a potential perf boost.|



## Samples

Expand Down