You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
[MIGraphx][Documentation] Update Documentation for ROCm 6.4 and additional MIGraphX EP variables (#24663)
### Description
<!-- Describe your changes. -->
- Add info on MIGraphX EP session options controled by Provider Options
or global ENV variables
- Update links to repo.radeon for ROCm 6.4 as well as support matrix
- Add descriptions for each variable used to control MIGraphX EP
- Format table to use python version number and fix formatting between
available versions
- Add links to rocm/onnxruntime Dockerhub environments
- Link and reference repo.radeon.com for roll whls and other files.
### Motivation and Context
<!-- - Why is this change required? What problem does it solve?
- If it fixes an open issue, please link to the issue here. -->
Make our documentation better for users who want to use MIGraphX EP for
development
---------
Co-authored-by: Ted Themistokleous <tedthemistokleous@amd.com>
Copy file name to clipboardExpand all lines: docs/execution-providers/MIGraphX-ExecutionProvider.md
+90-27Lines changed: 90 additions & 27 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -16,6 +16,7 @@ The [MIGraphX](https://github.com/ROCmSoftwarePlatform/AMDMIGraphX/) execution p
16
16
17
17
* TOC placeholder
18
18
{:toc}
19
+
19
20
## Install
20
21
21
22
**NOTE** Please make sure to install the proper version of Pytorch specified here [PyTorch Version](../install/#training-install-table-for-all-languages).
@@ -26,36 +27,44 @@ Pre-built binaries of ONNX Runtime with MIGraphX EP are published for most langu
26
27
27
28
Since ROCm 6.0.2, AMD supplies pre-built python wheels hosted on (https://repo.radeon.com/rocm/manylinux)
28
29
30
+
## Build from source
31
+
For build instructions, please see the [BUILD page](../build/eps.md#amd-migraphx). Prebuild .whl files are provided below in the requirements section and are hosted on repo.radeon.com. Ubuntu based docker development environments are provided in the Docker Support section. New whl and dockers are published each ROCm release
32
+
29
33
## Requirements
30
34
31
-
Below is the matrix of supported ROCm versions corresponding to Ubuntu builds.
35
+
Below is the matrix of supported ROCm versions corresponding to Ubuntu builds.
32
36
33
-
Links for prebuild Python Wheels (.whl) are linked below corresponding to python versions for the host OS based on Ubuntu support.
37
+
As of ROCm 6.0.2 Links for prebuild Python Wheels (.whl) are linked below corresponding to python versions for the host OS based on Ubuntu support.
38
+
All links can be found on AMD's [repo.radeon manylinux page](https://repo.radeon.com/rocm/manylinux) for each corresponding to the ROCm release
For simple workloads and/or prototyping AMD creates a Docker Images based on Ubuntu using the latest ROCm release and Supported ROCm-Pytorch builds found at [ROCM Dockerhub](https://hub.docker.com/r/rocm/onnxruntime/tags)
65
+
66
+
The intent is to get users up and running with their custom workload in python and provides an environment of prebuild ROCm, Onnxruntime and MIGraphX packages required to get started without the need to build Onnxruntime
For build instructions, please see the [BUILD page](../build/eps.md#amd-migraphx).
59
68
60
69
## Usage
61
70
@@ -84,8 +93,62 @@ You can check [here](https://github.com/scxiao/ort_test/tree/master/python/run_o
84
93
model on either the CPU or MIGraphX Execution Provider.
85
94
86
95
87
-
## Configuration Options
88
-
MIGraphX providers an environment variable ORT_MIGRAPHX_FP16_ENABLE to enable the FP16 mode.
96
+
## Session Variables
97
+
98
+
These flags can be invoked via the Provider Options struct [link](https://github.com/ROCm/onnxruntime/blob/rocm6.4_internal_testing/include/onnxruntime/core/session/onnxruntime_c_api.h#L615) when creating an Onnxruntime Session Object and invoking the MIGraphXExecutionProvider
99
+
100
+
Items are added as a python dictionary when invoking the MIGraphX execution provider when using python
101
+
102
+
|Session Option Flag | Values | Description|
103
+
|---|---|---|
104
+
| device_id | INT | Select the device ID specified for the session run (default will be device 0) |
105
+
| migraphx_fp16_enable | 1 or 0 | Enable FP16 quantization mode via the MIGraphX API of the input model. |
106
+
| migraphx_int8_enable | 1 or 0 | Enable int8 static quantization mode of the input model via the MIGraphX API. Requires calibration table path vars to be set (migraphx_int8_calibration_table_name=valid path).|
107
+
| migraphx_int8_calibration_table_name | <absolute path to calibration table> | Path to a set of input calibration data for int8 static model quantization. |
108
+
| migraphx_int8_use_native_calibration_table | 1 or 0 | Use a calibration table from Nvidia native int8 format or json dumped format. |
109
+
| migraphx_save_compiled_model | 1 or 0 | Enable saving a model as an MIGraphX (.mxr) format after compile when set to 1 |
110
+
| migraphx_save_compiled_path | path string | Path to write .mxr file (default is ./compiled_model.mxr). Path where the MIGraphX compiled model is stored. Requires migraphx_save_compiled_path to be set for this session |
111
+
| migraphx_load_compiled_model | 1 or 0 | Enable loading a model as an MIGraphX (.mxr) format after compile when set to 1 |
112
+
| migraphx_load_compiled_path | path string | Path to read .mxr file (default is ./compiled_model.mxr). Path where the MIGraphX compiled model is stored. |
113
+
| migraphx_exhaustive_tune | 1 or 0 (default 0) | Enable exhaustive tuning of parameters as part of compilation via the MIGraphX API. Adds additional compile time for a potential perf boost.|
114
+
| migraphx_mem_limit | INT | Set the memory limit used for memory arena. Default uses ORTs default_memory_arena_cfg value. |
115
+
| migraphx_external_alloc | Address | Address of external memory allocator function used for this EP. Useful for reading in larger models weights. |
116
+
| migraphx_external_free | Address | Address of external memory deallocator function used for this EP. Useful for unloadng what was allocated with the migraphx_external_alloc input. |
117
+
| migraphx_external_empty_cache | Address | Address of external memory cache used for this model. Useful for caching results of externally allocated models. |
118
+
119
+
120
+
## Environment Variables
121
+
122
+
Environment variables can be invoked on a global level. These are typically used via:
This will start an inference session and supersede inputs invoked via 'Session()'.
135
+
136
+
Users can invoke Environment and Session variables in the same run but Environment variables will take precident.
137
+
138
+
|Environment Option Flag | Values | Description|
139
+
|---|---|---|
140
+
| ORT_MIGRAPHX_DUMP_MODEL_OPS | 1 or 0 | Enable dumping of model operators during parsing. |
141
+
| ORT_MIGRAPHX_FP16_ENABLE | 1 or 0 | Enable FP16 quantization mode via the MIGraphX API of the input model. |
142
+
| ORT_MIGRAPHX_INT8_ENABLE | 1 or 0 | Enable int8 static quantization mode of the input model via the MIGraphX API.\n Requires calibration table path vars to be set (migraphx_int8_calibration_table_name=<valid path>).|
143
+
| ORT_MIGRAPHX_INT8_CALIBRATION_TABLE_NAME | <absolute path to calibration table> | Path to a set of input calibration data for int8 static model quantization. |
144
+
| ORT_MIGRAPHX_INT8_USE_NATIVE_CALIBRATION_TABLE | 1 or 0 | Use a calibration table from Nvidia native int8 format or json dumped format. |
145
+
| ORT_MIGRAPHX_SAVE_COMPILED_MODEL | 1 or 0 | Enable saving a model as an MIGraphX (.mxr) format after compile. |
146
+
| ORT_MIGRAPHX_SAVE_COMPILED_PATH | <string> - Path to write .mxr file (default is ./compiled_model.mxr) | Path where the MIGraphX compiled model is stored. Requires ORT_MIGRAPHX_SAVE_COMPILED_MODEL to be set to 1. |
147
+
| ORT_MIGRAPHX_LOAD_COMPILED_MODEL | 1 or 0 | Enable loading a model as an MIGraphX (.mxr) format after compile. |
148
+
| ORT_MIGRAPHX_LOAD_COMPILED_PATH | <string> - Path to read .mxr file (default is ./compiled_model.mxr) | Path where the MIGraphX compiled model is stored. Requires ORT_MIGRAPHX_LOAD_COMPILED_MODEL to be set to 1. |
149
+
| ORT_MIGRAPHX_EXHAUSTIVE_TUNE | 1 or 0 (default 0) | Enable exhaustive tuning of parameters as part of compilation via the MIGraphX API. Adds additional compile time for a potential perf boost. |
0 commit comments