You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
@@ -110,14 +111,17 @@ Items are added as a python dictionary when invoking the MIGraphX execution prov
110
111
|---|---|---|
111
112
| device_id | INT | Select the device ID specified for the session run (default will be device 0) |
112
113
| migraphx_fp16_enable | 1 or 0 | Enable FP16 quantization mode via the MIGraphX API of the input model. |
114
+
| migraphx_bf16_enable | 1 or 0 | Enable bf16 quantization mode via the MIGraphX API of the input model. |
113
115
| migraphx_int8_enable | 1 or 0 | Enable int8 static quantization mode of the input model via the MIGraphX API. Requires calibration table path vars to be set (migraphx_int8_calibration_table_name=valid path).|
116
+
| migraphx_fp8_enable | 1 or 0 | Enable fp8 static quantization mode of the input model via the MIGraphX API. Requires calibration table path vars to be set (migraphx_int8_calibration_table_name=valid path).|
114
117
| migraphx_int8_calibration_table_name | <absolute path to calibration table> | Path to a set of input calibration data for int8 static model quantization. |
115
118
| migraphx_int8_use_native_calibration_table | 1 or 0 | Use a calibration table from Nvidia native int8 format or json dumped format. |
116
119
| migraphx_exhaustive_tune | 1 or 0 (default 0) | Enable exhaustive tuning of parameters as part of compilation via the MIGraphX API. Adds additional compile time for a potential perf boost.|
117
120
| migraphx_mem_limit | INT | Set the memory limit used for memory arena. Default uses ORTs default_memory_arena_cfg value. |
118
121
| migraphx_external_alloc | Address | Address of external memory allocator function used for this EP. Useful for reading in larger models weights. |
119
122
| migraphx_external_free | Address | Address of external memory deallocator function used for this EP. Useful for unloadng what was allocated with the migraphx_external_alloc input. |
120
123
| migraphx_external_empty_cache | Address | Address of external memory cache used for this model. Useful for caching results of externally allocated models. |
124
+
121
125
| | | |
122
126
| Depricated | Release Removed | Description |
123
127
| migraphx_save_compiled_model | ROCm 6.4 | Enable saving a model as an MIGraphX (.mxr) format after compile when set to 1 |
@@ -149,12 +153,14 @@ Users can invoke Environment and Session variables in the same run but Environme
149
153
|---|---|---|
150
154
| ORT_MIGRAPHX_DUMP_MODEL_OPS | 1 or 0 | Enable dumping of model operators during parsing. |
151
155
| ORT_MIGRAPHX_FP16_ENABLE | 1 or 0 | Enable FP16 quantization mode via the MIGraphX API of the input model. |
156
+
| ORT_MIGRAPHX_BF16_ENABLE | 1 or 0 | Enable BF16 quantization mode via the MIGraphX API of the input model. |
152
157
| ORT_MIGRAPHX_INT8_ENABLE | 1 or 0 | Enable int8 static quantization mode of the input model via the MIGraphX API.\n Requires calibration table path vars to be set (migraphx_int8_calibration_table_name=<valid path>).|
158
+
| ORT_MIGRAPHX_FP8_ENABLE | 1 or 0 | Enable fp8 static quantization mode of the input model via the MIGraphX API.\n Requires calibration table path vars to be set (reuses migraphx_int8_calibration_table_name=<valid path>).|
153
159
| ORT_MIGRAPHX_INT8_CALIBRATION_TABLE_NAME | <absolute path to calibration table> | Path to a set of input calibration data for int8 static model quantization. |
154
160
| ORT_MIGRAPHX_INT8_USE_NATIVE_CALIBRATION_TABLE | 1 or 0 | Use a calibration table from Nvidia native int8 format or json dumped format. |
155
161
| ORT_MIGRAPHX_EXHAUSTIVE_TUNE | 1 or 0 (default 0) | Enable exhaustive tuning of parameters as part of compilation via the MIGraphX API. Adds additional compile time for a potential perf boost. |
156
-
| ORT_MIGRAPHX_MODEL_CACHE_PATH | <string> | Path to read and write model specific data such as weights or other model specific data |
157
-
| ORT_MIGRAPHX_MODEL_PATH | <string> | Path to read and write .mxr path occurs after MIGraphX model compile complete |
162
+
| ORT_MIGRAPHX_CACHE_PATH | <string> | Path to read and write model specific data such as weights or other model specific data |
163
+
| ORT_MIGRAPHX_MODEL_CACHE_PATH | <string> | Path to read and write .mxr path occurs after MIGraphX model compile complete |
158
164
| | | |
159
165
| Depricated | ROCm Version removed | Description |
160
166
| ORT_MIGRAPHX_SAVE_COMPILED_MODEL | ROCm 6.4 | Enable saving a model as an MIGraphX (.mxr) format after compile. ( 0 or 1) |
0 commit comments