This guide outlines the technical requirements for deploying GPU-accelerated ONNX inference on Nobara Linux (Fedora-based) using ROCm 7.1 and the MIGraphX engine.

I had a really hard time running gpu accel on my RX 6600m GPU, so i thought to document it.

1. System Dependencies

Install the required ROCm libraries and security utilities via DNF.

sudo dnf install miopen migraphx-devel rocm-hip-devel patchelf

Ensure the user is in the render group to access the Kernel Fusion Driver (KFD).

2. Python Environment

Initialize a Python 3.12 virtual environment. Avoid onnxruntime-rocm as it is deprecated in favor of onnxruntime-migraphx for ROCm 7.1+.

python3.12 -m venv venv
source venv/bin/activate
pip install --upgrade pip setuptools wheel
pip install onnxruntime-migraphx numpy opencv-python

3. Security Fix: Executable Stack

Nobara’s security policy rejects the onnxruntime-migraphx shared object due to an executable stack flag. You must strip this flag manually or the import will fail with an "Invalid argument" error.

patchelf --clear-execstack venv/lib64/python3.12/site-packages/onnxruntime/capi/onnxruntime_pybind11_state.so

4. Hardware-Specific Configuration

The RX 6600M (gfx1031) is not natively supported by ROCm's default kernels. You must force the system to use the gfx1030 (Navi 21) instruction set.

Environment Variables:

• HSA_OVERRIDE_GFX_VERSION=10.3.0: Forces RDNA 2 compatibility.
• ORT_MIGRAPHX_CACHE_PATH: Directory to save kernel tuning (prevents long re-compilation).

5. Script Implementation

Use the MIGraphXExecutionProvider. Note that in version 1.23.2+, provider options like migraphx_save_models are deprecated in favor of environment variables.

import onnxruntime as ort

providers = [
    ('MIGraphXExecutionProvider', {
        'device_id': 0,
        'migraphx_fp16_enable': True, # Reduces VRAM by ~50%
        'migraphx_exhaustive_tune': True
    }),
    'CPUExecutionProvider'
]

session = ort.InferenceSession("model.onnx", providers=providers)

6. Execution Command

Use this optimized command to run the project. The first run will take ~10 minutes to tune GEMM kernels; subsequent runs will be near-instant if the cache path is set.

HSA_OVERRIDE_GFX_VERSION=10.3.0 \
ORT_MIGRAPHX_CACHE_PATH=.cache \
ORT_MIGRAPHX_MODEL_CACHE_PATH=.cache \
python tagger.py --model model.onnx --dir images

Comparison: CPU vs. GPU

Previously i ran everything via CPU to run an image classifier model, and it took around 1 minute per 20 images.

Now, via GPU, after the long tuning, it took literally 20 seconds for 100 images.