(ROCm) AMD-GPU-Boost: Unlock Full Performance on Consumer AMD GPUs

[Painter3000]

https://github.com/Painter3000/AMD-GPU-BOOST

I've been frustrated with AMD GPU performance in AI/ML applications - my RX 6800 XT was only using ~25% of its potential in PyTorch. The issue? ROCm was designed for MI-series enterprise GPUs and severely underdetects consumer GPU capabilities.

ROCm reports only 36 compute units instead of 72, and uses warp size 32 instead of the optimal 64 for RDNA2/3. This affects the entire RX 6000/7000 series.

AMD-GPU-BOOST fixes this at runtime by monkey-patching PyTorch's device detection. Results: - 4x performance improvement in inference - "NVIDIA-only" apps now run perfectly on AMD - Works with ComfyUI, Stable Diffusion, WAN 2.1, etc.

The tool includes a GUI installer for easy Pinokio integration and supports 18+ GPU models from RX 6400 to RX 7900 XTX.

This has been a major pain point for the AMD AI community - curious what the HN crowd thinks about runtime hardware detection fixes vs. proper driver/framework solutions.

Demo: My RX 6800 XT went from 1152 threads (36×32) to 4608 threads (72×64) - exactly what the hardware specs promise.

“What do you think – is runtime patching a legitimate option or should AMD urgently address this issue?”

[schung-amd]

Hi @Painter3000, thanks for the interest in improving performance on RDNA devices! Did you actually benchmark real-world performance? I think part of this stems from a common point of confusion. The reported 36 multiprocessors is not the CU count but rather the workgroup count, as gfx10 and gfx11 devices (we don't specify gfx12 in the docs, I'll have to check) are in WGP mode by default; see https://rocm.docs.amd.com/projects/HIP/en/latest/how-to/hip_rtc.html#cu-mode-vs-wgp-mode and the footnotes in https://rocm.docs.amd.com/projects/HIP/en/latest/reference/hardware_features.html#id13. In this mode, the CUs are grouped in pairs (hence the reported multiprocessor count being half of the CU count for the device) and can execute twice as many wavefronts simultaneously, so the performance is not cut in half by doing this. The actual performance impact is application-dependent, and these devices do still support CU mode, so you can experiment with overriding to CU mode on a per-application basis.

CU mode can be enabled at runtime by setting GPU_ENABLE_WGP_MODE=0. After doing this, torch.cuda.get_device_properties().multi_processor_count shows the number of CUs (i.e. on my 7900XTX it shows 96). I tested this with a GEMM-heavy benchmark out of curiosity, and it provides a ~1% speedup.

As for the wavefront size, gfx10 and gfx11 devices (again, will have to check for gfx12) have a native wavefront size of 32 but there is a compiler flag (-mwavefrontsize64) to force wavefront size 64. There are tradeoffs; see pages 17 and 18 on https://gpuopen.com/download/RDNA_Architecture_public.pdf. You might see a performance boost in some applications specifically designed to utilize wavefront size 64 but I would not expect it to be near a 2x boost. I also believe there might be some compatibility issues with cross-wave instructions, but I'm a bit fuzzy on the details at the moment.

It's cool that you're looking into this, but I encourage benchmarking real-world performance as it's difficult to extrapolate from the multiprocessor count and wavefront size. You'll find hardware limitations, memory access pattern changes, etc. will get in the way. And if at the end of the day you do see a performance increase from these overrides, that's great! As mentioned the CU/WGP mode difference is a common point of confusion, so this could be a convenient way for users to apply these overrides for themselves, even if it just gives a small boost and not 4x.

[Painter3000]

Hi @schung-amd, thank you for the detailed explanation and the references – this really helps clarify the CU vs. WGP abstraction. I understand now that reporting 36 "multiprocessors" on my RX 6800 XT corresponds to 36 WGPs rather than 72 CUs, and that WGP mode is not supposed to mean half the performance, since each WGP can schedule two Wave32 concurrently.

That said, my motivation for overriding the reported values came from benchmarking real workloads. In PyTorch (and also xFormers/FlashAttention kernels), the scheduling logic assumes an NVIDIA-like SM model:

multi_processor_count is taken as the number of available SMs,

warp_size is assumed to be 64 in most CUDA-style kernels.

When these libraries see only 36 “SMs” with Wave32, they often under-schedule workgroups. With my override (multi_processor_count=72, warp_size=64), the libraries align better with the expected CUDA model.

For example, in a 1024×1024 GEMM benchmark (averaged over 50 runs on my RX 6800 XT), I measured:

Default ROCm/Torch (36 WGP, Wave32): ~0.55–0.65 ms

With override (72, Wave64): ~0.16–0.18 ms

rocBLAS (native): ~0.139 ms

So in practice, the override produces a ~3–4× speedup for small/medium matmul kernels in PyTorch, bringing it much closer to the rocBLAS baseline. For larger GEMMs (e.g., 4096×4096), the gains are smaller but still measurable, as memory bandwidth and cache utilization dominate.

I appreciate your clarification that WGP mode is not inherently “half performance” and that Wave32 is the native mode for RDNA. What I am doing is less about changing the hardware mode itself, and more about aligning the software expectations so that CUDA-centric libraries don’t underutilize the GPU.

Thanks again for the insight – I’ll continue to benchmark more real-world cases to see where the override helps and where it might not.

[schung-amd]

Awesome, I'd love to see if we can get more real-world speedups with these overrides, and if I get some spare bandwidth I'll look into doing some experiments on our end. As mentioned there might be some incompatibilities with certain instructions and Wave64, but they should be apparent if we ever run into them in real-world usecases.

At the very least we should probably expand the documentation around CU mode, especially if we see it giving concrete performance gains in common workloads.

[Painter3000]

Hi @schung-amd, thanks again for the thoughtful insights!

After further benchmarking, I believe the performance discrepancy is actually rooted in older default PyTorch versions rather than the override itself. I recently tested with the nightly build 2.9.0.dev20250824+rocm6.4, and even without any Patch, I achieved ~0.18 ms on a 1024×1024 GEMM benchmark. With the patch applied, the time dropped slightly further to ~0.153 ms.

In contrast, older stable releases like 2.4.1+rocm6.0 and 2.5.1+rocm6.2 consistently measured between 0.241 ms and 0.259 ms for the same workload — nearly 40–50% slower.

So for anyone looking to get the most out of RDNA GPUs, I’d strongly recommend trying the latest ROCm nightly builds:

pip install --pre torch --index-url https://download.pytorch.org/whl/nightly/rocm6.4/
pip install --pre torchvision --index-url https://download.pytorch.org/whl/nightly/rocm6.4/
pip install --pre torchaudio --index-url https://download.pytorch.org/whl/nightly/rocm6.4/

This alone can yield a significant speedup — even before applying any overrides. Patch remains useful for older versions or for aligning CUDA-style kernels with RDNA hardware, but the newer builds are clearly closing the gap.

Thanks again for engaging — I’ll keep testing and sharing results as they evolve!

[alain-bkr]

nearly one year old.

Does this have been integrated in a version rocm , if yes which one ?
Or does RX6800XT still need some hack to fully work ?

[schung-amd]

Don't think this has been tested in a while. We've done a lot of work on ROCm since the original submission. Most of the perf difference at the time was due to torch version, and we have been working on our GEMMs since so I would expect the gap to have shrunk even further.

[Painter3000]

Hi, thanks for asking. The short answer is: no, AMD-GPU-BOOST was not integrated into ROCm as a real fix, because the original assumption behind the project turned out to be wrong. At the time I interpreted multi_processor_count = 36 on the RX 6800 XT as ROCm detecting only 36 of the 72 Compute Units. That was incorrect. On RDNA/RDNA2 this value maps much better to WGPs, and one WGP contains two Compute Units. So 36 WGPs correspond to the expected 72 CUs. The same applies to the warp/wavefront size. I originally assumed that RDNA2 should be forced to 64, but ROCm/HIP commonly reports 32 for RDNA, and that is not automatically wrong or “half performance”. RDNA supports Wave32 and Wave64, but Wave32 is normal for many compute paths. So the RX 6800 XT does not need this hack to “fully unlock” the GPU. If ROCm/PyTorch supports your setup properly, the card should work without AMD-GPU-BOOST. Performance improvements since then mostly came from newer ROCm/PyTorch versions and better kernels/libraries, not from this monkey patch. I would not recommend using AMD-GPU-BOOST as a general fix anymore. At most, it may still be useful as an experimental workaround for badly written CUDA-oriented Python code that makes poor scheduling decisions based on torch.cuda.get_device_properties(). But it should not be treated as required for normal ROCm operation. Best recommendation: use a recent ROCm/PyTorch build for gfx1030, benchmark your real workload, and only consider patches if a specific application has a proven device-property heuristic bug.       Gesendet: Donnerstag, 18. Juni 2026 um 15:32 Von: "ALAIN BKR" ***@***.***> An: ROCm/ROCm ***@***.***> CC: Painter3000 ***@***.***>,Mention ***@***.***> Betreff: Re: [ROCm/ROCm] (ROCm) AMD-GPU-Boost: Unlock Full Performance on Consumer AMD GPUs (Discussion #5226)   nearly one year old. Does this have been integrated in a version rocm , if yes which one ? Or does RX6800XT still need some hack to fully work ? — Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications, keep track of coding agent tasks and review pull requests on the go with GitHub Mobile for iOS and Android. Download it today! You are receiving this because you were mentioned.Message ID: ***@***.***>