[Research] LLM GRPO Training on RX 7900 XTX — 3 Actionable Findings for RDNA3 Optimization

[ChenHongYu2026]

[Research] LLM GRPO Training on RX 7900 XTX — 3 Actionable Findings for RDNA3 Optimization

TL;DR: I completed a 15-day systematic efficiency study of LLM post-training (GRPO/RLVR) on a single consumer RX 7900 XTX — the most comprehensive open-source measurement on consumer RDNA3 that I could find. Here are 3 findings with raw data that may inform your consumer-GPU AI roadmap. No ask — just sharing.

---

Context

Item	Detail
GPU	AMD Radeon RX 7900 XTX (RDNA3, gfx1100, 24 GB GDDR6, 960 GB/s)
Workload	GRPO (RLVR) training, Qwen2.5-3B-Instruct, GSM8K
Framework	Unsloth 2026.7.4 + TRL 0.24 + ROCm 7.2.4
Duration	15 days, single card, solo researcher
Output	2 papers (15pp + 6pp), open-source code + data

Links:

• 📄 Paper (DOI): 10.5281/zenodo.21695686
• 💻 Code: github.com/ChenHongYu2026/llm-training-on-consumer-amd

---

Finding 1: 4-bit Quantization is Net-Negative on RDNA3

Configuration	Throughput	vs BF16
Unsloth BF16	129.4 tok/s	baseline
Unsloth NF4 (4-bit)	76.8 tok/s	−40.7%

Root cause: RDNA3 has no dedicated INT4 compute units. Every NF4→BF16 dequantization before GEMM is pure overhead with no hardware acceleration — unlike NVIDIA's INT4 Tensor Cores (1248 TOPS on A100) that absorb dequant cost within the compute pipeline.

Implication: QLoRA — the go-to technique for consumer-GPU fine-tuning — is actively harmful on RDNA3 for training workloads. A WMMA-native INT4 path or a fused dequant+GEMM kernel would unlock this segment.

Recommendation: Always prefer BF16 over 4-bit on RDNA3 for training. The VRAM "savings" (10.2 GB vs 13.9 GB = 27% reduction) are illusory because activations + optimizer state dominate at 3B scale.

---

Finding 2: Batch-1 GEMV Utilization is ~29% — The Fundamental Decoding Ceiling

Probe	Bandwidth Achieved	% of Peak (960 GB/s)
Pure memcpy (2 GB)	742 GB/s	77%
Model decode (batch=1)	280 GB/s	29%
Isolated GEMV [32768²]	88 GB/s	9%
GEMV + HIP graph capture	88 GB/s	9% (graph = no improvement)

Root cause: Batch-1 GEMV has arithmetic intensity ≈ 1 FLOP/byte. The WMMA execution path on RDNA3 cannot saturate bandwidth at this intensity — occupancy and memory access patterns are the bottleneck, not kernel launch overhead (graph capture yields zero improvement).

Scaling with batch size (the optimization lever):

Decode Batch	Aggregate tok/s	MFU (practical)	Scaling Efficiency
1	46.8	0.28%	1.00
4	172.5	1.05%	0.92
16	553.3	3.36%	0.74
64	1334.2	8.11%	0.45

Implication: Speculative decoding on RDNA3 caps at ~1.5× (confirmed across 4 methods: draft model, FLy, Medusa, EAGLE). The real lever is generation-batch scaling — decoupling it from the backward micro-batch raises GRPO training throughput to 795.5 tok/s (97.6% of the memory-wall-constrained optimum).

---

Finding 3: Unsloth Kernel Fusion Yields Only +5.2% on RDNA3

Framework	Throughput	MFU (practical)	VRAM
Unsloth BF16	129.4 ± 2.6 tok/s	2.36%	13.9 GB
HF Native BF16	123.1 ± 0.8 tok/s	2.24%	8.0 GB
Speedup	+5.2% (Cohen's d > 3)		+73.8% VRAM overhead

Root cause: Unsloth's fused kernels eliminate intermediate memory round-trips — highly effective on HBM (5.3 TB/s on MI300X) where memory traffic is the bottleneck. On GDDR6 (960 GB/s), the WMMA compute path is already the dominant constraint; fusing memory operations yields diminishing returns.

Implication: The "2× speedup" marketing (benchmarked on CDNA/Instinct) does not transfer to consumer RDNA3. Framework selection on consumer AMD contributes < 6% throughput variation — hyperparameter tuning (sequence length, generation batch) yields 30%+ improvements.

---

Bonus: 19 Documented Pitfalls for RDNA3 Training

The full paper includes a comprehensive pitfall-to-solution table. Top 5 most impactful:

#	Pitfall	Solution
1	HSA_OVERRIDE_GFX_VERSION missing → silent crashes	Set to 11.0.0
2	Ryzen APU iGPU steals GPU 0 → OOM on 15 GB shared mem	HIP_VISIBLE_DEVICES=0
3	Default allocator fragmentation (3.9 GB wasted)	expandable_segments:True
4	torch.compile(reduce-overhead) = zero improvement on decode	Don't bother; use static KV cache instead (+28%)
5	Flash Attention kernel crash on Gemma/LLaMA	attn_implementation='eager'

---

Methodology Note

All throughput values use a post-hoc corrected measurement caliber. Our initial profiler had a cumulative-token-counting bug (inflating values 5.5–10.6× depending on logging frequency). The correction (final cumulative tokens / wall time) was verified against per-step completion-length reconstruction. We report this openly as part of our validity protocol — measurement-pipeline bugs are exactly what per-step validity checking is designed to catch.

---

What Would Help the Consumer RDNA3 AI Ecosystem

Based on 15 days of measurement, the highest-impact improvements would be:

1. Fused NF4→BF16 dequant + GEMM kernel → unlock QLoRA for consumer segment
2. WMMA occupancy optimization for low-arithmetic-intensity GEMV → raise decode efficiency from 29% toward 50%+
3. Official continuous-batching support in ROCm inference stack → the primary throughput lever for RL training generation phases
4. Consumer-GPU benchmarks in ROCm documentation → current docs only cover Instinct; community is flying blind

---

Reproducibility

Everything is open-source with a Makefile one-liner to reproduce:

git clone https://github.com/ChenHongYu2026/llm-training-on-consumer-amd
cd llm-training-on-consumer-amd
make smoke   # 20-step verification (~3 min)
make train   # full 500-step optimal run (~8.5 hours)

Happy to share raw JSON benchmark data or discuss methodology if useful. No ask — just thought the data might save someone a few weeks of investigation on consumer RDNA3.

Best,
Hongyu Chen

---

DOI: 10.5281/zenodo.21695686 | Code: GitHub