Is there any plan to support mxfp4 for MI300X?

[ChangseokSong]

hello,

I saw AMD Instinct Development Roadmap (2025Q4) (#12890) and it mentioned additional quantization support

• More MxFP4/FP8 enablement

Does it include mxfp4 support for MI300X?

p.s.
I did a simple test on MI300X with following patch. (usnig lmsysorg/sglang:v0.5.5.post2-rocm700-mi30x)
It seems to work fine.

diff --git a/python/sglang/srt/utils/common.py b/python/sglang/srt/utils/common.py
index f1bee1b14..19153354f 100644
--- a/python/sglang/srt/utils/common.py
+++ b/python/sglang/srt/utils/common.py
@@ -3248,7 +3248,7 @@ def mxfp_supported():
     """
     if torch.version.hip:
         gcn_arch = torch.cuda.get_device_properties(0).gcnArchName
-        return any(gfx in gcn_arch for gfx in ["gfx95"])
+        return any(gfx in gcn_arch for gfx in ["gfx95", "gfx94"])
     else:
         return False

benchmark result (from https://github.com/sgl-project/sglang/blob/main/test/srt/test_gpt_oss_1gpu.py)

{'chars': 11160.454545454546, 'chars:std': 4283.374854822279, 'score:std': 0.48631931786709987, 'score': 0.3838383838383838}
Writing results to /tmp/gpqa_openai_gpt-oss-20b.json
Total latency: 164.521 s
Score: 0.384
Evaluation end: model=openai/gpt-oss-20b reasoning_effort=high expected_score=0.27 metrics={'chars': 11160.454545454546, 'chars:std': 4283.374854822279, 'score:std': 0.48631931786709987, 'score': 0.3838383838383838}

[Shayanthn]

Yes ! MI300X (gfx94x) does support MXFP4 at the hardware level. The current check in mxfp_supported() is just too strict. Adding "gfx94" is correct since MI300X devices report gfx94*, and the MXFP4/FP8 path is already functional there.

Your benchmark results make sense, and it's expected that the patch works. We should update the detection logic in sglang to include MI300X officially.

[fortuneguy97]

ROCm Support for MXFP4

According to AMD’s ROCm 7 solutions brief, ROCm 7 does introduce support for “low-precision … MXFP4” on MI300X.
AMD

So, at the hardware/software layer, there's some foundation for MXFP4 on MI300X.

FP4 Inference on MI300X via “Petit”

A project called Petit (by LMSYS) provides optimized mixed-precision kernels to run FP4 models on AMD MI300 series.
LMSYS

However, “Petit” doesn’t use native MXFP4 matmul on MI300X — instead, it dequantizes FP4 weights into BF16/FP16 for computation.
LMSYS

That means you're not truly running in MXFP4 arithmetic, but converting on the fly.

vLLM Status

On the vLLM side: the vLLM ROCm inference docs explicitly say that MXFP4 is supported only on MI355X and MI350X.
ROCm Docs

There is also a GitHub issue in llm-compressor about adding MXFP4 support (both dense & MoE).
GitHub

FP8 is already a feature request / topic for MI300X support in vLLM/ROCm.
GitHub

Model / Framework Support

For Llama 4, AMD + vLLM have optimized kernels for MI300X (but using BF16 in the published blog).
ROCm Blogs

On the quantization/model front, gpt-oss (which uses MXFP4 for its MoE weights) is supported by vLLM on MI300X, but in their blog post they mention Blackwell and Hopper GPUs, not AMD.
vLLM Blog

For llama.cpp, there is support for native MXFP4 (ggml backends) but that’s more on the CPU/CUDA/Vulkan sides, not necessarily ROCm.
GitHub