[Kernel][Perf] Tune H20 block-FP8 MoE low-batch configs (+21%)

[liuyao0322]

Purpose

Retune the existing H20-3e block-quantized FP8 fused MoE configuration for E=256, N=256. This shape is used by GLM-5.3 with TP=8 (K=6144, top-k 8) and by DeepSeek-family models (K=7168).

The shipped low-batch entries still use BLOCK_SIZE_N=128. On H20-3e, narrower N tiles expose more thread blocks and improve SM utilization during decode. This PR extends the low-batch N=64 tuning established for H100/H200 in #46642 to the H20-3e config that PR did not modify.

Related optimization tracker: #46654.

Change

For keys M=1, 2, 4, and 8:

• Change BLOCK_SIZE_N from 128 to 64.
• Change M=1 num_stages from 3 to 4.
• Change M=8 num_stages from 4 to 3.
• Keep BLOCK_SIZE_M=16, BLOCK_SIZE_K=128, GROUP_SIZE_M=1, and num_warps=4.
• Leave every M>=16 entry unchanged.

The K=128 choices were cross-checked at both GLM's K=6144 and DeepSeek's K=7168 because the config filename is shared and does not encode K.

Performance

A/B runs used the official benchmarks/kernels/benchmark_moe.py harness on the same H20-3e node and latest upstream main (22df3a34e098738b9e6fbd09f63967d60acc24c9).

Environment:

• GPU: NVIDIA H20-3e, 143771 MiB
• Driver: 580.105.08
• PyTorch: 2.13.0+cu130
• Triton: 3.7.1
• Model shape: zai-org/GLM-5.3, TP=8, block-FP8 W8A8
• Slurm jobs: baseline 2089, tuned 2092, both on vllm-h20-01

M	Baseline (us)	Tuned (us)	Latency reduction	Baseline / tuned
1	47.54	38.39	19.25%	1.238x
2	55.75	52.15	6.46%	1.069x
3	67.10	63.56	5.28%	1.056x
4	76.59	72.76	5.00%	1.053x
6	94.82	90.79	4.25%	1.044x
8	139.95	110.41	21.11%	1.268x

M=3 and M=6 use the nearest tuned keys M=2 and M=4, respectively. A separate bounded CUPTI scan used CUDA graphs, cold L2, 30 samples, and a native Torch correctness reference. With the tuned file installed, the selected defaults were within 0.6% of the fastest scanned candidate for K=6144 and within 0.4% for K=7168. Numerical correctness passed for both K values.

M>=16 entries are unchanged. Their cross-allocation timing variation is treated as noise and is not claimed as a performance change.

Test Plan and Results

.venv/bin/python -m pytest tests/kernels/moe/test_block_fp8.py \
  -k "test_w8a8_block_fp8_fused_moe and 83 and 512" -x -q

Result on both the baseline and final allocations:

16 passed, 2 skipped, 310 deselected

Official kernel benchmark:

.venv/bin/python benchmarks/kernels/benchmark_moe.py \
  --model zai-org/GLM-5.3 \
  --tp-size 8 \
  --dtype fp8_w8a8 \
  --batch-size 1 2 3 4 6 8 16 24 96 384 1024

Additional validation:

• git diff --check: passed.
• .venv/bin/python -m json.tool <config>: passed.
• Commit-time pre-commit hooks: passed. A node-local PRE_COMMIT_HOME was used because the shared NFS pre-commit cache returned Errno 524 while building hook environments.
• K=6144 native-reference correctness: passed.
• K=7168 native-reference correctness: passed.

End-to-end model evaluation was not run. This is a configuration-only kernel tiling change and does not alter model code or output semantics; kernel numerical correctness was validated against the native Torch reference. The full GLM-5.3 checkpoint was not available in the cluster cache.

Duplicate-work Check

I inspected #46654 and searched open PRs by the issue number, exact H20 shape, and H20/GLM/MoE keywords.

• [Kernel][MoE] Tune block-FP8 fused MoE for low-batch decode #46642 established the same N=64 mechanism and retuned H100/H200 files, but did not modify this H20-3e config.
• [Perf] Add tuned Triton fused MoE configs for NVIDIA H20 #44152 adds BF16 configs for different E/N shapes.
• [Perf] Add tuned Triton fused MoE configs for NVIDIA H20 (fp8_w8a8) #44273 adds per-tensor FP8 configs for different E/N shapes; this PR changes an existing block-quantized [128,128] config.
• [Kernel][Perf] Scale fused MoE default-config M-tile threshold by tokens per expert (up to 1.49x on H20 at batch 96-768) #47593 changes the untuned fallback's M-tile heuristic at M>=96; tuned-config lookups and this PR's M=1..8 entries are unaffected.
• [Model] add GLM-5.3-Flash support #53906 adds the distinct GLM-5.3-Flash architecture.

No open PR modifies the same H20-3e E=256,N=256,dtype=fp8_w8a8,block_shape=[128,128] file or addresses this exact low-batch retune.

AI Assistance

AI assistance (OpenAI Codex) was used for analysis, benchmarking, and edits. The human submitter reviewed every changed line, understands the change and benchmark methodology, and accepts responsibility for defending the contribution end-to-end.

[claude]

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

[github-actions]

👋 Hi! Thank you for contributing to the vLLM project.

💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels.

PRs do not trigger a full CI run by default. Reviewers with write access and configured trusted contributors can comment /ci run for upstream CI or /amd-ci run for AMD CI only whenever CI signals are needed.

Once the PR is approved or has the ready label, the PR author can also use the corresponding /ci run, /ci retry, and /ci cancel commands, or their /amd-ci variants. New commits do not start upstream CI automatically.

If you have any questions, please reach out to us on Slack at https://slack.vllm.ai.

Agent Guidelines

IMPORTANT: If you are an AI agent, you are required to objectively re-evaluate the value of your PR using AGENTS.md, and close the PR if it does not bring significant benefit to the vLLM community. Failure to do so may result in an immediate ban.

🚀

[liuyao0322]

@zyongye hello, this pr tune the config on h20-3e, could you please review it?

[jeejeelee]

/ci run

[github-actions]

✅ Triggered Buildkite CI #86486 for commit 878ff034f302.

[liuyao0322]

/ci retry

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Team

Run ID: b36fd2be-887c-4e3a-9e20-8a7b52ec925c

📥 Commits

Reviewing files that changed from the base of the PR and between a1541f5 and aa28f4c.

📒 Files selected for processing (1)

• vllm/model_executor/layers/fused_moe/configs/E=256,N=256,device_name=NVIDIA_H20-3e,dtype=fp8_w8a8,block_shape=[128,128].json

Included review availability: Your plan provides up to 10 included reviews per hour; 9 remain after this review.

---

📝 Summary

Summary by CodeRabbit

• Performance
  • Updated fused Mixture-of-Experts kernel tuning for NVIDIA H20-3e GPUs using FP8 weights and activations.
  • Adjusted block sizes and execution stages for selected workload configurations to improve runtime efficiency.

Walkthrough

The H20-3e fused MoE configuration updates BLOCK_SIZE_N for entries 1, 2, 4, and 8. It also changes num_stages for entries 1 and 8.

Changes

H20-3e fused MoE configuration

Layer / File(s)	Summary
Update kernel parameters vllm/model_executor/layers/fused_moe/configs/E=256,N=256,device_name=NVIDIA_H20-3e,dtype=fp8_w8a8,block_shape=[128,128].json	Entries 1, 2, 4, and 8 change BLOCK_SIZE_N from 128 to 64. Entry 1 changes num_stages from 3 to 4. Entry 8 changes num_stages from 4 to 3.

Estimated code review effort: 1 (Trivial) | ~3 minutes

Merge Risk: ⚪ Minimal · up to aa28f

This updates low-batch H20-3e FP8 fused MoE kernel tuning while retaining the existing runtime configuration-selection behavior. No concrete merge-blocking risk remains.

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly and concisely describes the main change: performance tuning of low-batch H20 block-FP8 MoE kernel configurations.
Description check	✅ Passed	The description directly explains the H20-3e configuration changes, performance results, correctness validation, and test coverage.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff. Analyzed 0 functions across 0…
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}