Skip to content

Commit 878ff03

Browse files
committed
[Kernel][Perf] Tune H20 block-FP8 MoE for low-batch decode
Retune the E=256, N=256 H20-3e block-FP8 configuration used by GLM-5.3 TP8 and DeepSeek models. Narrower N tiles improve SM utilization for M=1 through M=8 while leaving larger batches unchanged. AI assistance was used for analysis, benchmarking, and edits. Co-authored-by: Codex <noreply@openai.com> Signed-off-by: liuyao0322 <yaoliu548926@gmail.com>
1 parent 22df3a3 commit 878ff03

1 file changed

Lines changed: 6 additions & 6 deletions

File tree

vllm/model_executor/layers/fused_moe/configs/E=256,N=256,device_name=NVIDIA_H20-3e,dtype=fp8_w8a8,block_shape=[128,128].json

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -1,35 +1,35 @@
11
{
22
"1": {
33
"BLOCK_SIZE_M": 16,
4-
"BLOCK_SIZE_N": 128,
4+
"BLOCK_SIZE_N": 64,
55
"BLOCK_SIZE_K": 128,
66
"GROUP_SIZE_M": 1,
77
"num_warps": 4,
8-
"num_stages": 3
8+
"num_stages": 4
99
},
1010
"2": {
1111
"BLOCK_SIZE_M": 16,
12-
"BLOCK_SIZE_N": 128,
12+
"BLOCK_SIZE_N": 64,
1313
"BLOCK_SIZE_K": 128,
1414
"GROUP_SIZE_M": 1,
1515
"num_warps": 4,
1616
"num_stages": 4
1717
},
1818
"4": {
1919
"BLOCK_SIZE_M": 16,
20-
"BLOCK_SIZE_N": 128,
20+
"BLOCK_SIZE_N": 64,
2121
"BLOCK_SIZE_K": 128,
2222
"GROUP_SIZE_M": 1,
2323
"num_warps": 4,
2424
"num_stages": 3
2525
},
2626
"8": {
2727
"BLOCK_SIZE_M": 16,
28-
"BLOCK_SIZE_N": 128,
28+
"BLOCK_SIZE_N": 64,
2929
"BLOCK_SIZE_K": 128,
3030
"GROUP_SIZE_M": 1,
3131
"num_warps": 4,
32-
"num_stages": 4
32+
"num_stages": 3
3333
},
3434
"16": {
3535
"BLOCK_SIZE_M": 16,

0 commit comments

Comments
 (0)