Skip to content

[Triton/Gluon] gemm A16w16 tune for disable amdgpu-trackers - #5137

Open
yanxuer-999 wants to merge 3 commits into
ROCm:mainfrom
yanxuer-999:triton_3_8_rel_tune
Open

[Triton/Gluon] gemm A16w16 tune for disable amdgpu-trackers#5137
yanxuer-999 wants to merge 3 commits into
ROCm:mainfrom
yanxuer-999:triton_3_8_rel_tune

Conversation

@yanxuer-999

Copy link
Copy Markdown
Contributor

Motivation

Restore two triton 3.8 performance regressions on gfx950 by cherry-picking validated tuning fixes: paged-attention GQA decode (llama3, ~-18%) and GEMM-A16W16 at N=1280/K=8192 (tracker-off VGPR spill).

Technical Details

pa_decode.py: set waves_per_eu=2 on the two v1 w_dot decode kernels (non-quant + per-token FP8), so regalloc gets the full 256-VGPR budget instead of triton 3.8's inflated 202-VGPR/occupancy-2 state (s_waitcnt lgkmcnt(0) 72→50, s_nop 36→9); v2 w_dot is intentionally left unchanged because waves_per_eu=2 there spills and regresses.

GEMM-A16W16-N=1280-K=8192.json: new gfx950 config lowering waves_per_eu (M_LEQ_32 8→4, M_LEQ_256 6→4) to remove the tracker-off VGPR spill on this shape.

Test Result

llama3 pa_decode 8B/70B/405B recovered +43/+43/+42% (deepseek-V3/wo_dot unaffected at 0%); GEMM-A16W16 N=1280 problem M sizes recovered +242/+151/+157% (M=32/192/256) with no regression on untuned shapes (<2% spread).

yanxuer-999 and others added 2 commits August 31, 2026 01:18
…42% on llama3 GQA decode (triton 3.8 VGPR 160->202 occupancy drop)

Co-authored-by: Cursor <cursoragent@cursor.com>
(cherry picked from commit 429d355)
…Q_32 8->4, M_LEQ_256 6->4) to remove tracker-off VGPR spill

Cherry-picked from release/0829/aiter 34fae60; file remapped to the new
config layout (configs/gfx950/triton/gemm/gemm_a16w16/) since the flat
configs/gemm/gfx950-*.json path was reorganized on main.

Co-authored-by: Cursor <cursoragent@cursor.com>
@yanxuer-999
yanxuer-999 requested a review from a team August 31, 2026 06:25
@github-actions

Copy link
Copy Markdown
Contributor

🏷️ CI Guide

Runs automatically on every PR:

  • ✅ Pre-checks (submodule verification, code formatting)
  • ✅ Aiter op tests (gfx942 + gfx950)
  • ✅ Triton tests on MI35X (only when aiter/ops/triton/** or related paths are changed)

Extended tests (opt-in via labels):

Label Tests
ci:gfx1250-ffm-triton Run the five-shard gfx1250 FFM Triton test suite
ci:triton-300x Run an additional Triton test job on MI300X in PRs; main branch always runs both MI35X and MI300X
multigpu Aiter multi-GPU tests on the 8-GPU runner
ci:sglang SGLang integration tests: DeepSeek-R1-MXFP4 accuracy, Qwen 3.5 accuracy
ci:atom ATOM benchmark: DeepSeek-R1-0528, GPT-OSS-120B
ci:atom_full ATOM accuracy suite for PR and main models from ATOM models_accuracy.json
ci:vllm vLLM benchmark: GPT-OSS-120B, DeepSeek-R1-0528, Kimi-K2.5
ci:all All standard extended tests (excludes ci:atom_full)

Only add ci:atom_full for FlyDSL or Triton upgrades.
Add labels via the sidebar or gh pr edit 5137 --add-label <label>

PR title tags & labels:
Component tags ([Triton/Gluon], [HIP], [CK], [ASM], ...) are added to the PR title and as PR labels automatically from the changed files and re-synced on every push — change-type tags like [fix]/[Perf], op tags like [MLA], and human labels (ci:*) are left untouched. Add the no-auto-title label to opt this PR out.

@github-actions github-actions Bot changed the title [triton]gemm A16w16 tune for disable amdgpu-trackers [Triton/Gluon] gemm A16w16 tune for disable amdgpu-trackers Aug 31, 2026
@zufayu
zufayu requested review from a team and azaidy September 1, 2026 01:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants