MI355 GPT-OSS-120B: fix six deadlocks and two silent MFMA hazards; recover the latency cost; add perplexity harness - #6

Merged

sangeeta0201 merged 25 commits into

ROCm:amd_mi355_gpt_oss120bfrom

sangeeta0201:amd_mi355_gpt_oss120b

Aug 5, 2026

Commits on Aug 5, 2026

Fix split-KV chunk deadlock; scale KV chunks with sequence length

sangeeta0201
and
claude
committed
0e946df
Fix intermittent fused-layer deadlock: widen QKV epoch barrier to all workers

sangeeta0201
and
claude
committed
3a931ae
Fix degenerate MoE W13->W2 barrier and untruncate device timing average

sangeeta0201
and
claude
committed
3a2440d
Validate multi-layer pointer tables on the host before upload

sangeeta0201
and
claude
committed
d99e8c3
Add MPK_NIL_TRIPWIRE: survivable breadcrumbs for the nil-address fault

sangeeta0201
and
claude
committed
27ac114
Tripwire: sub-phase breadcrumbs for the nil-address fault

sangeeta0201
and
claude
committed
1231ff6
Fence TopK routing data before releasing MoE workers

sangeeta0201
and
claude
committed
e0da2ef
Refresh all TaskDesc pointer slots in the multi-layer loop

sangeeta0201
and
claude
committed
0681f4c
MI350: fence split-KV partials before the attention chunk barrier

sangeeta0201
and
claude
committed
49bd7ac
Fix MoE W13->W2 barrier deadlock: false sharing between the L2 atomic

sangeeta0201
and
claude
committed
c600e70
Fix MoE deadlock: missing early-clobber on multi-load inline asm

sangeeta0201
and
claude
committed
6a571d9
Add regression test for gfx950 scaled-MFMA pipeline hazards

sangeeta0201
and
claude
committed
a25a2c6
Fix two silent gfx950 scaled-MFMA hazards in the four pipelined loops

sangeeta0201
and
claude
committed
6f3c42c
Perplexity for the MPK GPT-OSS path: logits sink + PPL_MODE

sangeeta0201
and
claude
committed
5c21c37
Perplexity sweep 512 -> 32768, and chunk the PPL diagnostics

sangeeta0201
and
claude
committed
8cd5584
PPL_MXFP4_MATCH: score the Torch reference at MPK's weight precision

sangeeta0201
and
claude
committed
6ab2a09
Fix --max-layers not truncating the reference; add raw-logit dump

sangeeta0201
and
claude
committed
63b0278
Remove the dead activation-format switches; FP8 is unconditional

sangeeta0201
and
claude
committed
902d23a
Recover most of the f1fa720 latency: narrow the Phase 2 wait, not the arrival

sangeeta0201
and
claude
committed
aef3e89
Derive fused-layer barrier values from a deterministic layer index

sangeeta0201
and
claude
committed
4f09181
Gate the worker-state breadcrumbs behind MPK_WORKER_STATE

sangeeta0201
and
claude
committed
2dc9a61
Scope the attention-chunk fences to the XCD they actually span

sangeeta0201
and
claude
committed
671e60c
Gate the g_ws_dev publication too

sangeeta0201
and
claude
committed
386a3ab
Drop the always-on startup/summary diagnostics; ignore build artifacts

sangeeta0201
and
claude
committed
4ed604e
Apply clang-format 15 to the files this branch touches

sangeeta0201
and
claude
committed
ed61f0a

Navigation Menu

Uh oh!

MI355 GPT-OSS-120B: fix six deadlocks and two silent MFMA hazards; recover the latency cost; add perplexity harness - #6

MI355 GPT-OSS-120B: fix six deadlocks and two silent MFMA hazards; recover the latency cost; add perplexity harness#6
sangeeta0201 merged 25 commits into
ROCm:amd_mi355_gpt_oss120bfrom
sangeeta0201:amd_mi355_gpt_oss120b

Commits on Aug 5, 2026

Fix split-KV chunk deadlock; scale KV chunks with sequence length

Fix intermittent fused-layer deadlock: widen QKV epoch barrier to all workers

Fix degenerate MoE W13->W2 barrier and untruncate device timing average

Validate multi-layer pointer tables on the host before upload

Add MPK_NIL_TRIPWIRE: survivable breadcrumbs for the nil-address fault

Tripwire: sub-phase breadcrumbs for the nil-address fault

Fence TopK routing data before releasing MoE workers

Refresh all TaskDesc pointer slots in the multi-layer loop

MI350: fence split-KV partials before the attention chunk barrier

Fix MoE W13->W2 barrier deadlock: false sharing between the L2 atomic

Fix MoE deadlock: missing early-clobber on multi-load inline asm

Add regression test for gfx950 scaled-MFMA pipeline hazards

Fix two silent gfx950 scaled-MFMA hazards in the four pipelined loops

Perplexity for the MPK GPT-OSS path: logits sink + PPL_MODE

Perplexity sweep 512 -> 32768, and chunk the PPL diagnostics

PPL_MXFP4_MATCH: score the Torch reference at MPK's weight precision

Fix --max-layers not truncating the reference; add raw-logit dump

Remove the dead activation-format switches; FP8 is unconditional

Recover most of the f1fa720 latency: narrow the Phase 2 wait, not the arrival

Derive fused-layer barrier values from a deterministic layer index

Gate the worker-state breadcrumbs behind MPK_WORKER_STATE

Scope the attention-chunk fences to the XCD they actually span

Gate the g_ws_dev publication too

Drop the always-on startup/summary diagnostics; ignore build artifacts

Apply clang-format 15 to the files this branch touches