Repository navigation
Sync rdna_extras with upstream opengfx1030/vllm-rdna 3e274335, and pciescope knows the v620-Experimentation box - #5
Merged
Conversation
LLGemm1_kernel launches K/8 threads, so for K <= 8192 a wave32 block has up to 32 waves, but the second-stage reduction reads only red_smem[..][0..15]. For K > 4096 on gfx10/gfx11/gfx12 the partials of waves 16..31 were dropped and the result silently lost part of the dot product. Fold them into the first 16 lanes before the shuffle. wave64 (gfx9) has at most 16 waves in this range and is unchanged (compile-time WARP_SIZE check). Ported from FaisalBiyari/vLLM-ALU-RDNA2@a8f8c02 (csrc/rocm/skinny_gemms.cu).
gfx10 (RDNA2) is neither __HIP__GFX9__ nor __HIP__GFX1X__, so the wvSplitK kernels are the assert(false) stub there and any call faults the queue (HSA_STATUS_ERROR_EXCEPTION). The Python dispatcher already routes gfx10 decode to gemv_f16_rdna2; make direct callers fail with a clear error. Tests: skip the wvSplitK tests on gfx10, and add an LLMM1 case with K=4104 (17 waves on wave32) that exercises the second-stage partials 16..31. Adapted from the gfx1030 wvSplitK dispatcher guard in FaisalBiyari/vLLM-ALU-RDNA2@a8f8c02 (csrc/rocm/skinny_gemms.cu); this tree does not build the wave32 path for gfx10, so the guard is unconditional. Co-authored-by: FaisalBiyari <11506542+FaisalBiyari@users.noreply.github.com>
w4a8_gemm_rdna2 launches config 8 (a8_lds_k32_ag) only. Stop instantiating the other eight explore configs, and drop the uninstantiated probe kernel and the unused instantiate macro. Co-authored-by: Cursor Grok 4.7 <cursoragent@cursor.com> Signed-off-by: BlivionIaG <kev29lt@gmail.com>
…opengfx1030#106) - tools: delete probe_force_split.py (its env var has no reader after opengfx1030#104) - docs: drop removed debug knobs from v0.31.0-port-notes (opengfx1030#103) - docs: tunableop README points at the single tools/rdna2_028 helper (opengfx1030#101) - test: GDN KKT hang tests opt in via VLLM_RUN_GDN_KKT_HANG_TEST=1 (opengfx1030#106)
…lacement solver (CPU-tested, no GPU run) [skip ci] VLLM_EXPERT_STATS_DIR turns on a cudagraph-safe per-expert routing counter at the router output (one is-not-None test per MoE call and per step when off). tools/rdna/hetero_moe converts the stats, solves a hub/V620 placement for the 2 x (1 W7800 + 4 V620) box, validates plans and simulates them. Sample plan from synthetic stats; doc docs/rdna2/expert-stats-2026-11-07.md. Signed-off-by: BlivionIaG <kev29lt@gmail.com>
…t model, box budget defaults [skip ci] Adds island_mode partition (each expert in one island, global owner indices, per-island solve), the cost model constants and fixed terms from the design spec, a spill-aware simulated step time, --mtp-expert-bytes with MTP policy hub by default, W7800 48 GB and V620 26 GiB budget defaults, and a partition sample plan. Signed-off-by: BlivionIaG <kev29lt@gmail.com>
Signed-off-by: BlivionIaG <kev29lt@gmail.com>
…P rank (rank agreement check) [skip ci] Signed-off-by: BlivionIaG <kev29lt@gmail.com>
Signed-off-by: BlivionIaG <kev29lt@gmail.com>
…llector on V620 slot B [skip ci] Signed-off-by: BlivionIaG <kev29lt@gmail.com>
…skip ci] Signed-off-by: BlivionIaG <kev29lt@gmail.com>
…ip ci] Signed-off-by: BlivionIaG <kev29lt@gmail.com>
…ecks) and tests [skip ci] Signed-off-by: BlivionIaG <kev29lt@gmail.com>
… scaffold [skip ci] Signed-off-by: BlivionIaG <kev29lt@gmail.com>
…sfs_link/aer/amdgpu/kfd [skip ci] Signed-off-by: BlivionIaG <kev29lt@gmail.com>
… Perfetto export, report, fixture capture, core tests [skip ci] Signed-off-by: BlivionIaG <kev29lt@gmail.com>
…re), replay, Perfetto export, goldens and real-box session fixture [skip ci] Signed-off-by: BlivionIaG <kev29lt@gmail.com>
…rt notice [skip ci] Signed-off-by: BlivionIaG <kev29lt@gmail.com>
…bs, plx status masks), switch counter reasons [skip ci] Signed-off-by: BlivionIaG <kev29lt@gmail.com>
…n blockers, repeats median) [skip ci] Signed-off-by: BlivionIaG <kev29lt@gmail.com>
Signed-off-by: BlivionIaG <kev29lt@gmail.com>
…helper, port map, CLI, tests, doc section [skip ci] Signed-off-by: BlivionIaG <kev29lt@gmail.com>
…n hip order; session attach [skip ci] Signed-off-by: BlivionIaG <kev29lt@gmail.com>
…skip ci] Signed-off-by: BlivionIaG <kev29lt@gmail.com>
…captured trees make it unavailable instead of touching live hardware [skip ci] Signed-off-by: BlivionIaG <kev29lt@gmail.com>
Signed-off-by: BlivionIaG <kev29lt@gmail.com>
…t and GPU name; ports/status columns, hip tag, doc refresh [skip ci] Signed-off-by: BlivionIaG <kev29lt@gmail.com>
…readable); capture stays sysfs-only; tests order- and timing-independent [skip ci] Signed-off-by: BlivionIaG <kev29lt@gmail.com>
…BAR0 reads cause no errors or runtime-PM wake-ups [skip ci] Signed-off-by: BlivionIaG <kev29lt@gmail.com>
…the live two-lane proof [skip ci] Fills the measured sections of concurrency-probe-noise-2026-11-12.md from the five boots on HIP 2-5 (tree 1136bb9 unless named): fixed probe 0 of 20 FAIL runs on flashnext-mtp2, 0 of 20 on flashnext-mtp0, 0 of 20 with VLLM_RDNA2_MOE_DETERMINISTIC=1 (solo runs bit-identical in 20 of 20, strict mode). The known-bad two-lane head twolane-B-adffce5765 fails 5 of 5 runs (3 or 4 failing checks, both margins far above 1 nat) while its base twolane-A-3896928d18 passes 5 of 5. Signed-off-by: BlivionIaG <kev29lt@gmail.com>
… the n_dec>=2 repro, stale comment replaced (opengfx1030#165) [skip ci]
…rrectly on non-deterministic servers (opengfx1030#167) [skip ci]
…e wvSplitK on gfx10 (opengfx1030#54) [skip ci] LLMM1 timing on V620 (HIP 0), base 1136bb9 vs PR head 8efea0e, two ABBA blocks, 14 shapes (fp16/bf16, M 4-8, K 1024-8192): no shape where both head runs are worse than both base runs; medians -0.5 to -2.0 %. gfx1030 serving does not reach LLMM1 or wvSplitK (gemv_f16_rdna2 / rdna_gemm_smallm / rocBLAS); gfx9 code is unchanged (per-function disassembly diff: only LLGemm1_kernel differs, on gfx1030/1100/1200).
…8 sdot4 config (opengfx1030#89) The launched kernels are byte-identical on gfx1030 (4 of 4 functions exact after the trim; the 24 removed instantiations and the probe kernels had no caller; config 8 was the only selectable id). w4a8_gemm_rdna2 timing on V620 (HIP 0), base 1136bb9 vs PR head e7ee449, two ABBA blocks, six prefill shapes: no measurable difference (medians -0.4 to +0.5 %).
…_LEAPQUANT) Port of LeapQuant (arXiv 2609.38166, vllm#64725) to the RDNA2 GDN path: the FP32 [V, K] state per value head becomes a 28,768-byte blob (INT8 residual + rank-4 FP16 compensator + FP16 per-token records, window 16). MTP verify commits accepted records instead of per-token state copies; align-mode prefix-cache copies commit bias + 1 records in the HIP copy kernel. Default off; gfx10x only. decode_kl.py: decode-path top-k KL between served configurations. Signed-off-by: BlivionIaG <kev29lt@gmail.com>
mono_gdn_lq_rdna2 is the fused mono GDN launch (conv + delta rule + gated norm) with the LeapQuant blob load/append in place of the FP32 state load/store; gdn_lq_normalize_rdna2 commits the accepted records first. The flag no longer refuses --monokernel, and the FP32 Triton decode warmup is skipped under the flag (it never runs there). Shared blob layout moves to gdn_leapquant_layout.cuh. decode_kl.py accepts a /v1/completions URL. Signed-off-by: BlivionIaG <kev29lt@gmail.com>
All LeapQuant GDN layers normalize with the same state indices and accepted counts each step, so the first layer of the model now normalizes every layer's paged blobs in one launch (gdn_lq_normalize_rdna2 gains an optional layer_offsets [layers] table, grid.y = layer; the group is rebuilt on bind_kv_cache, never under capture) and the other layers skip. Decode arenas stay per layer. Replaces 36 launches (~3 us each) and lets the folds of all layers run concurrently instead of one layer after the other. The record replay loops unroll by 2 (same arithmetic order). Per step, 36 layers, one V620, HIP graph (FP32 MonoKernel -> LeapQuant before -> after): decode n=1 343 -> 574 -> 361 us, verify n=1 ~500 -> 867 -> 582 us, decode n=8 1556 -> 1345 -> 1134 us, verify n=8 3323 -> 2217 -> 1851 us. test_grouped_normalize_matches_per_layer: same bytes as per-layer normalize, folds and NULL slots included. Signed-off-by: BlivionIaG <kev29lt@gmail.com>
…yer group Signed-off-by: BlivionIaG <kev29lt@gmail.com>
…he group Hybrid models spread their GDN layers over several KV-cache groups, each with its own block table, so a request sits at different state slots per group. The grouped normalize (8ba674a) let the first GDN layer of the model normalize every GDN layer at its own group's slots: the other groups' layers committed records at foreign slots and never at their own. On flashnext-mtp2 (6 KV-cache groups) decode fell into repetition loops (10-21 of 32 outputs at c=8 vs 2-7 for the base) while prefill KL and PPL stayed inside the floor. Groups are now built per KV-cache group in bind_kv_cache_to_layers, next to share_replayssm_ring_trackers (same namespace rule); without the group map every layer keeps its own launch. Signed-off-by: BlivionIaG <kev29lt@gmail.com>
Signed-off-by: BlivionIaG <kev29lt@gmail.com>
…ip ci] Signed-off-by: BlivionIaG <kev29lt@gmail.com>
… ci] Signed-off-by: BlivionIaG <kev29lt@gmail.com>
…ffs [skip ci] Signed-off-by: BlivionIaG <kev29lt@gmail.com>
…k pause [skip ci] Signed-off-by: BlivionIaG <kev29lt@gmail.com>
Signed-off-by: BlivionIaG <kev29lt@gmail.com>
Signed-off-by: BlivionIaG <kev29lt@gmail.com>
Signed-off-by: BlivionIaG <kev29lt@gmail.com>
Signed-off-by: BlivionIaG <kev29lt@gmail.com>
…indow 32, MTP2, F&P) Signed-off-by: BlivionIaG <kev29lt@gmail.com>
…rs and docs [skip ci] Signed-off-by: BlivionIaG <kev29lt@gmail.com>
…=1 and c=8 [skip ci] Signed-off-by: BlivionIaG <kev29lt@gmail.com>
…st [skip ci] Signed-off-by: BlivionIaG <kev29lt@gmail.com>
…eduler and flashnext-leapquant-mtp2 recipe
Sync the fork with upstream rdna_extras at 3e27433 (2026-11-14), all of it, on the owner's request (2026-11-15): "Add everything, even the stuff you said doesn't impact our serving path. New models may be added later, so use it all." The fork's own commits on rdna_extras (the launchers' 90 °C thermal abort and its tests, #2) merge without conflicts. The range 750fff8..3e27433 is 119 commits on the same vLLM v0.33 base: - LeapQuant INT8 GDN recurrent state for Flash-Next (opt-in VLLM_RDNA_GDN_LEAPQUANT, lossy), its admission and decoder-first scheduling, and the flashnext-leapquant-mtp2 recipe (512-token blocks, MTP2, MonoKernel) (opengfx1030#168); - the adaptive mixed-step prefill fit recovering after cold steps (recipes with VLLM_RDNA_DECODE_STALL_MS); - LLMM1's wave32 reduction fix and wvSplitK refused on gfx10 instead of faulting the queue (opengfx1030#54); - W4A8 sdot4: only the launched config is built (opengfx1030#89); - opt-in MoE routing statistics (VLLM_EXPERT_STATS_DIR) and the hetero-MoE placement solver (opengfx1030#158); - pciescope, the PCIe fabric monitor: passive sysfs/AER/amdgpu/KFD collectors, the PEX88096 switch reader, the active probe, a TUI and an MCP server (opengfx1030#157); - the concurrency probe's noise triage (opengfx1030#167), the GQA decode validation (opengfx1030#165), cleanup follow-ups (opengfx1030#118), and upstream's own disaggregation and T10 notes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CnjpEQc6W7ePB6ggy5Tkhe
…one PEX880xx) pciescope's lease check and port map take a box's layout from its GPU buses, and a box that matches no documented layout gets NO_LAYOUT: every GPU is refused, as the lease README asks until the table is re-derived. The owner of jmacpolin/v620-Experimentation wants pciescope on that box (2026-11-15), so it gets its layouts: - four V620 behind one Broadcom PEX880xx (1000:c010, the chip pciescope's switch reader knows), one lease key v620x4 for all four, slot A; - the buses follow the CPU population: 64 69 6f 72 with two Xeon Gold 6144 (since 2026-10-11), b1 b6 bc bf with one, from that repository's builds/v620-cards.tsv (rocm-smi inventories; HIP order = bus order). Both tables (probe/hw.py LAYOUTS, topology.DOCUMENTED_LAYOUTS) get them, as their own test requires; a new test covers the key, the lookup by one bus and that a subset is not the box. par1-cs25's layouts are unchanged. The pciescope suite: 968 passed, 1 skipped (967 + 1 before the change). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CnjpEQc6W7ePB6ggy5Tkhe
jmacpolin
marked this pull request as ready for review
November 15, 2026 17:42
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Overview
This syncs
rdna_extraswith upstreamopengfx1030/vllm-rdnardna_extrasat3e274335(2026-11-14): 119 commits on the same vLLM v0.33 base, taken whole. One commit of our own follows, so pciescope recognises the four-V620 box of jmacpolin/v620-Experimentation.The owner asked for all of it (2026-11-15): "Add everything, even the stuff you said doesn't impact our serving path. New models may be added later, so use it all." Then: "I want the pciescope".
Claims
170fdd7a) equals upstream's tree plus this branch's own launcher thermal abort (jmacpolin/vllm-rdna#2). It is the same 10 files, +428/−8, as before the merge.VLLM_RDNA_GDN_LEAPQUANT=1) and lossy, and brings its admission and decoder-first scheduling and theflashnext-leapquant-mtp2recipe (upstream feat(rdna2): add LeapQuant GDN state with 512-token pages opengfx1030/vllm-rdna#168).VLLM_EXPERT_STATS_DIR) and the hetero-MoE placement solver (upstream feat(rdna): opt-in MoE routing statistics (VLLM_EXPERT_STATS_DIR) and hetero placement solver opengfx1030/vllm-rdna#158).976339b3, ours: pciescope's layout tables (probe/hw.pyLAYOUTSandtopology.DOCUMENTED_LAYOUTS) learn this box.1000:c010), with one lease key,v620x4.64 69 6f 72with two Xeon Gold 6144,b1 b6 bc bfwith one. Both come from the v620 repo'sbuilds/v620-cards.tsv.NO_LAYOUTand its lease check refuses every GPU. upstream's par1-cs25 layouts are unchanged.Validation
976339b3(+1 test for the new layouts).3e274335intordna_extrashad no conflicts, and the merged tree differs from upstream's only by our thermal-abort commits.Details
tools/pciescope.sh, which holds thev620x4lease while no engine runs (its one-engine rule).regs.KNOWN_CHIPS. That needs a read on the host first, and is not part of this PR.Pull Request Checklist
I used vLLM's
/pr-checklistskill. (Mandatory for agents, optional for humans).AI assistance was used during the creation of this PR.
Design Fit: Minimizes impact on core components, reuses existing functionality, and justifies added complexity.
Testing and Validation: Validates the change and ensures any added tests are meaningful and reliable, with CI coverage or documented CI resource constraints and validation performed outside CI.
Code Quality and Style: Keeps code and comments clear and concise, and updates relevant documentation and examples.
Pull Request Contents: Includes a brief summary and relevant links, supports claims with evidence, explains root causes and implementation trade-offs, and follows the contributing guide.
Generated by Claude Code