Repository navigation
feat(rdna3): gfx1100 Flash-Next W4A16 port (fat gfx1030;gfx1100 _rocm_C) - #160
Draft
BlivionIaG wants to merge 12 commits into
Draft
BlivionIaG wants to merge 12 commits into
BlivionIaG wants to merge 12 commits into
Conversation
…lies opt-in Make the Qwen4Exp AMD Flash-Next path (MTP2, QSA, GDN, PLE, HC, compiled FULL_AND_PIECEWISE, fp16) buildable and loadable on gfx1100, add the tools to validate it on a W7800 pair, and keep gfx1030 untouched. - platforms: VLLM_RDNA3_HIP_OPS families (hc, gdn, qsa, se, dense, ar, moeu, v1, or all) with probe-op checks and on_rdna3_hip(). Every widened gate is "old term or on_rdna3_hip(family)", so gfx10x keeps its old value. - build: rdna_fused_glue is built for gfx1100 and its registrations sit under VLLM_ROCM_RDNA_GLUE (fixes 4 undefined symbols of a gfx1100-only _rocm_C). - rocm: amdsmi arch query is mixed-arch safe (falls back to the torch device). - moe: Q32 deterministic mode (VLLM_RDNA3_MOE_DETERMINISTIC), block-M override, ignore_invalid_experts, expert_map checks, moe_fixedpoint_to_half_rdna3. - sampler: radix top-k decode default on gfx10 and gfx11, VLLM_TOPK_DECODE_ALGO. - rdna_ar: opt-in on gfx1100 (ar family), group-wide arch agreement, ar_probe.py. - tests: gfx1100 gates for 25 per-op GPU tests, CPU dispatch, MoE and AR tests. - tools: build_rocm_c.sh, code-object fingerprint oracle, load check, perop_expectations.py, cpu_pytest.sh, flashnext-w2-mtp2 recipe. - docs: docs/rdna2/rdna3-flashnext-port-2026-11-07.md. Nothing was run on a GPU; the GPU test plan is in the document. Signed-off-by: BlivionIaG <kev29lt@gmail.com>
The rocm_sdk wheel venvs that carry the gfx1100 device code (par1-cs25: venv-7.14.0) ship no amdsmi module. vllm.platforms.rocm logged "Failed to import from amdsmi" and then left amdsmi_init and the other names undefined, so the first RocmPlatform.get_device_name() raised NameError (found by G4 of the W7800 port: the Triton fused_experts reference of test_rdna2_unquantized_moe.py). get_device_name() is on the engine startup path (v1/worker/startup_plan.py, Triton MoE/SSM config lookups), so a serve from that venv would die at startup. Define stand-ins for AmdSmiException, amdsmi_init, amdsmi_shut_down and amdsmi_get_processor_handles (no GPU enumerated) in the ImportError branch. Every caller already handles an amdsmi that enumerates no GPU (torch fallback in get_device_name / get_device_total_memory, IndexError -> not connected in is_fully_connected / is_pix_connected, the arch query falls back to torch). Venvs with amdsmi are unchanged. A subprocess test blocks the amdsmi import and checks the context wrapper, the handle list, both connectivity queries and the name fallback; the probe raises the NameError on the phase-1 tree. Signed-off-by: BlivionIaG <kev29lt@gmail.com>
moe_gptq_gemm_rdna3_wmma (csrc/rocm/moe_q_gemm_rdna3_wmma.cu) runs the experts of a token block on v_wmma_f32_16x16x16 and reduces the whole K in registers (fp32 accumulators, no split-K, no atomics on the GEMM result). It takes the weights and routing tensors of moe_gptq_gemm_rdna3, block_size_m 16, 32 or 64, and either stores each scheduled pair row once (fp16 or bf16 c) or folds the top-k reduction into the Q32 int64 accumulator (fp16 only). The scalar kernel stays the decode path and the fallback. The TU is a stub on gfx1030. rdna3_moe.py sends batches of at least VLLM_RDNA3_MOE_WMMA_MIN_M tokens (256) down GEMM 1 (direct store), the activation, GEMM 2 (Q32, output_topk = top_k) and moe_fixedpoint_to_half_rdna3 when VLLM_RDNA3_MOE_WMMA=1 (default off). That path is bit-deterministic whatever VLLM_RDNA3_MOE_DETERMINISTIC says; fp16 only. VLLM_RDNA3_MOE_WMMA_BLOCK_M picks the tile (0 = from the mean pair rows per expert, a guess to tune on the GPU). ISA of the gfx1100 object (llvm-objdump, nothing run): 9 instantiations, 0 scratch and 0 spills, 32 WMMA, 2 s_barrier and 6 ds_store each; VGPR 190, 148, 212 (fp16 BM16, BM32, BM64) and 233, 170, 232 (bf16), LDS 8,832, 17,664 and 18,944 bytes. Tests, all on the CPU: test_rdna3_moe_wmma_model.py emulates the kernel mapping index for index (31 mutants x 3 tiles caught), 11 new dispatch tests in test_rdna3_moe_modes.py (150 passed with the real wrapper), 2 compile guards. test_rdna3_moe_wmma.py is the GPU test and has not run on a GPU. Signed-off-by: BlivionIaG <kev29lt@gmail.com>
…d per-op tests First GPU pass of the Flash-Next RDNA3 port (one W7800, HIP 0; venv 7.14.0): tools/rdna/rdna3/gpu_g0_g7.sh runs G0-G8 and G16 under the box lease with a thermal and KFD-eviction watch, per-command reruns (--labels) and a dry-run plan that tests/tools/rdna/test_gpu_g0_g7.py checks. New tools: build_fat.sh, gpu_preflight.py, gdn_decode_bench.py (TP1 and TP2 geometries), topk_decode_bench.py (radix against insertion), tensile_probe.py, gdn_delta_h_oracle.py (HIP and Triton against an exact fp64 recurrence, with a dump and replay for another arch), trunc_checkpoint.py and trunc_e2e.py (layer-truncated checkpoint and per-layer numerics compare). What the run found, and the changes that follow: - delta-h outlier (1 of 3.9M v_new elements, TP1 shape): HIP and Triton have the same error against fp64 on every case of a 48-case grid, so the pairwise v_new atol goes from 1e-3 to 2e-3 and a new test pins the error against the oracle. - test_worker_ops_rdna2.py was stale against the kernels (gather and slot signatures, the 2-D write grid); the helpers now match, and apply_write has a torch oracle. The HIP ops matched it at every length. - skinny GEMM tests: the fp8 test is skipped on gfx11 (torch._scaled_mm needs MI300+), LLMM1 at K above 4096 is an xfail on gfx11 (wrong results there, but the dispatcher never reaches it). - the pytest groups run with VLLM_RDNA3_HIP_OPS=all (mrope needs a family on). - decode_ops_bench.py gets --torch (the shared-expert op ignores the family variable, the model-level gate reads it); gemv_crossover_bench.py stops at 32 rows for gemv. - G7 (per-op microseconds against the model) and G8 are in the doc: hc and se are faster than the default at decode widths, gdn only at 1-2 sequences, VLLM_TOPK_DECODE_ALGO=insertion is 1.3-2x faster at 4k-8k columns. Per-op GPU tests are widened to the TP1 and TP2 head counts; the recipe notes the venv and the measured family results; the port doc gets section 7.1 (S1b) and section 16 (the run). Signed-off-by: BlivionIaG <kev29lt@gmail.com>
…unner step tools/rdna/rdna3/gpu_g0_g7.sh - The S1b WMMA kernel tests run in three stages (smoke 34 tests, one shape per stage 15, the whole file 227): a kernel that has never run on a GPU fails on the smallest stage. A failing stage stops the run; do not use --keep-going for the first execution. - New last step G13, outside the default range (--only G13 or --to G13): three engine starts on the four-layer truncated checkpoint (the W2 recipe as it is, every HIP family, every family plus VLLM_RDNA3_MOE_WMMA=1) and two trunc_e2e.py compare runs (base against all, all against wmma). The kernel-selecting variables are read from the recipe file, the step refuses to start with less than 130 GiB of MemAvailable (the pinned 95 GiB PLE table), and every GPU command carries HIP_VISIBLE_DEVICES=0 like the others. - The lease file now records the selected steps instead of "G0-G7". tools/rdna/rdna3/trunc_e2e.py - Two longer built-in prompts (1536 and 5000 tokens) give the WMMA kernel its 32 and 64 row tiles (rows per expert = tokens * 10 / 512); --max-model-len defaults to 8192. docs/rdna2/rdna3-flashnext-port-2026-11-07.md - Section 16.5: the S1b rebuild (127 s incremental, 178 _rocm_C ops) and the gfx1030 code-object oracle against the untouched base (69 objects identical, 1 stub churn, 1 new stub, 0 changed; gfx1100: the scalar MoE TU changed as in phase 1, one new object), the ISA resources re-read from the build, the two new build-table rows, the G13 note and the updated not-done list. Tested: tests/tools/rdna/test_gpu_g0_g7.py (13 passed, incl. the stage order and the G13 plan) and test_trunc_checkpoint.py (26 passed) on the box CPU tree; pre-commit on the touched files. No GPU run of this commit. Signed-off-by: BlivionIaG <kev29lt@gmail.com>
HeteroDesign asked for per-op W7800 figures at the TP1 and TP2 hub shapes (3, 24 and 48 rows). This adds what is needed to take them; no GPU number is in this commit. Shared expert above 32 rows: the two tile GEMMs of the fused op (rdna_se_gate_up_tile, rdna_se_down_tile) checked M <= 32, but the kernel under them already tiles the rows in blocks of 32 (row-tile grid axis, tail rows masked). The host-side check now says 64, no device code changes, so the gfx1030 code objects are untouched. The wave kernels still stop at 32, so above 32 rows rdna_shared_expert runs only when both tile halves can run (fp16 weights, tile and down-tile switches on, the gate_up output tiles within the 128 arrival tickets) and otherwise takes the torch composition. VLLM_RDNA_SE_DECODE_MAX is clamped to 64; its default stays 32, so every route up to 32 rows is as before. Benches (graph-replayed over cold weight banks, one process per route): tools/rdna/rdna3/qsa_decode_bench.py times the QSA decode core (store, compress, paged MQA select with top-k and expansion, sparse attention) at the TP1 and TP2 head counts; tools/rdna/rdna3/moe_decode_bench.py times the router and Rdna3WNA16Experts.apply at M = 1 to 48 in the default and Q32 modes, with a hub variant (192 of 512 experts resident) and a block-size sweep. Runner: gpu_g0_g7.sh gains G7 cells for the 48-row shared expert, the QSA and MoE benches and PyTorch TunableOp tuning of the lm_head GEMM at both vocabulary shards (tune, then read the table back). A cell may now carry several NAME=value words as its extra environment. Tests: twelve CPU dispatch cases for the shared-expert row cap and tile gates, GPU cases for two row tiles, 64 rows, the ticket refusal and graph replay at 48 rows, and runner plan tests for the new cells. Signed-off-by: BlivionIaG <kev29lt@gmail.com>
The QSA select (paged MQA scoring, top-k of 512 blocks, expansion) runs the decode top-k, whose auto choice is radix on gfx11 and measured 1.3 to 2 times slower than insertion at 4k to 8k live columns. The new cell times the composite select with VLLM_TOPK_DECODE_ALGO=insertion next to the default-algorithm cells, so the open decision on that variable has the composite number and not only the top-k one. Signed-off-by: BlivionIaG <kev29lt@gmail.com>
moe_decode_bench.py gains a wmma mode (VLLM_RDNA3_MOE_WMMA=1, the S1b kernel from 256 tokens up) and a TFLOPS column over the resident pairs, and the runner gets three G7 cells that run it at 256 to 2048 tokens against the scalar kernel: all experts resident, 192 of 512 resident through the expert map, and a sweep of the WMMA tile height. This is the timing the S1b kernel was still missing: its tests only check results. Signed-off-by: BlivionIaG <kev29lt@gmail.com>
The hetero stage-1 hub holds 130 expert slots per layer. A bank of 130 experts routed over 264 experts touches 14, 79 and 110 resident experts at 3, 24 and 48 rows, against the 16.5, 81 and 109 that the T10 routing statistics measured for that hub, so the new G7 cell prints microseconds per distinct expert at the residency the hub will actually see. Signed-off-by: BlivionIaG <kev29lt@gmail.com>
…inct counts The hetero model replaces its attention-core term with a measurement at the 1k contexts of the TP4 anchors, and carries its MoE cost per distinct expert with the hub keeping K distinct experts per width. Both benches now answer that directly. qsa_decode_bench.py: default contexts 1k, 2k, 8k and 32k; every line states the KV length, the live compressed columns the scoring sees and the page-table columns; the sparse attention reads min(2048, ctx) tokens per row with the -1 sentinel after them, as the indexer expansion leaves the row; the runner cells use a page table of 262144 tokens (a captured decode graph at the recipe's maximum model length) and one cell keeps the table as wide as the live context. moe_decode_bench.py: --distinct forces exactly K distinct experts live per --m entry (consecutive entries of one random order, -1 mask when K is below the top-k), with the expert map the hub contract carries. New G7 cells run it at the H1 and H2 hub tables of the design. Signed-off-by: BlivionIaG <kev29lt@gmail.com>
The mixed-arch arch query of 30b6a47 asked amdsmi for the asic info of every handle. amdsmi lists every GPU of the machine, ignores HIP_VISIBLE_DEVICES and ROCR_VISIBLE_DEVICES and powers up idle cards, so a process pinned to one GPU woke the others (seen on the hetero box at 16:50 with pinned workers of a venv that ships amdsmi; not isolated from the platform plugin's own amdsmi probe, which this commit does not touch). _query_gcn_arch_from_amdsmi is the base function again (first handle, one query). _get_gcn_arch now follows two rules: - no HIP/CUDA/ROCR visible-device list set (empty counts as unset): that query, as before, no HIP context; - a list set: no amdsmi at all. rocm_visible.visible_gcn_arch reads gfx_target_version from /sys/class/kfd/kfd/topology/nodes/*/properties (static files: no GPU access, no HIP context). Every GPU of the machine sharing one arch is the answer; on a mixed machine it is the arch of the first visible GPU in HIP's device order (KFD node order, the ROCR list first, then the HIP list). Torch stays the last resort when the files cannot say (no KFD, a UUID list on a mixed machine, an index outside the GPUs). It creates a HIP context, which is why it is not the rule: a context in the parent process also forces vLLM from fork to spawn (see get_device_total_memory). The arch string of a single-arch box is unchanged. The node order matches the HIP-to-bus map of the box after the case move (nodes 1 to 6 = buses d1 d4 d9 dc df e2 = HIP 0 to 5; read from sysfs) and of the box before it. Checked on the box with HIP_VISIBLE_DEVICES=99 (no GPU reachable): the real vllm.platforms.rocm import gives _GCN_ARCH gfx1030 with torch.cuda not initialised. Tests: fake topologies (numeric node order, junk nodes, no KFD), ROCR then HIP lists, UUID and out-of-range lists, the pinned process calling neither amdsmi nor torch (each of the three variables), torch as the last resort, the unpinned process calling amdsmi once. cpu_pytest.sh puts cpu_stubs/rocminfo first on PATH: aiter (imported through flash_attn) runs rocminfo at import, which enumerates every GPU and, on the V620-only box, reports a gfx1030 that aiter does not list (KeyError; 35 dispatch tests failed). The stand-in reports one gfx1100 agent and asks no GPU. CPU run on the box: 240 passed (test_rdna3_dispatch.py, test_rocm_amdsmi_index.py, test_rocm_capability_alias_config.py, test_gpu_g0_g7.py, test_trunc_checkpoint.py). Signed-off-by: BlivionIaG <kev29lt@gmail.com>
The first W7800 window timed one batch of 20 graph replays straight after the capture. The MoE block-size sweep then showed identical configurations apart: block size 0 (1 up to 4 tokens, 4 above) against the same size given explicitly was 6, 19 and 12% slower for the explicit one at 3, 8 and 24 tokens, so those rows were good to about 20% only. graph_us in qsa_decode_bench.py and moe_decode_bench.py now replays for 0.3 s first (the clocks ramp), then times three batches of 20 replays and returns the best batch with the spread of the three; both benches print a spread column. The block-size sweep and every other cell are to be re-timed with it. The doc says so (section 16.7). Signed-off-by: BlivionIaG <kev29lt@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Draft. RDNA3 (W7800) port: WMMA MoE prefill, dispatch gates, arch from KFD sysfs in pinned processes (no amdsmi/HIP context). gfx1030 code objects byte-identical in the fat build; 240 box CPU tests pass.
Remaining: S1b WMMA tests and G0-G8 cells on a W7800. Blocked: no W7800 in the box since the 2026-11-08 case move. Merges cleanly into
rdna_extrastoday.Doc:
docs/rdna2/rdna3-flashnext-port-2026-11-07.md.