Skip to content

cleanup(rocm): drop W4A16 prefill dispatcher diary and experiment env hooks - #48

Closed
BlivionIaG wants to merge 1 commit into
rdna_extra/v0.28.0from
cursor/drop-prefill-dispatcher-hooks-ac45
Closed

BlivionIaG wants to merge 1 commit into
rdna_extra/v0.28.0from
cursor/drop-prefill-dispatcher-hooks-ac45

Conversation

@BlivionIaG

Copy link
Copy Markdown

Purpose

Subtraction-only cleanup of fork-owned W4A16 prefill dispatch in csrc/rocm/q_gemm_rdna2_prefill.cu. Two stacked comments above select_config contradicted each other, and three VLLM_RDNA2_PREFILL_* getenv hooks existed only for finished experiments. Default dispatch is unchanged when those variables are unset.

No other file is modified. This does not touch the QSA kernel, W4A8 sources, rdna_allreduce.cu, ops.h, torch_bindings.cpp, or fa_rdna2.cu.

AI assistance was used (Cursor Grok 4.7). A human should review every changed line before marking this ready.

Why this is not a duplicate

Open PRs on opengfx1030/vllm-rdna were listed before this branch was opened: #47, #44, #42, #41, #40, #39, #37, #35, #33, #31, #25, #23, #18, #10, #2. None of them edit csrc/rocm/q_gemm_rdna2_prefill.cu. #41 is the W4A8 sweep, #42 is tools, #44 is Python debug probes, #33 is fa_rdna2.cu, #18 is GPTQ BLOCK_KN for gfx1030.

What was deleted

  • Both essays above select_config. The first said this function never sees an out-of-envelope M. The second narrated the 2026-09-10 "53% of peak" profile and the size_m > 256 branch. The branch is still the documentation; it was not rewritten into a new comment.
  • VLLM_RDNA2_PREFILL_DEBUG printf and its debug_split cache at the end of compute_split_k.
  • VLLM_RDNA2_PREFILL_FORCE_CONFIG early return inside select_config.
  • maybe_force_split_k and VLLM_RDNA2_PREFILL_FORCE_SPLIT_K. launch_dispatch now passes compute_split_k<Config*> straight through.
  • #include <cstdio> and #include <cstdlib>, which were only used by those hooks.

The four-line tile note already sitting on if (size_m > 256) was left as it was. No new comment was added.

Net: 58 deletions, 3 insertions (55 lines removed). The three insertions are the same compute_split_k calls with the force wrapper removed.

Default control flow (unchanged when the env vars are unset)

compute_split_k: both the power-of-two path and the factor-enumeration path still assign result and return it. Previously a process-lifetime debug_split flag printed that value only when VLLM_RDNA2_PREFILL_DEBUG was set. Unset, the print was skipped and the return value was already result. The split search above the return is untouched.

select_config: previously a cached VLLM_RDNA2_PREFILL_FORCE_CONFIG (atoi, default -1) returned early when >= 0. Unset, that return never fired. The remaining branches are the same, in the same order:

  • size_m > 256: size_n >= 4096 returns ConfigA, otherwise ConfigC
  • size_m < 4 && size_n > 4096 returns ConfigA
  • 4 <= size_m <= 8 && size_n >= 2048 && size_k >= 1024 returns ConfigC
  • size_m == 12 && 2560 <= size_n <= 8192 && size_k >= 512 returns ConfigC
  • size_m == 32 && 5120 <= size_n <= 6144 && size_k >= 1536 returns ConfigC
  • otherwise ConfigV1

launch_dispatch: each arm was maybe_force_split_k(compute_split_k<Config>(...)). That helper returned the computed split unless VLLM_RDNA2_PREFILL_FORCE_SPLIT_K was a positive integer. Unset, it returned the computed split. Each arm now passes that computed split to the same launch_for_config call. The switch cases are unchanged.

Setting any of the three variables no longer changes behavior. That is the point of removing the experiment hooks. The unset path is the same.

#42 dependency

rg VLLM_RDNA2_PREFILL_ on this tree finds no readers under vllm/, tests/, benchmarks/, or scripts/. Readers were only this .cu file plus tools:

  • tools/rdna2_028/probe_prefill_split.py (VLLM_RDNA2_PREFILL_DEBUG)
  • tools/rdna2_028/probe_force_split.py (VLLM_RDNA2_PREFILL_FORCE_SPLIT_K)
  • tools/rdna2_028/audit_w4a16.sh (VLLM_RDNA2_PREFILL_DEBUG)
  • tools/rdna2_028/audit_split/audit_split_swap.py does not read the variable. It splices this .cu file and uses static const bool debug_split = as its end anchor.

Draft PR #42 deletes all four of those files (changeType: DELETED). Nothing under tools/, scripts/, or docs/ is edited here. Merge this after #42 so audit_split_swap.py is already gone; if this lands first, that script's anchor no longer exists.

VLLM_RDNA2_PREFILL_FORCE_CONFIG had no tool or recipe reader.

bench_results/2026-09-29_w4a16-regression-audit/SUMMARY.md names VLLM_RDNA2_PREFILL_DEBUG and VLLM_RDNA2_PREFILL_FORCE_SPLIT_K in a past writeup. It does not set or read them. Left untouched.

#47 cherry-pick

On rdna_extra/v0.31.0 (PR #47) this file's blob is f4b5abc32387024e16f42f39f5d6d5630a471ae4, the same blob as rdna_extras before this commit. The port does not already edit the file, so this commit should cherry-pick cleanly.

Test Plan

  • Confirm no non-tool reader of VLLM_RDNA2_PREFILL_ (rg over vllm/, tests/, benchmarks/, scripts/).
  • hipcc --offload-arch=gfx1030 -fsyntax-only for this translation unit, if a ROCm toolchain is present.
  • Otherwise, brace/paren/bracket balance on the .cu file and rg that the removed symbols are gone from it.
  • No GPU benchmark.

Test Result

  • No reader under vllm/, tests/, benchmarks/, or scripts/.
  • hipcc is not on this machine (command -v hipcc empty, no /opt/rocm). No gfx1030 syntax compile was run.
  • A comment/string-stripped scan of csrc/rocm/q_gemm_rdna2_prefill.cu found (), {}, and [] balanced, with no unclosed comment or string.
  • rg for VLLM_RDNA2_PREFILL_, maybe_force_split_k, debug_split, getenv, atoi, and printf in this file: no matches.
  • Model evals: not applicable. The unset dispatch path is the same code; no serving or accuracy change is claimed, and no GPU run was done.

Left in place on purpose

The block comment above the ConfigId enum (around the old select_config(M, N) M<64 / N<=1024 / otherwise rules) does not match the function that remains. It was outside the two audited hunks, so it was not edited.

Open in Web Open in Cursor 

… hooks

The two config-selection essays contradicted each other, and the three
VLLM_RDNA2_PREFILL_* hooks only existed for finished experiments. Default
select_config branches and compute_split_k dispatch are unchanged.

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants