Repository navigation
cleanup(rdna): delete env-gated debug probes and orphan modules - #45
Closed
BlivionIaG wants to merge 1 commit into
Closed
BlivionIaG wants to merge 1 commit into
BlivionIaG wants to merge 1 commit into
Conversation
Remove owner-only forensics that write /tmp logs, print, torch.save, or host-scan for NaNs. Env-unset paths keep the same kernel selectors and copy-mismatch fallback. Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Author
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Purpose
Subtraction-only cleanup of leftover owner forensics on
rdna_extras. Env-unset behavior is unchanged: no new flags, helpers, state, or replacement logging, andvllm/envs.pyis untouched.Not a duplicate. Open PRs on
opengfx1030/vllm-rdna(including #41 W4A8 trim, #40 hetero MoE, #37 expert residency, #33 FA decode, #31 W4A16 explore, #25 all-reduce) do not delete these probes. Hands-off files owned by those branches were not edited:vllm/envs.py,vllm/v1/worker/gpu/model_runner.py,vllm/v1/attention/backends/rdna_attn.py,vllm/models/qwen4_exp/amd/hyperconnection.py,vllm/model_executor/layers/quantization/exl3.py,vllm/distributed/device_communicators/rdna_all_reduce.py,csrc/,docs/,bench_results/.AI assistance was used (Cursor cloud agent). No model-output, accuracy, or serving change: the deleted branches run only when an env var is set, and unset is the default.
Kept / dropped
vllm/utils/rocm_poolwatch.py,vllm/utils/rocm_graph_keepalive.pyrgfound no Python imports. Docs mention the keepalive filename only. C++rdna2_immortal_zerosincsrc/left in place.qwen_gdn_linear_attn.pyVLLM_LOG_GDN_PTRS,VLLM_GDN_OUT_DEBUG,VLLM_GDN_NAN_DEBUG,VLLM_GDN_STATE_DEBUG/VLLM_GDN_SAVE(torch.saveto/tmp/gdn_l6.pt),VLLM_GDN_DBG. KeptVLLM_GDN_HIP_PREFILLandVLLM_GDN_DECODE_RDNA2.VLLM_MAMBA_COPY_DEBUGand thedebuglocal inmamba_utils.py_DEBUG/_qsa_dump_bad_rowsqsa_compress_groups_rdna2takes no debug argument, so the call is unchanged. Not dropped.csrc/rocm/qsa_rdna2.custillgetenvsVLLM_QSA_RDNA2_DEBUGfor a printf; that file is hands-off.VLLM_PIECE_IN_DEBUG,VLLM_CG_NAN_INPUT_CHECK,VLLM_CG_REPLAY_LOG. KeptVLLM_CG_SKIP_REPLAYand capture-time_nan_to_num_tree.backends.pyVLLM_CG_REPLAY_LOGandVLLM_PIECE_DUMP/VLLM_PIECE_TPVLLM_W4A16_PTR_DEBUGin_tp_all_reduceimport os. Restored the pre-probereturn get_tp_group().all_reduce(input_).VLLM_W4A16_PTR_DEBUGlogs, the unreachablekernel_name == "awq_prefill"arm (selector returns onlyw4a8_prefill/prefill/exllama/rdna2_decode), the else-armops.awq_gemm_rdna2_prefillcall, and unused_awq_prefill_available. Kept the gptq/exllama fallback.awq_gemm_rdna2_prefillis not incsrc/.VLLM_BT_DEBUGinblock_table.pyandworker/utils.pymodel_runner.pyandrdna_attn.py.VLLM_DEBUG_PREFIX_HITVLLM_NO_PARTIAL_HASHand the existingVLLM_TRACE_PREFIX_CACHElogs (not in the delete list; they are a separate trace switch).VLLM_MOE_NAN_DEBUG,VLLM_LAYER_NAN_DEBUG,VLLM_PLE_NAN_DEBUGNothing in this list was dropped as a live path. Remaining references are only in hands-off trees.
Proof
rgof the deleted knobs/modules/functions. Hits left outsidedocs/andbench_results/:VLLM_BT_DEBUG:vllm/v1/worker/gpu/model_runner.py,vllm/v1/attention/backends/rdna_attn.py(hands off).VLLM_QSA_RDNA2_DEBUG:csrc/rocm/qsa_rdna2.cuprintf (hands off).awq_gemm_rdna2_prefill: string inside the kernel-name regex inbenchmarks/kernels/w4a16_compile_dispatch/probe.py(not a call; not edited).VLLM_LOG_GDN_PTRS,_awq_prefill_available, androcm_graph_keepalive.pyunderdocs/only.Zero hits in
vllm/**/*.pyfor:VLLM_LOG_GDN_PTRS,VLLM_GDN_OUT_DEBUG,VLLM_GDN_NAN_DEBUG,VLLM_GDN_STATE_DEBUG,VLLM_GDN_SAVE,VLLM_GDN_DBG,VLLM_MAMBA_COPY_DEBUG,VLLM_QSA_RDNA2_DEBUG,_qsa_dump_bad_rows,VLLM_PIECE_IN_DEBUG,VLLM_CG_NAN_INPUT_CHECK,VLLM_CG_REPLAY_LOG,VLLM_PIECE_DUMP,VLLM_PIECE_TP,VLLM_W4A16_PTR_DEBUG,_awq_prefill_available,VLLM_DEBUG_PREFIX_HIT,VLLM_MOE_NAN_DEBUG,VLLM_LAYER_NAN_DEBUG,VLLM_PLE_NAN_DEBUG,rocm_poolwatch,rocm_graph_keepalive.Ruff 0.14.0 (repo pre-commit rev) on the touched files. No new diagnostics versus
rdna_extrasHEAD. Remainingruff checkhits are pre-existing on untouched lines (HEAD had more: the deleted probes were the extra E501/UP031/SIM105/B018):backends.pyE501 line 714 (same line on HEAD)cuda_graph.pySIM102 in kept_nan_to_num_treerdna2_w4a16.pyB018 on the kepttorch.ops._rocm_C.w4a8_gemm_rdna2probeqwen_gdn_linear_attn.pyI001 plus two pre-existing E501sqsa_rdna2.pyI001 (pre-existing import gap)ruff format --check: 7 touched files already formatted (communication_op.py,ple_offload_layer.py,model.py,kv_cache_coordinator.py,block_table.py,mamba_utils.py,utils.py). The other 5 also failformat --checkon HEAD; they were not reformatted, so surviving lines stay as they were.Diff: 14 files, 4 insertions, 985 deletions. The four insertions restore the pre-probe return / gptq fallback (
elifbecameifafter the AWQ arm was removed).Test Plan
python -m py_compileon every touched fileruff checkandruff format --checkwith ruff 0.14.0 andpyproject.tomlrgfor each deleted knobtests/compile/test_cudagraph_replay_inputs.pytests/kernels/quantization/test_rdna2_w4a16_selection.pytests/v1/core/prefix_cache/test_partial_prefix_cache_hits.pyTest Result
py_compile: pass.
Ruff: see Proof. No new findings versus
rdna_extras.Pytest (CPU torch 2.13.0+cpu,
PYTHONPATH=/workspace, no ROCm extension build):Imports succeeded without a compiled ROCm extension.
import_kernelsonly warns whenvllm._Cis missing.CUDA/HIP required,ROCm only), including the threeVLLM_CG_SKIP_REPLAYcapture/replay cases.cleanup_dist_env_and_memorycallstorch.accelerator.empty_cache()because platform detection does not see this tree as CPU (vLLM package was not foundwhen importing fromPYTHONPATH). The test bodies of the W4A16 selector cases (test_rdna2_w4a16_inner_dispatch, registry order, backend map, compile dispatch) ran and were not assertion failures.test_should_copy_and_wrap_eager_piecewise_graphmodulesandtest_mrope_get_positions_contiguous_per_capture_sizeraiseFailed to infer device type/pin_memory=Truewith no accelerator.test_mamba_align_split_*cases raiseAttributeError: 'types.SimpleNamespace' object has no attribute '_effective_lpt'invllm/v1/core/sched/scheduler.py, which this PR does not touch.Model eval: not applicable. No kernel math, sampler, or default serving path changed.