Skip to content

feat(qwen4-prep): Qwen4 config probe, monokernel stage planner, opt-in dense MLP + 4 upstream Qwen4Exp backports - #164

Draft
BlivionIaG wants to merge 8 commits into
rdna_extrasfrom
feat/qwen4-prep
Draft

BlivionIaG wants to merge 8 commits into
rdna_extrasfrom
feat/qwen4-prep

Conversation

@BlivionIaG

Copy link
Copy Markdown

Draft: GPU validation pending. Merges cleanly into rdna_extras (branch is 189 commits behind).

Fork commits:

  • f53fd9e74d opt-in config probe for Qwen4-family checkpoints (vllm/models/qwen4_prep/config_probe.py)
  • 142c93cae4 config-to-stage-graph planner for the Qwen4 monokernel (qwen4_prep/mono_plan.py)
  • 962054a6f5 opt-in dense MLP for Qwen4 checkpoints without experts (common/dense_mlp.py, mlp_selection.py)
  • c3971d2c89 test fix (TP group init for the dense MLP meta tests)

Upstream backports (applied in order without conflicts):

Touches the production Flash-Next path: vllm/models/qwen4_exp/amd/model.py and common/ngram_embedding.py.

Checks so far: py_compile on every touched file; the CPU tests in tools/rdna/qwen4/ (config probe, dense MLP, mono plan).

Before ready:

  • PLE offload overlap: [Qwen4Exp] PLE disk offload vllm-project/vllm#59958 overlaps the fork's PLE CPU/disk sidecar (vllm/v1/ple_offload/, VLLM_PLE_CPU_OFFLOAD, VLLM_PLE_QUANT_DIR). Readiness doc: keep the sidecar, take only the config surface, never run two offload mechanisms. Confirm the default path still goes through the sidecar.
  • GPU validation (docs/rdna2/qwen4/readiness.md, table 'GPU validation: pending'): compiled F&P boot of flashnext-mtp2 and flashnext-mtp0 (PLE int4), greedy probes, PPL, prefix cache, 16k/1k c8 vs rdna_extras, 0 faults.
  • Rebase: merge rdna_extras in first.

BlivionIaG and others added 8 commits November 3, 2026 08:19
Signed-off-by: BlivionIaG <kev29lt@gmail.com>
Signed-off-by: BlivionIaG <kev29lt@gmail.com>
…n int64 (vllm-project#64820)

Signed-off-by: Sean Bell <seanbell@resolve.ai>
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg>
Co-authored-by: Kimi Code <noreply@moonshot.cn>
Co-authored-by: vllm-agent <claw@inferact.ai>
…y profiling (vllm-project#60490)

Signed-off-by: Nick Kuhn <95254386+nkuhn-vmw@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg>
…/k/v and a BF16 QSA indexer (vllm-project#62318)

Signed-off-by: Seungryong Kim <137975233+srkim-business@users.noreply.github.com>
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg>
Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg>
Co-authored-by: Kimi <noreply@moonshot.ai>
Signed-off-by: Stefano Castagnetta <scastagnetta@nvidia.com>
Signed-off-by: Trosfy <61226342+Trosfy@users.noreply.github.com>
Co-authored-by: Kartikey Vyas <kartikey@artificialanalysis.ai>
Co-authored-by: Trosfy <61226342+Trosfy@users.noreply.github.com>
Co-authored-by: Artem Perevedentsev <aperevedents@nvidia.com>
Signed-off-by: BlivionIaG <kev29lt@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants