[Bug][Spec Decode] Hybrid GDN (Qwen3.5/Qwen3.8 27B-class) + MTP: scheduler runs only ~3 concurrent sequences at batch >= 4 — acceptance/throughput collapse

[jsn1993]

Title

[Bug][Spec Decode] Hybrid GDN (Qwen3.5/Qwen3.8 27B-class) + MTP: scheduler runs only ~3 concurrent sequences at batch >= 4 — acceptance/throughput collapse

Summary

On a 27B Qwen3.5-family hybrid GDN model (64 layers = 48 linear-attention
GDN + 16 full attention) with MTP enabled, the v1 scheduler stops running
the full batch under speculative decoding: at batch = 8, every decode
iteration schedules exactly 3 of the 8 sequences (per-seq dist
[2, 2, 2] = 1 target + 1 draft token each); the other 5 wait
indefinitely and rotate in only as earlier sequences finish. The batch
"rotates" through a ~3-wide MTP concurrency window, so mean emitted
tokens per sequence per iteration drops to 0.27-0.6 and MTP becomes 2-3x
slower than no-spec decoding. bs <= 3 is healthy (per-position
acceptance 0.67-0.74, +33-72% throughput).

The same harness on the 24-layer-GDN 9B sibling model schedules all
batches up to bs = 20 with MTP and shows no collapse — the cap scales
with the GDN layer count / per-sequence mamba state size, which points
at the mamba cache budget being shared between target states and MTP
draft slots in the scheduler accounting.

Environment

• vLLM built from source at f4eccda (current main as of 2026-09-06);
  also reproduces on several builds over the past week
• torch 2.13.0+cu130, RTX 4090 D (SM89), WSL2, driver 615.65.06
• model: Qwen3.8-27B GPTQ INT4 g128 (Marlin path) — quant-path
  independent (W4A16 vs W4FP8 identical), lm_head fp8 on/off identical
• offline LLM(...), VLLM_ENABLE_V1_MULTIPROCESSING=0,
  mamba_ssm_cache_dtype=bfloat16, gpu_memory_utilization 0.90,
  max_model_len 4608, max_num_seqs 25, KV bf16 and fp8 both tested

Repro

llm = LLM(model=<27B hybrid GDN checkpoint>, max_model_len=4608,
          gpu_memory_utilization=0.90, max_num_seqs=25,
          mamba_ssm_cache_dtype="bfloat16",
          speculative_config={"method": "mtp",
                              "num_speculative_tokens": 1})
# 8 prompts x 256 tokens, greedy, 48 generated tokens

Patch Scheduler.schedule to log
(total_num_scheduled_tokens, len(num_scheduled_tokens), dist) per call
after warmup. Every decode iteration shows total=6, dist=[2,2,2].

Full trace: run 8x256-token prompts, greedy 256 tokens, and record
per-iteration emitted — iterations average 4.95 emitted tokens for 8
active sequences (e.g. 414 iterations for 8x256 tokens).

Observed

bs	no-spec tok/s	MTP k=1 tok/s	mean accepted len (per seq per iter)
1	56.4	72.0	0.86
3	150.6	203.7	0.84
4	195.4	157.8	-0.03
8	367.1	197.0	-0.38
20	649.0	210.9	-0.73

(mean accepted len < 1 means some iterations schedule a given sequence
with zero tokens; decode iterations exceed generated tokens — 962
iterations for 20x256 tokens at bs=20.)

• Correctness is preserved: greedy outputs still match the no-spec
  baseline on some legs (2/2), so this is a concurrency/scheduling
  limitation, not output corruption.
• Path-independent: identical collapse on GPTQ-Marlin W4A16 and
  W4A8-FP8 (int4 x fp8 activations), on bf16 and fp8 KV cache, with
  Triton-FLA and fused-CUDA GDN decode kernels.
• Upstream 874df93 (preserve mamba state for padded prompt tails)
  fixed exactly the bs=4 point; bs>=5 persists.

Expected

All batch sequences are scheduled under MTP every iteration (mean
accepted length per sequence per iteration >= 1.0, as observed on the
24-GDN-layer 9B sibling model up to bs=20).

Related but distinct

• [Bug]: Prefill misdispatched into spec-decode FULL cudagraph when prompt length == 1 + num_speculative_tokens → silent GDN state loss, garbage output (hybrid/Qwen3-Next models) #53051: GDN state loss when prompt length == 1 + k (shape-aliasing
  prefill dispatch) — different trigger; our repro uses 256-token
  prompts at k=1 on current main.
• [Bug] Qwen3.8-Flash-Next (qwen4_exp) + MTP: episodic 0% draft acceptance + repetition collapse in thinking block until max_tokens (SM120, bf16 KV, nightly @73029d4) #55357: episodic 0% acceptance on qwen4_exp — similar symptom,
  different model family.
• [Performance]: Qwen3.5 native MTP can be slower than no-MTP CUDA graph baseline despite good acceptance #47277 / [Performance]: Qwen3.5 with mtp is slower than without #39680: "Qwen3.5 MTP slower than no-MTP" performance reports
  — this trace is (we believe) the mechanism at 27B scale.

Note

The debugging data above was collected with an agent-driven harness
under human supervision; happy to run anything else needed (traces,
config variations) on the 27B checkpoint.