[Feature]: [Batch-Invariant-Kernels] GDN_ATTN does not support batch-invariant mode for Qwen3.5/Qwen3.6 GDN models

[cm2435]

🚀 The feature, motivation and pitch

Summary

I would like to request (and offer to implement) batch-invariant compute support for the GDN_ATTN backend used by public Qwen GDN architectures.

Today, VLLM_BATCH_INVARIANT=1 is unsupported for these models. The GDN_ATTN backend inherits the default supports_batch_invariance() = False, so vLLM correctly rejects these models during engine initialization rather than running them under an unsupported BIC path.

I validated the current support boundary with:

• Qwen/Qwen3.5-0.8B
• Qwen/Qwen3.6-35B-A3B
• vLLM 0.25.1

As a control, Qwen/Qwen2.5-VL-3B-Instruct works for a small image-request BIC smoke test with tp=1 and tp=2. The unsupported path appears when moving to Qwen3.5/Qwen3.6 architectures that include GDN layers.

Motivation

My motivation is to help extend BIC coverage to newer hybrid/GDN architectures used in VLA-style models. We do large scale post-training of models in this family and would like to contribute upstream support for bitwise-deterministic serving paths where possible, starting from public Qwen3.5/Qwen3.6 validation cases rather than any private model-specific setup.

Public validation cases

Small public repro

VLLM_BATCH_INVARIANT=1 \
CUDA_VISIBLE_DEVICES=0 \
vllm serve Qwen/Qwen3.5-0.8B \
  --host 0.0.0.0 \
  --port 8000 \
  --tensor-parallel-size 1 \
  --max-model-len 4096 \
  --gpu-memory-utilization 0.80 \
  --trust-remote-code

Current behavior:

Resolved architecture: Qwen3_5ForConditionalGeneration
Using backend AttentionBackendEnum.FLASH_ATTN for vit attention
Using FlashInfer GDN prefill kernel (requested=auto, head_k_dim=128).
Using FLASH_ATTN attention backend out of potential backends: ['FLASH_ATTN', 'TRITON_ATTN', 'FLEX_ATTENTION'].
Loading weights took 0.37 seconds
Model loading took 1.72 GiB memory and 7.535573 seconds
RuntimeError: VLLM batch_invariant mode is not supported for GDN_ATTN.

Larger public repro

VLLM_BATCH_INVARIANT=1 \
CUDA_VISIBLE_DEVICES=0,1 \
vllm serve Qwen/Qwen3.6-35B-A3B \
  --host 0.0.0.0 \
  --port 8000 \
  --tensor-parallel-size 2 \
  --enable-expert-parallel \
  --max-model-len 4096 \
  --max-num-batched-tokens 8192 \
  --max-num-seqs 32 \
  --gpu-memory-utilization 0.85 \
  --mm-encoder-tp-mode data \
  --trust-remote-code \
  --safetensors-load-strategy prefetch

Current behavior:

Resolved architecture: Qwen3_5MoeForConditionalGeneration
Using backend AttentionBackendEnum.FLASH_ATTN for vit attention
Using FlashInfer GDN prefill kernel (requested=auto, head_k_dim=128).
Using FLASH_ATTN attention backend out of potential backends: ['FLASH_ATTN', 'TRITON_ATTN', 'FLEX_ATTENTION'].
Loading weights took 11.58 seconds
Model loading took 33.27 GiB memory and 42.684858 seconds
RuntimeError: VLLM batch_invariant mode is not supported for GDN_ATTN.

Implementation starting points

Potential places to start:

• vllm/v1/attention/selector.py: contains the BIC support check for Mamba/GDN-style attention backends.
• vllm/v1/attention/backends/gdn_attn.py: defines GDNAttentionBackend, which reports GDN_ATTN.
• vllm/v1/attention/backend.py: defines the default supports_batch_invariance() = False behavior inherited by unsupported backends.
• vllm/model_executor/layers/mamba/gdn/qwen_gdn_linear_attn.py: Qwen GDN implementation path selected during the validation runs.

The validation logs show this path selecting the GDN prefill backend:

Using FlashInfer GDN prefill kernel (requested=auto, head_k_dim=128).

Requested support

Add BIC support for GDN_ATTN so Qwen3.5/Qwen3.6 GDN models can run under VLLM_BATCH_INVARIANT=1.

A conservative implementation path could be:

1. Add a public validation harness using Qwen/Qwen3.5-0.8B.
2. Validate deterministic solo-vs-batched behavior for the supported GDN backend/backend combination.
3. Enable supports_batch_invariance() only for the GDN path that passes those invariance checks.
4. Document any remaining unsupported GDN backend combinations explicitly.

Environment

• vLLM: 0.25.1
• GPU: NVIDIA H100 80GB
• CUDA runtime as packaged in the vLLM environment
• Models:
  • Qwen/Qwen3.5-0.8B
  • Qwen/Qwen3.6-35B-A3B

Question

Would a contribution adding GDN_ATTN batch-invariance support be welcome?

I am happy to start with a small public validation harness using Qwen/Qwen3.5-0.8B, then narrow the implementation to the supported GDN backend path. A first pass could be conservative, for example enabling BIC only for the GDN backend/backend combination that passes invariance checks, rather than broadly marking all GDN paths as supported.

Alternatives

No response

Additional context

No response

Before submitting a new issue...

• Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.

[cm2435]

@yewentao256 I found #42960 and #45819 after opening #48613, sorry for the duplicate issue.

I can see there is some ongoing debate in #45819 about whether the current tests are sufficient / whether simply enabling supports_batch_invariance() is enough for GDN_ATTN.

I’m going to try following the feedback you gave @yuvalluria: get the determinism tests set up locally, run larger-batch validation on an H100 node with public Qwen3.5/Qwen3.6 models, and then see what implementation work is actually needed from there.

Let me know if this is duplicative or unnecessary and I’ll avoid wasting maintainer time :)

[cm2435]

Update: I did a small validation pass to check whether the support-flag-only approach is sufficient.

I tested a local spike equivalent to the core backend change in #45819:

GDNAttentionBackend.supports_batch_invariance() -> True

That is enough to get Qwen/Qwen3.5-0.8B past engine initialization under VLLM_BATCH_INVARIANT=1, but it did not pass bs=1 vs bs=N bitwise validation.

Setup:

• vLLM 0.25.1
• NVIDIA H100 80GB
• model: Qwen/Qwen3.5-0.8B
• VLLM_BATCH_INVARIANT=1
• compare the same prompts run individually (bs=1) vs as one batch (bs=N)
• compare generated token IDs and selected-token logprobs exactly
• sampling: temperature=0.6, top_p=1.0, max_tokens=8, seed=42, logprobs=5

Results:

Default GDN prefill backend:

Using FlashInfer GDN prefill kernel (requested=auto, head_k_dim=128).
[result] mismatches=18 batch_size=32

Forced Triton/FLA GDN prefill backend:

Using Triton/FLA GDN prefill kernel (requested=triton, head_k_dim=128).
[result] mismatches=22 batch_size=32 gdn_backend=triton

The default/auto run included one sampled-token divergence plus selected-token logprob mismatches. The forced Triton run also had selected-token logprob mismatches.

So my current read is that setting supports_batch_invariance() = True is necessary to pass the BIC support gate, but not sufficient for correctness. There appears to be remaining batch-invariance work needed in the GDN execution path and/or recurrent state handling.

I pushed the validation spike here in case it is useful for review:

https://github.com/cm2435/vllm/tree/gdn-bic-validation-spike

[yewentao256]

Feel free to submit a PR, I guess @yuvalluria is busy recently and haven't updated the PR for weeks.

Thanks for the work!