[CPU][Spec Decode] Optimize GDN conv path for speculative decoding

[tianmu-li]

Purpose

Improve CPU serving performance for Qwen3.5 GDN speculative decoding with MTP by replacing fallback torch conv1d path with cpp implementation when AMX is available.

Test Plan

Perf:
Server:

VLLM_CPU_SGL_KERNEL=1 \
vllm serve Qwen/Qwen3.5-9B \
  --max-model-len 2048 \
  --max-num-seqs 8 \
  --limit-mm-per-prompt '{"image":0,"video":0}' \
  --speculative-config '{"method":"mtp","num_speculative_tokens":<1-or-3>}'

Client:

vllm bench serve \
  --backend openai \
  --endpoint /v1/completions \
  --model Qwen/Qwen3.5-9B \
  --dataset-name sonnet \
  --dataset-path benchmarks/sonnet.txt \
  --sonnet-input-len 256 \
  --sonnet-output-len 256 \
  --max-concurrency 8 \
  --num-prompts 24

Accuracy:

import json
import os

import lm_eval


def main() -> None:
    config = os.environ["ACC_CONFIG"]
    out = os.environ["ACC_OUT"]

    model_args = {
        "pretrained": "Qwen/Qwen3.5-9B",
        "dtype": "bfloat16",
        "trust_remote_code": True,
        "max_model_len": 4096,
        "max_num_seqs": 1,
        "enable_prefix_caching": False,
        "enable_thinking": False,
        "limit_mm_per_prompt": {"image": 0, "video": 0},
    }

    if config == "mtp3":
        model_args["speculative_config"] = {
            "method": "mtp",
            "num_speculative_tokens": 3,
        }
    elif config != "no_mtp":
        raise ValueError(f"unknown ACC_CONFIG={config}")

    result = lm_eval.simple_evaluate(
        model="vllm",
        model_args=model_args,
        tasks=["gsm8k"],
        num_fewshot=5,
        batch_size=1,
        limit=100,
        apply_chat_template=True,
        fewshot_as_multiturn=True,
        gen_kwargs={"max_gen_toks": 512, "temperature": 0.0},
        log_samples=True,
    )

    with open(out, "w") as f:
        json.dump(result, f, indent=2, default=str)


if __name__ == "__main__":
    main()

Test Result

Performance

config	before output tok/s	after output tok/s	delta
no spec decode	70.29	70.16	-0.18%
MTP k=1	63.81	74.82	+17.26%
MTP k=3	72.82	86.29	+18.50%

Accuracy

Run	Exit	strict-match	flexible-extract
before/no_mtp	0	0.93	0.94
before/mtp3	0	0.93	0.94
after/no_mtp	0	0.93	0.94
after/mtp3	0	0.93	0.94

---

Essential Elements of an Effective PR Description Checklist

• The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
• The test plan, such as providing test command.
• The test results, such as pasting the results comparison before and after, or e2e results
• (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.

[claude]

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

[bigPYJ1151]

Suggested change

	if not can_use_native_conv:
	else: