Skip to content

ref_fused_moe fails during CUDA graph capture: "wait method cannot be used for an event associated with a command graph" (FP8 MoE, dual Intel Arc Pro B70, TP=2) #567

Description

@uldiseihenbergs

Reproduced on 2× Intel Arc Pro B70 (Battlemage), tensor-parallel-size=2, serving Qwen/Qwen3.6-35B-A3B-FP8 (official Qwen release) via vLLM.

Image: vllm/vllm-openai-xpu@sha256:f01e24f6c7ff01f1e0662234255a1372297d1dbd89d003cf13c8fad3eab1ba4f

Weights load successfully, the MTP draft model loads successfully, torch.compile completes for both TP ranks. Failure occurs during CUDA graph capture warmup:

File "vllm_xpu_kernels/moe_utils.py", line 341, in ref_fused_moe
    n_tokens = rows_per_expert[i].item()
RuntimeError: wait method cannot be used for an event associated with a command graph.

Full call path: vllm/model_executor/layers/quantization/fp8.py (apply) → moe_kernel.applyvllm_xpu_kernels/fused_moe_interface.py (_apply_ref) → ref_fused_moe in moe_utils.py.

The .item() call forces a blocking host sync, which appears incompatible with being captured inside a CUDA/SYCL graph — this is the ref_fused_moe (reference, non-optimized) MoE path, so it's plausible the optimized path simply isn't graph-capture-safe yet on XPU for FP8 MoE.

Workaround (not a fix): --enforce-eager avoids the crash by skipping graph capture entirely, confirming the fault is isolated to the capture step — but throughput drops to ~3 tokens/s, so not usable in practice.

Reproducer script and full container log attached.

launch-vllm-2B70-fp8.sh

b70-fp8-mtp2-cache-on-2026-09-02T09-33-43.log

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions