Skip to content

Commit 11ef9a5

Browse files
committed
fix(models): reject DeepSeek V4 FP4 tensor parallelism
Fail closed before loading unsupported FP4 MoE tensor-parallel configurations. Fixes #47528 Co-authored-by: Devin AI <noreply@devin.ai> Signed-off-by: Sidharth <dumko.raj@gmail.com> Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> Signed-off-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
1 parent 1ff9429 commit 11ef9a5

1 file changed

Lines changed: 7 additions & 0 deletions

File tree

vllm/models/deepseek_v4/quant_config.py

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -140,6 +140,13 @@ def get_quant_method(self, layer, prefix):
140140
):
141141
return UnquantizedFusedMoEMethod(layer.moe_config)
142142
if self.expert_dtype == "fp4":
143+
parallel_config = get_current_vllm_config().parallel_config
144+
if parallel_config.tensor_parallel_size > 1:
145+
raise ValueError(
146+
"DeepSeek V4 FP4 MoE is not supported with tensor "
147+
"parallelism. Use data parallelism with expert "
148+
"parallelism instead."
149+
)
143150
if self.moe_quant_algo == "NVFP4":
144151
from vllm.model_executor.layers.quantization.modelopt import (
145152
ModelOptNvFp4FusedMoE,

0 commit comments

Comments
 (0)