Skip to content

Commit 9f3a99d

Browse files
committed
fix autotuning
1 parent e4f7303 commit 9f3a99d

1 file changed

Lines changed: 9 additions & 1 deletion

File tree

flashinfer/fused_moe/core.py

Lines changed: 9 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2154,9 +2154,17 @@ def trtllm_mxint4_block_scale_moe_op(
21542154
use_shuffled_weight=True,
21552155
)
21562156
tunning_config = MoERunner.tuning_config_no_hidden_states_scales
2157+
# Create placeholder for tuning when routing_logits is None (routed mode)
2158+
routing_logits_for_tuning = (
2159+
routing_logits
2160+
if routing_logits is not None
2161+
else torch.empty(
2162+
num_tokens, num_experts, dtype=routing_dtype, device="meta"
2163+
)
2164+
)
21572165
inputs = [
21582166
output,
2159-
routing_logits,
2167+
routing_logits_for_tuning,
21602168
topk_ids,
21612169
expert_weights,
21622170
hidden_states,

0 commit comments

Comments
 (0)