Skip to content

Commit 60a3947

Browse files
committed
test(recipe): moonlight TE cudagraphs + only moe_shared_expert_overlap off (true minimal crash-fix)
Signed-off-by: oliver könig <okoenig@nvidia.com>
1 parent 5c9ea4c commit 60a3947

1 file changed

Lines changed: 1 addition & 1 deletion

File tree

src/megatron/bridge/recipes/moonlight/moonlight_16b.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -156,7 +156,7 @@ def moonlight_16b_pretrain_config() -> ConfigContainer:
156156
cfg.comm_overlap = CommOverlapConfig(tp_comm_overlap=False)
157157
cfg.comm_overlap.delay_wgrad_compute = False
158158
cfg.comm_overlap.overlap_moe_expert_parallel_comm = False
159-
cfg.model.moe_shared_expert_overlap = True
159+
cfg.model.moe_shared_expert_overlap = False
160160

161161
# Checkpoint config
162162
cfg.checkpoint.save_interval = 2000

0 commit comments

Comments
 (0)