Skip to content

Commit de8f429

Browse files
chore: llama-3.1-8b enable turbo (#250)
1 parent e539dc7 commit de8f429

File tree

1 file changed

+5
-0
lines changed

1 file changed

+5
-0
lines changed

examples/megatron/configs/MI300X/llama3.1_8B-pretrain.yaml

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -66,6 +66,11 @@ modules:
6666
disable_last_saving: true
6767
ckpt_format: torch
6868

69+
# Turbo
70+
enable_primus_turbo: true
71+
use_turbo_attention: true
72+
use_turbo_grouped_mlp: true
73+
6974
# Cross entropy flags
7075
cross_entropy_fusion_impl: "te"
7176
cross_entropy_loss_fusion: true

0 commit comments

Comments
 (0)