FlagScale Train Upgrade: Synchronization with Megatron-LM Core0.17.0 (#1193)

**FlagScale Training Upgrade**
Sync FlagScale training layer with upstream Megatron-LM core_r0.17.0.
Key changes:
- Replace training/ and legacy/ with upstream 0.17.0 content, then
re-apply FlagScale patches
- Unify tokenizer system with registration-based factory dispatch,
remove legacy_tokenizer flag
- Replace torch.cuda calls with cur_platform abstraction in
training/models
- Sync arguments.py with upstream FP4 param gather changes, guard
hetero-specific args
- Fix loss accumulator reset bug (halved lm_loss at iter 2)
- Remove FlagScale-forked Muon optimizer code, use upstream 0.17.0
implementation
  - Remove stale moe_use_legacy_grouped_gemm references     
- Found that test_train_robobrain_x0.py was leaking MagicMock objects
into sys.modules during collection, breaking test_parallel_context.py.
Applied the fix by restoring modules immediately after import instead of
deferring cleanup.

---------

Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
Co-authored-by: heavyrain <1528794076@qq.com>

Author: 3 people

docs/getting-started.md

@@ -49,7 +49,7 @@ folder to get started.
     pip install -U scikit-build-core==0.11 pybind11 ninja cmake
     git clone https://github.com/flagos-ai/FlagGems
     cd FlagGems
-    pip install --no-build-isolation . 
+    pip install --no-build-isolation .
     ```
     See more details in [FlagGems](https://github.com/flagos-ai/FlagGems)
 

examples/aquila/conf/train/1_8b.yaml

@@ -60,6 +60,7 @@ model:
   clip_grad: 1.0
   train_samples: 8192  #8192 #29297664 #120B tokens 100step
   eval_iters: 0
+  eval_interval: 1000
   micro_batch_size: 8
   global_batch_size: 1024
   seed: 42

examples/aquila/conf/train/3b.yaml

@@ -68,6 +68,7 @@ model:
   clip_grad: 1.0
   train_samples: 244141056 #1TB tokens #29297664 #120B tokens
   eval_iters: 0
+  eval_interval: 1000
   micro_batch_size: 2
   global_batch_size: 2048
   seed: 42

examples/aquila/conf/train/7b.yaml

@@ -39,6 +39,7 @@ model:
   clip_grad: 1.0
   train_samples: 1002539063
   eval_iters: 0
+  eval_interval: 1000
   micro_batch_size: 2
   global_batch_size: 1728
   seed: 1234
@@ -57,7 +58,6 @@ data:
   data_path: ${data_path:??}
   split: 1
   tokenizer:
-    legacy_tokenizer: true
     tokenizer_type: AquilaTokenizerFS
     vocab_file: ./examples/aquila/tokenizer/vocab.json
     merge_file: ./examples/aquila/tokenizer/merges.txt

examples/deepseek_v3/conf/train/11b_a2b.yaml

@@ -90,6 +90,7 @@ model:
   micro_batch_size: 1
   global_batch_size: 256
   eval_iters: 0
+  eval_interval: 1000
   train_samples: 244141056 #1T #29297664 #120B tokens
 
   optimizer:

examples/deepseek_v3/conf/train/16b_a3b.yaml

@@ -89,6 +89,7 @@ model:
   micro_batch_size: 1
   global_batch_size: 128 #2048
   eval_iters: 0
+  eval_interval: 1000
   train_iters: 102400
 
   optimizer:

examples/deepseek_v3/conf/train/16b_a3b_finetune.yaml

@@ -88,6 +88,7 @@ model:
   micro_batch_size: 1
   global_batch_size: 128 #2048
   eval_iters: 0
+  eval_interval: 1000
   train_iters: 102400
 
   # lora finetune

examples/deepseek_v3/conf/train/engram.yaml

@@ -120,6 +120,7 @@ model:
   micro_batch_size: 2
   global_batch_size: 2048
   eval_iters: 0
+  eval_interval: 1000
   train_iters: 20
 
   optimizer:
@@ -142,7 +143,6 @@ data:
   split: 1
   no_mmap_bin_files: true
   tokenizer:
-    legacy_tokenizer: true
     tokenizer_type: QwenTokenizerFS
     tokenizer_path: /workspace/qwentokenizer
     vocab_size: 151851

examples/llava_onevision/conf/train/1_5b.yaml

@@ -50,6 +50,7 @@ model:
   profile_ranks: 7
   use_pytorch_profiler: True
   eval_iters: 0
+  eval_interval: 1000
   micro_batch_size: 2
   global_batch_size: 512
   allow_missing_vision_projection_checkpoint: True

examples/llava_onevision/conf/train/7b.yaml

@@ -44,6 +44,7 @@ model:
   clip_grad: 1.0
   train_iters: 625
   eval_iters: 0
+  eval_interval: 1000
   micro_batch_size: 1
   global_batch_size: 320
   allow_missing_vision_projection_checkpoint: True