You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
@@ -39,8 +39,8 @@ Before going open-source, LoongForge was developed as **AIAK-Training-LLM**, Bai
39
39
40
40
## 🔥 Latest News
41
41
42
-
-**[2026/05]** ✨ Added training support for **Kimi K2.5 / K2.6**, and introduced **NVFP4** PTQ quantization.
43
-
-**[2026/05]** ⚡ Accelerated **Wan 2.2** training by **130%**, and added context parallelism (CP) and data packing support.
42
+
-**[2026/05]** ✨ Added training support for **Kimi K2.5 / K2.6**, and introduced **INT4 / NVFP4** PTQ.
43
+
-**[2026/05]** ⚡ Accelerated **Wan 2.2** training by **130%**, and added CP and data packing support.
44
44
-**[2026/05]** 🎉 **v0.1.0** — first official tagged release of LoongForge.
45
45
-**[2026/05]** 🌟 Powered the training and public release of **LLaVA-OneVision-2.0**.
46
46
-**[2026/05]** 🤖 Expanded VLA coverage with **GR00T N1.6**; **60%+ speedup** on Pi0.5 and GR00T training.
@@ -65,9 +65,9 @@ See the full documentation for installation, tutorials, and advanced usage — [
65
65
## ✨ Key Features
66
66
67
67
***🧩 Flexible Multi-Modal Composition** — Configuration-driven assembly of VLMs from interchangeable ViT and LLM components.
68
-
***⚡ Heterogeneous Parallelism** — Independent TP / DP / recompute per model component (e.g., ViT vs. LLM) for optimal throughput and memory.
68
+
***⚡ Heterogeneous Parallelism** — Independent TP / DP / recompute per model component (e.g., ViT vs. LLM) for optimal throughput and memory.[[blog](https://baidu-baige.github.io/LoongForge/blog/2026-05-loongforge-heterogeneous-parallel-training.html)]
69
69
***🔀 Decoupled Encoder-Decoder Training** — Separates ViT and LLM into independent tasks, eliminating encoder-induced pipeline bubbles.
***🚀 MoE-Native Optimization** — Overlapped All2All / activation offload / compute, with **further memory reduction** beyond upstream Megatron-LM on DeepSeek-V3, Qwen3-MoE, etc.
72
72
***🔬 Adaptive FP8 Training** — End-to-end FP8 for LLMs and VLMs with standard **blockwise FP8**; optional **adaptive** mode picks per-operator precision by GEMM shape and efficiency.
73
73
***🔧 Custom Fused Operators** — Fused kernels like **FusedDSA** for DSA-style models — TileLang version open-sourced, high-performance CUDA version available on Baidu Baige platform.
0 commit comments