Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 2 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,7 @@

> 🐉 LoongForge is part of Baidu Baige's **Loong** open-source series — named after the traditional Chinese **loong boat (龙舟)**, a symbol of coordinated power and forward momentum.

**LoongForge** is a unified training framework for **LLMs, VLMs, VLAs, and diffusion models**, covering **pre-training**, **continued pre-training**, and **SFT**. Built upon Megatron-LM with deep systemic enhancements across **model coverage**, **training performance**, and **hardware support**, it delivers **significant speedups over mainstream open-source baselines**.
**LoongForge** is a unified training framework for **LLMs, VLMs, diffusion, and embodied models**, covering **pre-training**, **continued pre-training**, and **SFT**. Built upon Megatron-LM with deep systemic enhancements across **model coverage**, **training performance**, and **hardware support**, it delivers **significant speedups over mainstream open-source baselines**.

Before going open-source, LoongForge was developed as **AIAK-Training-LLM**, Baidu Baige's training acceleration stack. It has supported production training for enterprise customers across **Education**, **Computer Vision**, and **Embodied AI**, typically delivering **30%~50% speedup over customer baselines**, with the largest production runs reaching **5,000+ XPUs**.

Expand Down Expand Up @@ -95,7 +95,7 @@ See the full documentation for installation, tutorials, and advanced usage — [

Measured on **v0.1.1** across LLM, VLM, VLA and DIT workloads against mainstream open-source training baselines:

<img width="2422" height="1264" alt="image" src="https://github.com/user-attachments/assets/1553b7eb-d379-4e63-b293-2a716c4996be" />
<img alt="LoongForge Benchmark Speedup" src="docs/assets/images/benchmark_speedup.png" />

<details>
<summary><b>📋 Detailed configurations & footnotes</b></summary>
Expand All @@ -109,7 +109,6 @@ Measured on **v0.1.1** across LLM, VLM, VLA and DIT workloads against mainstream
| Qwen3-VL-30B-A3B | VLM | VeOmni<sup>†</sup> | 32 × A800<sup>‡</sup> · GBS 128 · 32K | **1.45×** |
| GR00T N1.6 | VLA | LeRobot<sup>†</sup> | 8 × A800<sup>‡</sup> · GBS 128 · 224×224 | **2.31×** |
| Pi0.5 | VLA | OpenPI<sup>†</sup> | 8 × A800<sup>‡</sup> · GBS 112 · 224×224 | **1.65×** |
| Wan2.2 | DIT | DiffSynth<sup>†</sup> | 8 × A800<sup>‡</sup> · 480×832x49 | **2.16×** |

> <sup>§</sup> Due to test-bed scale limits, **DeepSeek-V3.2** was validated separately on a reduced-layer configuration — LoongForge's **DSA CUDA kernel optimizations** still deliver **~5× speedup** over Megatron-LM and reach **64K sequence** (baseline OOMs beyond 8K).<br>
> <sup>†</sup> Numbers reflect baseline and LoongForge versions at the time of measurement, and may evolve as implementations change.<br>
Expand Down
5 changes: 2 additions & 3 deletions README_zh.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,7 @@

> 🐉 LoongForge 是百度百舸 **Loong** 开源系列的一员 —— 名字源于中国传统 **龙舟**,象征协同发力与破浪前行。

**LoongForge** 是面向 **LLM、VLM、VLADiffusion 模型** 的统一训练框架,覆盖 **预训练(Pre-training)**、**持续预训练(Continued Pre-training)** 和 **SFT**。基于 Megatron-LM 在 **模型覆盖度**、**训练性能** 和 **硬件支持** 三个维度做了深度系统性增强,相对主流开源训练方案有**显著的性能提升**。
**LoongForge** 是面向 **LLM、VLM、DiffusionEmbodied 模型** 的统一训练框架,覆盖 **预训练(Pre-training)**、**持续预训练(Continued Pre-training)** 和 **SFT**。基于 Megatron-LM 在 **模型覆盖度**、**训练性能** 和 **硬件支持** 三个维度做了深度系统性增强,相对主流开源训练方案有**显著的性能提升**。

在开源之前,LoongForge 的前身是 **AIAK-Training-LLM** —— 百度百舸的训练加速栈,已在 **教育**、**计算机视觉** 和 **Embodied AI** 等多家企业客户的生产训练中落地,相对客户原有方案通常带来 **30%~50% 加速**,最大规模的生产训练任务达到 **5,000+ XPU**。

Expand Down Expand Up @@ -94,7 +94,7 @@
## 📊 性能 Benchmark

在 **v0.1.1** 版本上针对 LLM、VLM、VLA、DIT 四类工作负载,与主流开源训练方案的对比结果:
<img width="2422" height="1264" alt="image" src="https://github.com/user-attachments/assets/1553b7eb-d379-4e63-b293-2a716c4996be" />
<img alt="LoongForge Benchmark Speedup" src="docs/assets/images/benchmark_speedup.png" />

<details>
<summary><b>📋 细节描述</b></summary>
Expand All @@ -108,7 +108,6 @@
| Qwen3-VL-30B-A3B | VLM | VeOmni<sup>†</sup> | 32 × A800<sup>‡</sup> · GBS 128 · 32K | **1.45×** |
| GR00T N1.6 | VLA | LeRobot<sup>†</sup> | 8 × A800<sup>‡</sup> · GBS 128 · 224×224 | **2.31×** |
| Pi0.5 | VLA | OpenPI<sup>†</sup> | 8 × A800<sup>‡</sup> · GBS 112 · 224×224 | **1.65×** |
| Wan2.2 | DIT | DiffSynth<sup>†</sup> | 8 × A800<sup>‡</sup> · 480×832x49 | **2.16×** |

> <sup>§</sup> 受测试台规模限制,**DeepSeek-V3.2** 在减层配置下单独验证 —— LoongForge 的 **DSA CUDA Kernel 优化** 相对 Megatron-LM 仍带来 **~5× 加速**,并可支持 **64K 序列长度**(基线在 8K 以上即 OOM)。<br>
> <sup>†</sup> 数据反映测量时对应基线的实现,后续可能随实现演进而变化。<br>
Expand Down
1 change: 0 additions & 1 deletion docs/assets/images/benchmark_image.py
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,6 @@
ROWS = [
("DeepSeek-V3.2 Lite", "MoE + DSA", "Megatron-LM", "Reduced layers · GBS 128 · 8K", 5.04, "§"),
("GR00T N1.6", "VLA", "LeRobot", "8 × A800 · GBS 128 · 224×224", 2.31, ""),
("Wan 2.2", "DIT", "DiffSynth", "8 × A800 · 480×832×49", 2.16, ""),
("Pi 0.5", "VLA", "OpenPI", "8 × A800 · GBS 112 · 224×224", 1.65, ""),
("Qwen3-VL-30B-A3B", "VLM", "VeOmni", "32 × A800 · GBS 128 · 32K", 1.45, ""),
("Qwen3-30B-A3B", "MoE", "Megatron-LM", "32 × A800 · GBS 1024 · 32K", 1.16, ""),
Expand Down
Binary file added docs/assets/images/benchmark_speedup.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading