Skip to content

Commit dddcffe

Browse files
committed
Add figures comparing LoRA on FSDP and megatron
Signed-off-by: Hollow Man <[email protected]>
1 parent 2542c79 commit dddcffe

File tree

1 file changed

+7
-0
lines changed

1 file changed

+7
-0
lines changed

docs/advance/ppo_lora.rst

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -132,6 +132,13 @@ Make sure you use Megatron-Bridge later than 0.2.0, and we recommended using `th
132132
freeze_vision_projection: True
133133
freeze_language_model: True
134134
135+
LoRA training experiment with Qwen3-8B on 8 * H200 single node comparing FSDP and Megatron backend (script adapted from examples/grpo_trainer/run_qwen2-7b_math_megatron_lora.sh):
136+
137+
.. image:: https://github.com/user-attachments/assets/0482f423-01a3-4e52-a7ee-8b9cd79b7b1a
138+
.. image:: https://github.com/user-attachments/assets/6ce10400-8164-47d8-90a6-c1bf002fb9e8
139+
.. image:: https://github.com/user-attachments/assets/092d3a43-4eba-425e-a584-8d83c1f02de4
140+
141+
135142
Best Practices and Notes
136143
-------------------------
137144

0 commit comments

Comments
 (0)