[v1] support training with fsdp2

[frozenleaves]

What does this PR do?

1. 新增功能支持：

   • fsdp2+sft训练
   • 通过Meta Device初始化模型
   • 同时支持hf格式的模型权重与dcp格式的模型权重加载

2. 完善v1 launch启动命令，在 USE_V1 时，当前支持如下cli命令:

   以下命令与v0兼容，功能相同：

   • llamafactory-cli help
   • llamafactory-cli env
   • llamafactory-cli version
   • llamafactory-cli chat

   以下命令与v0不兼容：

   • llamafactory-cli train: 启动 sft 训练，在 v0 下所有训练均通过该接口启动，v1 下该接口仅支持启动 sft 训练
   • llamafactory-cli sft: 启动 sft 训练, 与llamafactory-cli train 等价
   • llamafactory-cli dpo：启动 dpo 训练，v1 特有接口
   • llamafactory-cli rm：启动 rm 训练，v1 特有接口

3. 增加 sft+fsdp2训练端到端ut，看护基本训练能力

4. 增加 max_steps参数支持

5. 修复部分代码bug

[gemini-code-assist]

Summary of Changes

Hello @frozenleaves, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

此拉取请求通过集成 FSDP2 (Fully Sharded Data Parallel v2) 显著增强了 LLaMA Factory 的分布式训练能力，特别是针对 SFT (Supervised Fine-Tuning)。它引入了通过 Meta Device 进行灵活模型初始化，并支持从 Hugging Face 和 Distributed Checkpoint (DCP) 两种格式加载模型权重。此外，PR 还优化了 v1 命令行启动命令，以更好地分类训练任务，并增加了 max_steps 参数以更精细地控制训练时长。这些改进旨在提升大型模型训练的效率和稳定性。

Highlights

• 新增功能支持: 支持 FSDP2 + SFT 训练，通过 Meta Device 初始化模型，并同时支持 Hugging Face 格式与 Distributed Checkpoint (DCP) 格式的模型权重加载。
• 完善 v1 启动命令: 优化了 llamafactory-cli 的 v1 启动命令，train 命令现在仅支持 SFT 训练，并新增了 sft (等价于 train)、dpo (尚未实现) 和 rm (尚未实现) 命令。同时保持了与 v0 兼容的 help、env、version 和 chat 命令。
• 增加 SFT + FSDP2 训练端到端单元测试: 新增了针对 SFT + FSDP2 训练的端到端单元测试，以确保基本训练功能的稳定性。
• 增加 max_steps 参数支持: 引入了 max_steps 参数，允许用户指定最大训练步数，该参数将优先于 num_train_epochs。
• 修复部分代码 bug: 修复了代码中的一些 bug，提高了系统的健壮性。

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Code Review

此 PR 引入了 FSDP2 支持，是一项重大的功能新增。实现内容包括一个新的 FSDP 插件、针对 torchrun 的启动器重构，以及一个端到端的测试。代码整体结构良好。

我在启动器脚本中发现一个关于 subprocess.run 参数解析的潜在错误，如果参数包含空格，可能会导致执行失败。此外，我也在 FSDP 插件中发现了一些可以提升代码稳健性的细节。总体而言，这是一次出色的贡献。

[hiyouga]

LGTM