Skip to content

[fsdp, model, training_utils, hardware] feat: add MXFP QAT for dense linear and MoE - #1

Merged
Sunwish merged 6 commits into
Ascend-TechDev:lab/A2_MXFP_RL_Labfrom
Sunwish:pr/01-mxfp-qat-core
Oct 29, 2026
Merged

Sunwish merged 6 commits into
Ascend-TechDev:lab/A2_MXFP_RL_Labfrom
Sunwish:pr/01-mxfp-qat-core

Conversation

@Sunwish

@Sunwish Sunwish commented Oct 29, 2026

Copy link
Copy Markdown
Collaborator

What does this PR do?

Adds configurable MXFP QAT support to the FSDP2 engine for dense linear layers and packed MoE experts.

The implementation supports MXFP8 W8A8 on the Ascend NPU backend and MXFP4 modes through the Torch QDQ backend. Dense linear layers can independently apply fake quantization and optional seeded block rotation to fprop, dgrad, and wgrad. The MoE path applies QAT at the packed NPU grouped-matmul boundary.

Related work includes #7895, which adds Megatron/ModelOpt MXFP4 expert weight QAT, #8353, which adds MiniMax M3-specific activation QAT, and #6992, which adds W4A8 QAT simulation. This PR adds a configurable FSDP2 path for dense linear transforms and Ascend packed MoE grouped matmuls.

Checklist Before Starting

Test

  • Added CPU tests for MXFP quantization, rotation, dense-linear gradient paths, and MoE operand transforms.
  • git diff --check f5aabaff..19f96b16 passed.
  • Attempted python -m pytest tests/utils/test_mxfp_quant_cpu.py tests/utils/test_mxfp_moe_cpu.py -q; collection could not run because PyTorch is not installed in the current environment.
  • No Ascend end-to-end training or evaluation result is included in this commit range. Add an NPU smoke run or experiment result before requesting review.

API and Usage Example

Example FSDP2 engine configuration for MXFP8 QAT on Ascend:

strategy: fsdp2
qat:
  enable: true
  mode: mxfp_w8a8
  group_size: 32
  quant_backend: npu
  fake_quant_targets: [Fprop, Dgrad, Wgrad]
  backward_reduction_axis_fake_quant: true
  rotation_enable: true
  rotation_targets: [Fprop, Dgrad, Wgrad]
  experts:
    enable: true
    npu_backend: grouped_matmul

### Design & Code Changes

1. Add MXFP8/MXFP4 quantization primitives and optional deterministic block rotation.
2. Add an nn.Linear replacement with independent fprop, dgrad, and wgrad transforms, plus optional caching of backward reduction-axis operands.
3. Integrate QAT with the packed NPU grouped-matmul path used by the Qwen3-family routed MoE implementation.
4. Extend FSDP engine configuration and skip the legacy QAT fusion path for MXFP modes.
5. Add CPU tests and low-volume runtime diagnostics for the configured execution paths.

### Checklist Before Submitting

> [!IMPORTANT]
> Please check all the following items before requesting a review, otherwise the reviewer might deprioritize this PR for review.

- [ ] Read the [Contribute Guide](https://github.com/verl-project/verl/blob/main/CONTRIBUTING.md).
- [ ] Apply [pre-commit checks](https://github.com/verl-project/verl/blob/main/CONTRIBUTING.md#code-linting-and-formatting): `pre-commit install && pre-commit run --all-files --show-diff-on-failure --color=always`
- [ ] Add / Update [the documentation](https://github.com/verl-project/verl/tree/main/docs).
- [ ] Add unit or end-to-end test(s) to [the CI workflow](https://github.com/verl-project/verl/tree/main/.github/workflows) to cover all the code. If not feasible, explain why: ...
- [ ] Once your PR is ready for CI, send a message in [the `ci-request` channel](https://verl-project.slack.com/archives/C091TCESWB1) in [the `verl` Slack workspace](https://join.slack.com/t/verl-project/shared_invite/zt-3855yhg8g-CTkqXu~hKojPCmo7k_yXTQ). (If not accessible, please try [the Feishu group (飞书群)](https://applink.larkoffice.com/client/chat/chatter/add_by_link?link_token=772jd4f1-cd91-441e-a820-498c6614126a).)
- [ ] If your PR is related to the `recipe` submodule, please also update the reference to the submodule commit via `git submodule update --remote` or `cd recipe && git pull origin main`.

@Sunwish
Sunwish merged commit 490d46d into Ascend-TechDev:lab/A2_MXFP_RL_Lab Oct 29, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant