Skip to content

[fsdp, trainer, rollout, vllm] feat: add MXFP4 TFS and CX scale algorithms - #4

Merged
Sunwish merged 3 commits into
Ascend-TechDev:lab/A2_MXFP_RL_Labfrom
Sunwish:pr/04-mxfp4-tfs-cx
Oct 30, 2026
Merged

Sunwish merged 3 commits into
Ascend-TechDev:lab/A2_MXFP_RL_Labfrom
Sunwish:pr/04-mxfp4-tfs-cx

Conversation

@Sunwish

@Sunwish Sunwish commented Oct 29, 2026 •

Copy link
Copy Markdown
Collaborator

What does this PR do?

Adds configurable MXFP4 shared-scale algorithms for MXFP QAT: standard, tfs, and cx. The default remains standard, preserving existing behavior. The CX scale calculation is corrected to select its power-of-two scale using the effective maximum norm before quantize/dequantize.

The selected algorithm is configurable in FSDP QAT and is forwarded to vLLM-Ascend rollout fake quantization. This PR also fails early with a clear error when MXFP QAT is enabled on a non-FSDP backend, and limits PPO probe-step configuration lookup to FSDP/FSDP2 actors.

Related work: #7895 adds Megatron MXFP4 expert QAT, while this PR adds scale-algorithm choices to the FSDP MXFP QDQ path and forwards the setting to vLLM-Ascend.

Checklist Before Starting

Test

  • Added CPU tests for scale-algorithm name and numeric-alias normalization, expected scale/dequantization values, and propagation of the selected algorithm to vLLM-Ascend configuration.
  • git diff --check c3f0c8fe^..8e5cca4c passed.
  • Attempted python -m pytest tests/utils/test_mxfp_quant_cpu.py tests/utils/test_mxfp_qat_vllm_ascend.py -q; test collection could not complete because PyTorch and OmegaConf are not installed in the current environment.
  • No NPU experiment or comparison against a hardware reference is included in this commit range.
  • A2 machine actual test of RL training rewards curve under different configurations of mxfp4_scale_alg (Qwen3-30B-A3B, GSM8K):
image

API and Usage Example

Configure MXFP4 W4A8 QAT with the CX scale algorithm:

actor_rollout_ref:
  actor:
    strategy: fsdp
    fsdp_config:
      qat:
        enable: true
        mode: mxfp_w4a8
        group_size: 32
        quant_backend: torch
        mxfp4_scale_alg: cx
  rollout:
    qat_fake_quant: true

mxfp4_scale_alg accepts standard, tfs, or cx. standard is the default. The quantization helpers also accept the algorithm directly, for example quantize_mxfp4(x, scale_alg="cx").

Design & Code Changes

  1. Add MXFP4 shared-scale algorithm normalization, including the supported numeric aliases.
  2. Implement standard, TFS, and CX scale selection and apply the selected algorithm consistently during quantization and dequantization.
  3. Correct CX scale selection and add regression cases for input magnitudes 7, 8, and 15.
  4. Propagate mxfp4_scale_alg through FSDP QAT configuration, the QDQ probe, and the vLLM-Ascend environment bridge.
  5. Validate the supported MXFP QAT backend at worker initialization and guard PPO probe-step lookup by actor strategy.

Checklist Before Submitting

Important

Please check all the following items before requesting a review, otherwise the reviewer might deprioritize this PR for review.

@Sunwish
Sunwish requested a review from iceyzhao1 October 29, 2026 09:23
@Sunwish

Sunwish commented Oct 29, 2026

Copy link
Copy Markdown
Collaborator Author

vllm-ascend 关联 PR:Ascend-TechDev/vllm-ascend#5

@Sunwish
Sunwish merged commit 49f6d7c into Ascend-TechDev:lab/A2_MXFP_RL_Lab Oct 30, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants