Skip to content

[megatron] feat: weight QAT for ModelOpt MIXED_PRECISION checkpoints via the ModelOpt recipe - #10

Draft
aoshen02 wants to merge 6 commits into
upstream-main-1d367a4f4from
megatron/modelopt-mixed-qat
Draft

aoshen02 wants to merge 6 commits into
upstream-main-1d367a4f4from
megatron/modelopt-mixed-qat

Conversation

@aoshen02

@aoshen02 aoshen02 commented Oct 11, 2026 •

Copy link
Copy Markdown
Owner

What does this PR do?

Trainer side of weight QAT for a ModelOpt MIXED_PRECISION rollout checkpoint (W4A16 NVFP4 + FP8, here Nemotron-H Lightning NVFP4) on the Megatron engine, starting from BF16 masters. The quantization rule comes from ModelOpt 0.47, not from code in this repository.

  • qat.mode=modelopt_mixed: load_recipe of ModelOpt's recipe for this model (models/nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16/ptq/w4a16_nvfp4_4o6) and one mtq.quantize call on the MCore model. Weight quantizers are calibrated statically (MSE) on the BF16 masters at start-up and stay frozen. The recipe's FP8 input quantizers (Mamba in_proj / out_proj) and FP8 KV quantizers are pinned to the deployment checkpoint's static scales through the recipe field constant_amax, so no calibration data is needed.
  • Forward of a quantized weight = dequantize(exporter(w)) with a straight-through gradient, where the exporter is ModelOpt's to_quantized_weight on the frozen scales and the dequantize is NVFP4QTensor.dequantize. The trained weight is therefore the served weight.
  • mixed_precision_export=True: every full-tensor weight sync quantizes the current masters with the same function and scales and sends the static input_scale / k_scale / v_scale of the quantizers.
  • hf_weights_path: build and initialize the Megatron model from a checkpoint other than model.path (BF16 masters while model.path is the quantized rollout checkpoint).
  • Checkpointing: quantizers are rebuilt from the recipe at every start, so a checkpoint load takes _extra_state from the live model instead of the checkpoint.
  • nvidia-modelopt pinned to 0.47.0 (first release with the recipe).

ModelOpt ships this recipe as a PTQ recipe; there is no QAT recipe for the model. It is used here as the definition of the deployment format that the QAT forward has to reproduce.

What is still hand-written, and why

Piece Why not ModelOpt's own
STE forward on the exporter output ModelOpt's fake-quant kernels are not bit-identical to its exporter (FP8: 1.3-5.3% of codes on the Mamba tensors checked; NVFP4: 0.012-0.022% of elements), so training on fake-quant would train a weight that is not the served one.
FP8 weight encode E4M3(BF16(w / scale)) written out to_quantized_weight applies this rule, but on the GPU PyTorch rounds its 0-dim scale to BF16 before dividing. On the released checkpoint's 46 FP8 tensors (8.9e8 codes): the written-out rule reproduces all codes; to_quantized_weight on GPU differs in 1.72% of codes (35 of 46 tensors); on CPU it reproduces all codes.
Streaming exporter glue ModelOpt's GPTModelExporter is a one-shot whole-model export on CPU; Megatron-Bridge's ModelOpt export rejects static NVFP4 and has no FP8 path. The glue calls the same ModelOpt functions per tensor on Bridge's weight stream and gathers expert scales over the EP group once.
MCore to HF module name mapping Needed to match quantizers to the deployment's hf_quant_config.json; start-up fails if the two sets differ.

Dependencies

Known limitations

  • TP > 1 and expert TP > 1 are rejected at config time (static scales are calibrated per shard; ModelOpt itself rejects static NVFP4 with TP > 1).
  • The recipe path and the module mapping are specific to Nemotron-H Lightning with grouped-GEMM experts.
  • The quantizer forward is replaced on the instance (quantizer.forward = ...); a ModelOpt hook for a custom forward would be cleaner.
  • Each rank compiles ModelOpt's FP8 CUDA extension on first start (about 100 s).
  • The weight scales are frozen at their start-up calibration for the whole run. The long-run effect compared with recomputing scales at every sync has not been measured.
  • mixed_precision_export supports only the full-tensor sync (checkpoint_engine.backend=naive). The ref worker does not inherit hf_weights_path.

Test

No test files in this PR.

  • Start-up census on the full model matches hf_quant_config.json: 2944 + 2944 routed expert projections, 46 shared-expert projections, 1 output layer, 46 FP8 Mamba weights with 46 input quantizers, 6 + 6 KV quantizers; exported input_scale / k_scale / v_scale equal the checkpoint's bit for bit.
  • NVFP4QTensor.dequantize compared with the previous decode on 8.9e8 elements: identical.
  • 8 nodes x 4 GB200 (32 ranks, DP8 x CP4, EP4, MTP trained), SWE-bench Pro GRPO: the revision before the FP8 encode change ran sync, checkpoint save and resume in a new process; replicated scales were identical on all 32 ranks. The run is still in progress.
  • Resume-load host memory on one node (same MTP + CP4 config), trainer anonymous memory peak: 396 GB with this flow, 387 GB with the previous hand-written rule.
  • The exact revision of this branch (with the FP8 encode change), one node (4 GB200, MTP + CP4): sync, checkpoint save and resume in a new process all pass; with the forward-vs-deployed comparison enabled, the QAT forward weight equals the dequantized synced tensors on every element at every sync (routed experts 7.3e9, shared experts 4.6e8, Mamba in/out_proj 8.9e8, lm_head 3.5e8 elements; 0 differing).

Not a duplicate of verl-project#6992 (FSDP W4A8 simulation) or verl-project#7895 (MXFP4 experts): this targets ModelOpt MIXED_PRECISION checkpoints on the Megatron engine.

AI assistance was used for this change.

🤖 Generated with Claude Code

aoshen02 and others added 3 commits October 11, 2026 07:35
Train the Megatron actor on the weights a ModelOpt MIXED_PRECISION
(W4A16 NVFP4 + FP8) rollout checkpoint deploys:

- qat.mode=modelopt_mixed replaces the weight quantizers of the quantized
  deployment layers with a requantize + STE quantizer, including TE grouped
  linears (weight-only ModelOpt quant modules, registered for one
  mtq.quantize call).
- mixed_precision_export requantizes every full-tensor weight sync into the
  checkpoint format with the same rule and forwards the checkpoint's static
  activation scales.
- hf_weights_path builds and initializes the Megatron model from a
  checkpoint other than model.path (BF16 masters for a quantized rollout).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: aoshen02 <aoshen@inferact.ai>
The exporter's receiver and the rules stay in Megatron, so both arms of the
comparison share one requantize.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: aoshen02 <aoshen@inferact.ai>
aoshen02 and others added 3 commits October 11, 2026 08:31
The rollout inherits the actor's qat config; modelopt_mixed serves the
checkpoint's own ModelOpt config and has no quantization_config_path.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: aoshen02 <aoshen@inferact.ai>
The QAT forward quantizes each rank's shard and the exporter the full tensor,
so their scales differ once a weight is split across ranks.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: aoshen02 <aoshen@inferact.ai>
Replace the hand-written requantize rule with ModelOpt 0.47:
mtq.quantize with the model's recipe (static MSE weight calibration on
the BF16 masters, FP8 input / KV quantizers pinned to the deployment
checkpoint's static scales through constant_amax), forward on the
dequantized exporter output with a straight-through gradient, and
weight sync through the same exporter functions. No dependency on
megatron.lite any more.

Deployment scales are kept out of the quantizer attributes (ModelOpt
pickles those into every module's _extra_state), and a checkpoint load
takes _extra_state from the live model.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: aoshen02 <aoshen@inferact.ai>
@aoshen02 aoshen02 changed the title [megatron] feat: weight QAT for ModelOpt MIXED_PRECISION checkpoints [megatron] feat: weight QAT for ModelOpt MIXED_PRECISION checkpoints via the ModelOpt recipe Oct 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant