feat(recipe): DSV3 GB200 MXFP8 full-iter CG recipe

[dingqingy-nv]

Summary

Mirrors the GB300 MXFP8 full-iter CG recipe (PR #3983) on GB200. Recipe shape matches GB300 except for recompute_modules=["mla_up_proj"] to fit GB200's smaller HBM budget; GB300 can run with recompute_modules=[].

Changes

scripts/performance/configs/deepseek/deepseek_workload_base_configs.py

• DEEPSEEK_V3_PRETRAIN_CONFIG_GB200_FP8_MX_V1 no longer aliases the bf16 V1 config. It now enables:
  • cuda_graph_impl=\"full_iteration\", cuda_graph_scope=[]
  • moe_a2a_overlap=True
  • cutedsl_fused_grouped_mlp=True
  • recompute_modules=[\"mla_up_proj\"]
• V2 (GBS=4096) and VR200 mxfp8 V1/V2 inherit via the existing replace(..., global_batch_size=4096) / alias chain.

scripts/performance/configs/deepseek/deepseek_llm_pretrain.py

• Adds the fp8_output_proj=True gate for the GB200 mxfp8 recipe, mirroring the GB300 gate that landed via PR [perf, recipe] feat: DSV3 GB300 MXFP8 full-iter CG recipe #3983.

Measured impact

64 GB200 nodes / 256 GPUs, DSv3-671B mxfp8, GBS=4096 (V2). Steady-state iters 14-19, averaged.

variant	iter (s)	TF/s/GPU
partial CG + `[core_attn]` offload + `[mlp]` recompute	13.89	1226
full CG + `[core_attn,attn_proj]` offload + no recompute	15.10	1128
full CG + no offload + `[mla_up_proj]` recompute (this PR)	12.94	1316

The third row is what this recipe ships: +7.3% throughput vs the prior partial-CG baseline, and a close match to the MLPerf reference config (~14 s/iter on similar parallelism shape).

Test plan

• Empirical perf measured at 64 GB200 nodes, 256 GPUs (see iter-time / TF/s/GPU table above)
• Loss curve healthy through iter 20 (iter 20 lm loss 8.149, mtp_1 0.081, grad norm 0.391)
• `ruff check` and `ruff format --check` clean on both files
• NeMo CI L0 (will run on push)

[dingqingy-nv]

/claude review

[claude]

Question: GB300 MX V1 sets fp8_dot_product_attention=True (line 74) but the new GB200 MX V1 does not. The PR description says "Recipe shape matches GB300 except for recompute_modules=["mla_up_proj"]" — is the omission of fp8_dot_product_attention intentional (e.g. incompatible with GB200's PP=4/VP=4 layout), or should it be added here?

[claude]

Light Code Review

Clean, well-scoped change. The GB200 MX V1 config correctly mirrors the GB300 full-iter CG recipe shape with the expected recompute_modules adjustment for GB200 smaller HBM, and V2 properly inherits from V1 with only global_batch_size=4096 changed. The fp8_output_proj gate in the GB200 config function mirrors the existing GB300 gate.

Questions

1. Missing fp8_dot_product_attention=True vs GB300 (inline comment posted) - DEEPSEEK_V3_PRETRAIN_CONFIG_GB300_FP8_MX_V1 sets fp8_dot_product_attention=True (line 74) but the new GB200_FP8_MX_V1 does not. The PR description states the recipe matches GB300 except for recompute_modules. Is this omission intentional?

2. VR200 missing fp8_output_proj gate (pre-existing, but now more impactful) - DEEPSEEK_V3_PRETRAIN_CONFIG_VR200_FP8_MX_V2 (line 218) aliases GB200_FP8_MX_V2, which now carries full-iter CG + mxfp8 settings. However, deepseek_v3_pretrain_config_vr200() (line 144) lacks the fp8_output_proj=True gate that both GB300 (line 81) and GB200 (line 121) have. If fp8_output_proj is needed for correctness with mxfp8, VR200 MX V2 would be missing it. This is pre-existing but worth noting since VR200 MX V2 now actually uses the mxfp8-tuned config rather than a plain alias.

Suggested test cases

No perf tests impacted. The changed configs (DEEPSEEK_V3_PRETRAIN_CONFIG_GB200_FP8_MX_V1, V2, and transitive VR200_FP8_MX_V2) are 256-GPU / 128-GPU scale perf recipes with no corresponding L0/L1/L2 functional test launch scripts. The existing test_deepseek_v3_perf_config_instantiation unit test only covers H100 BF16. Consider extending it to cover GB200 mxfp8 instantiation to catch config wiring issues in CI.