Skip to content

feat(recipe): DSV3 GB200 MXFP8 full-iter CG recipe #4226

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
dingqingy-nv merged 2 commits into from
Jun 10, 2026
Merged

feat(recipe): DSV3 GB200 MXFP8 full-iter CG recipe #4226

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
f106bf4
feat(recipe): DSV3 GB200 MXFP8 full-iter CG recipe
dingqingy-nv Jun 9, 2026
dece659
fix(recipe): wire GB200 FP8_MX_V2 to inherit FP8_MX_V1
dingqingy-nv Jun 9, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions scripts/performance/configs/deepseek/deepseek_llm_pretrain.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -118,6 +118,9 @@ def deepseek_v3_pretrain_config_gb200(
cfg = pretrain_config()
cfg.mixed_precision = precision_config

if cfg.mixed_precision.fp8_recipe == "mxfp8":
cfg.model.fp8_output_proj = True

# Apply model-specific settings that were previously passed as constructor args
cfg.model.pipeline_model_parallel_size = base_cfg.pipeline_model_parallel_size
cfg.model.virtual_pipeline_model_parallel_size = base_cfg.virtual_pipeline_model_parallel_size
Expand Down
14 changes: 12 additions & 2 deletions scripts/performance/configs/deepseek/deepseek_workload_base_configs.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -95,7 +95,14 @@
recompute_modules=["mla_up_proj"],
)
DEEPSEEK_V3_PRETRAIN_CONFIG_GB200_FP8_CS_V1 = DEEPSEEK_V3_PRETRAIN_CONFIG_GB200_V1
DEEPSEEK_V3_PRETRAIN_CONFIG_GB200_FP8_MX_V1 = DEEPSEEK_V3_PRETRAIN_CONFIG_GB200_V1
DEEPSEEK_V3_PRETRAIN_CONFIG_GB200_FP8_MX_V1 = replace(
DEEPSEEK_V3_PRETRAIN_CONFIG_GB200_V1,
cuda_graph_impl="full_iteration",
cuda_graph_scope=[],
moe_a2a_overlap=True,
cutedsl_fused_grouped_mlp=True,
recompute_modules=["mla_up_proj"],
)
Comment on lines +98 to +105

@claude claude Bot Jun 9, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Question: GB300 MX V1 sets fp8_dot_product_attention=True (line 74) but the new GB200 MX V1 does not. The PR description says "Recipe shape matches GB300 except for recompute_modules=["mla_up_proj"]" — is the omission of fp8_dot_product_attention intentional (e.g. incompatible with GB200's PP=4/VP=4 layout), or should it be added here?

DEEPSEEK_V3_PRETRAIN_CONFIG_GB200_NVFP4_V1 = DEEPSEEK_V3_PRETRAIN_CONFIG_GB200_V1


Expand Down Expand Up @@ -199,7 +206,10 @@
global_batch_size=4096,
)
DEEPSEEK_V3_PRETRAIN_CONFIG_GB200_FP8_CS_V2 = DEEPSEEK_V3_PRETRAIN_CONFIG_GB200_V2
DEEPSEEK_V3_PRETRAIN_CONFIG_GB200_FP8_MX_V2 = DEEPSEEK_V3_PRETRAIN_CONFIG_GB200_V2
DEEPSEEK_V3_PRETRAIN_CONFIG_GB200_FP8_MX_V2 = replace(
DEEPSEEK_V3_PRETRAIN_CONFIG_GB200_FP8_MX_V1,
global_batch_size=4096,
)
DEEPSEEK_V3_PRETRAIN_CONFIG_GB200_NVFP4_V2 = DEEPSEEK_V3_PRETRAIN_CONFIG_GB200_V2


Expand Down
Loading