Skip to content

Add DSV4 GB300 1k1k STP disagg configs#1530

Open
yhyang201 wants to merge 5 commits into
mainfrom
add-dsv4-gb300-1k1k-stp-disagg-configs
Open

Add DSV4 GB300 1k1k STP disagg configs#1530
yhyang201 wants to merge 5 commits into
mainfrom
add-dsv4-gb300-1k1k-stp-disagg-configs

Conversation

@yhyang201
Copy link
Copy Markdown
Collaborator

@yhyang201 yhyang201 commented May 20, 2026

Summary

Port 9 non-MTP disagg configs from NVIDIA/srt-slurm#161:

  • 1p1d dep8/dep16, 1p4d, 1p6d, 2p1d dep12/dep16/dep48
  • low-latency dep4/tp4 with zip overrides

Reference: NVIDIA/srt-slurm#161


Note

Low Risk
Adds benchmark/CI YAML and Slurm recipes only; no runtime application or auth changes.

Overview
Adds DeepSeek-V4-Pro GB300 1k/1k STP (non-MTP) disaggregated benchmark coverage ported from NVIDIA/srt-slurm#161: new Slurm recipes under benchmarks/multi_node/srt-slurm-recipes/sglang/deepseek-v4/1k1k/ and a matching CI entry dsv4-fp4-gb300-dynamo-sglang-1k1k-stp in nvidia-master.yaml.

High-concurrency Dynamo paths (1p1d dep8/dep16, 2p1d dep12/dep16/dep48) use Dynamo KV routing, megamoe MoE, Mooncake disagg, and custom benchmark_serving.py sweeps at very high concurrency (e.g. 8k–18k). Multi-decode STP paths (1p4d, 1p6d) use an SGLang frontend, DEP4 prefill + TP4 decode workers, and sa-bench at 8x64 / 32x64. Low-latency base recipes (disagg-low-latency-dep4.yaml, disagg-low-latency-tp4.yaml) add zip overrides over decode scale-out; they are included in the repo but not wired in the new master block in this diff.

Stack updates reflected in recipes/changelog: SGLang image nightly-dev-cu13-20260602-98a1b58c, deepep → megamoe on relevant roles, trimmed obsolete SGLANG_OPT_* / deepep env, and normalized Dynamo model container to dsv4-grace-blackwell. perf-changelog.yaml documents the new config key and those recipe tweaks.

Reviewed by Cursor Bugbot for commit db553d8. Bugbot is set up for automated code reviews on this repo. Configure here.

@yhyang201 yhyang201 requested a review from a team May 20, 2026 14:22
@github-actions
Copy link
Copy Markdown
Contributor

Thanks for the contribution! For vLLM & SGLang, please ensure that your recipes is similar to the official vLLM recipes and/or the SGLang cookbook

If it is not, please create a PR first before we can merge your single node PR into the master branch. Let's ensure that the documentation is first class such that the entire ML community can benefit from your hard work! Thank you

PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. If re-running failed jobs is attempted, PR authors are responsible for ensuring it passes. See GitHub's docs on re-running failed jobs: https://docs.github.com/en/actions/how-tos/manage-workflow-runs/re-run-workflows-and-jobs#re-running-failed-jobs-in-a-workflow

As a rule of thumb, generally, PR authors should request a review & get a PR approval from the respective companies' CODEOWNERS before requesting a review from core maintainers.

If additional help is needed, PR authors can reach out to core maintainers over Slack.

@github-actions
Copy link
Copy Markdown
Contributor

@github-actions
Copy link
Copy Markdown
Contributor

2 similar comments
@github-actions
Copy link
Copy Markdown
Contributor

@github-actions
Copy link
Copy Markdown
Contributor

yhyang201 added a commit that referenced this pull request May 22, 2026
@yhyang201 yhyang201 force-pushed the add-dsv4-gb300-1k1k-stp-disagg-configs branch from dbef1ac to 4f923af Compare May 22, 2026 17:35
@github-actions
Copy link
Copy Markdown
Contributor


name: "disagg-2p1d-dep12-conc18432"
slurm:
time_limit: 03:00:00
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unquoted Slurm time limit

Medium Severity

slurm.time_limit is set as an unquoted 03:00:00, while every other new dynamo recipe in this change uses a quoted "03:00:00". YAML 1.1 parsers often coerce colon-separated values to sexagesimal integers (e.g. 10800), not an HH:MM:SS string, which can yield the wrong Slurm wall-clock limit or type errors downstream.

Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit 1aaa3cc. Configure here.

@yhyang201 yhyang201 force-pushed the add-dsv4-gb300-1k1k-stp-disagg-configs branch from 1aaa3cc to e208645 Compare June 4, 2026 19:29
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Jun 4, 2026

Comment thread perf-changelog.yaml
…, switch to megamoe

Migrate the 7 STP disagg recipes to the megamoe MoE backend (deepep ->
megamoe, drop deepep-config) and strip obsolete SGLANG_OPT_*/SGLANG_DEEPEP
env vars now defaulted upstream, mirroring the b300 migration (#1506).
Clean the 5 dynamo recipes: fix container to dsv4-grace-blackwell, remove
personal extra_mount and hardcoded nodelist pins so they run on CI.
@yhyang201 yhyang201 force-pushed the add-dsv4-gb300-1k1k-stp-disagg-configs branch from e208645 to db553d8 Compare June 4, 2026 19:32
Copy link
Copy Markdown

@cursor cursor Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

There are 2 total unresolved issues (including 1 from previous review).

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit db553d8. Configure here.

prefill_workers: 1
decode_nodes: 4
decode_workers: 4

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Missing GB300 sbatch directives

Medium Severity

The 1p4d and 1p6d recipes wired into nvidia-master.yaml omit sbatch_directives (cpus-per-task: 144, mem: 0) that other GB300 multinode DSV4 recipes include. On gb300-cw, Slurm may default to one CPU per task and tight memory, risking slow or failed runs.

Additional Locations (1)
Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit db553d8. Configure here.

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Jun 4, 2026

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

Status: No status

Development

Successfully merging this pull request may close these issues.

1 participant