[None][test] Add deepseek v4 pro cases on the qa side#16611
[None][test] Add deepseek v4 pro cases on the qa side#16611fredricz-20070104 wants to merge 2 commits into
Conversation
Signed-off-by: xinhe-nv <200704525+xinhe-nv@users.noreply.github.com>
|
/bot skip --comment "Only add deepseek cases for the qa side" |
📝 WalkthroughWalkthroughAdded GB300 DeepSeek-V4-Pro disaggregated benchmark configurations, fixed MoE load-balancer fixtures, and four QA performance test entries for disaggregated and multinode runs. ChangesDeepSeek-V4-Pro performance coverage
Estimated code review effort: 3 (Moderate) | ~20 minutes Possibly related PRs
Suggested reviewers: 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Warning Review ran into problems🔥 ProblemsGit: Failed to clone repository. Please run the Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 3
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In
`@tests/scripts/perf/disaggregated/deepseek-v4-pro-eplb/moe_load_balancer_ctx_ep4_384.yaml`:
- Line 1: Add the repository-standard NVIDIA copyright header at the beginning
of each affected YAML file, before its existing top-level key:
tests/scripts/perf/disaggregated/deepseek-v4-pro-eplb/moe_load_balancer_ctx_ep4_384.yaml
(lines 1-1),
tests/scripts/perf/disaggregated/deepseek-v4-pro-eplb/moe_load_balancer_gen_ep16_slots384.yaml
(lines 1-1),
tests/scripts/perf/disaggregated/deepseek-v4-pro-eplb/moe_load_balancer_gen_ep16_slots416.yaml
(lines 1-1),
tests/scripts/perf/disaggregated/deepseek-v4-pro-eplb/moe_load_balancer_gen_ep32_slots384.yaml
(lines 1-1),
tests/scripts/perf/disaggregated/gb300_deepseek-v4-pro-fp4_8k1k_con666_ctx6_dep4_gen1_dep16_eplb384_mtp3_ccb-NIXL.yaml
(lines 1-1), and
tests/scripts/perf/disaggregated/gb300_deepseek-v4-pro-fp4_8k1k_con8_ctx1_dep4_gen4_tep8_eplb0_mtp3_ccb-NIXL.yaml
(lines 1-1).
In
`@tests/scripts/perf/disaggregated/deepseek-v4-pro-eplb/moe_load_balancer_gen_ep32_slots416.yaml`:
- Line 1: Prepend the standard NVIDIA copyright header to each affected YAML
file:
tests/scripts/perf/disaggregated/deepseek-v4-pro-eplb/moe_load_balancer_gen_ep32_slots416.yaml,
moe_load_balancer_gen_ep4_slots384.yaml,
moe_load_balancer_gen_ep4_slots416.yaml,
moe_load_balancer_gen_ep8_slots384.yaml,
moe_load_balancer_gen_ep8_slots416.yaml, and the two gb300_deepseek-v4-pro YAML
files listed in the review. Keep the existing initial_global_assignments content
unchanged after the header.
In
`@tests/scripts/perf/disaggregated/gb300_deepseek-v4-pro-fp4_8k1k_con180_ctx3_dep4_gen1_dep32_eplb384_mtp3_ccb-NIXL.yaml`:
- Around line 17-27: Replace the <dataset_file> placeholder with a concrete,
resolvable dataset path in the benchmark configuration at
tests/scripts/perf/disaggregated/gb300_deepseek-v4-pro-fp4_8k1k_con180_ctx3_dep4_gen1_dep32_eplb384_mtp3_ccb-NIXL.yaml
lines 17-27 and the sibling configuration at
tests/scripts/perf/disaggregated/gb300_deepseek-v4-pro-fp4_8k1k_con4301_ctx12_dep4_gen1_dep8_eplb384_mtp1_ccb-NIXL.yaml
lines 17-27. Keep the existing benchmark settings unchanged.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Enterprise
Run ID: 825e6e97-6979-48d3-b1d0-4f01cf913555
📒 Files selected for processing (15)
tests/integration/test_lists/qa/llm_perf_disagg.ymltests/integration/test_lists/qa/llm_perf_multinode.txttests/scripts/perf/disaggregated/deepseek-v4-pro-eplb/moe_load_balancer_ctx_ep4_384.yamltests/scripts/perf/disaggregated/deepseek-v4-pro-eplb/moe_load_balancer_gen_ep16_slots384.yamltests/scripts/perf/disaggregated/deepseek-v4-pro-eplb/moe_load_balancer_gen_ep16_slots416.yamltests/scripts/perf/disaggregated/deepseek-v4-pro-eplb/moe_load_balancer_gen_ep32_slots384.yamltests/scripts/perf/disaggregated/deepseek-v4-pro-eplb/moe_load_balancer_gen_ep32_slots416.yamltests/scripts/perf/disaggregated/deepseek-v4-pro-eplb/moe_load_balancer_gen_ep4_slots384.yamltests/scripts/perf/disaggregated/deepseek-v4-pro-eplb/moe_load_balancer_gen_ep4_slots416.yamltests/scripts/perf/disaggregated/deepseek-v4-pro-eplb/moe_load_balancer_gen_ep8_slots384.yamltests/scripts/perf/disaggregated/deepseek-v4-pro-eplb/moe_load_balancer_gen_ep8_slots416.yamltests/scripts/perf/disaggregated/gb300_deepseek-v4-pro-fp4_8k1k_con180_ctx3_dep4_gen1_dep32_eplb384_mtp3_ccb-NIXL.yamltests/scripts/perf/disaggregated/gb300_deepseek-v4-pro-fp4_8k1k_con4301_ctx12_dep4_gen1_dep8_eplb384_mtp1_ccb-NIXL.yamltests/scripts/perf/disaggregated/gb300_deepseek-v4-pro-fp4_8k1k_con666_ctx6_dep4_gen1_dep16_eplb384_mtp3_ccb-NIXL.yamltests/scripts/perf/disaggregated/gb300_deepseek-v4-pro-fp4_8k1k_con8_ctx1_dep4_gen4_tep8_eplb0_mtp3_ccb-NIXL.yaml
| @@ -0,0 +1,65 @@ | |||
| initial_global_assignments: | |||
There was a problem hiding this comment.
📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win
Add the required NVIDIA copyright header to all new YAML files.
tests/scripts/perf/disaggregated/deepseek-v4-pro-eplb/moe_load_balancer_ctx_ep4_384.yaml#L1-L1: add the header beforeinitial_global_assignments.tests/scripts/perf/disaggregated/deepseek-v4-pro-eplb/moe_load_balancer_gen_ep16_slots384.yaml#L1-L1: add the header beforeinitial_global_assignments.tests/scripts/perf/disaggregated/deepseek-v4-pro-eplb/moe_load_balancer_gen_ep16_slots416.yaml#L1-L1: add the header beforeinitial_global_assignments.tests/scripts/perf/disaggregated/deepseek-v4-pro-eplb/moe_load_balancer_gen_ep32_slots384.yaml#L1-L1: add the header beforeinitial_global_assignments.tests/scripts/perf/disaggregated/gb300_deepseek-v4-pro-fp4_8k1k_con666_ctx6_dep4_gen1_dep16_eplb384_mtp3_ccb-NIXL.yaml#L1-L1: add the header beforemetadata.tests/scripts/perf/disaggregated/gb300_deepseek-v4-pro-fp4_8k1k_con8_ctx1_dep4_gen4_tep8_eplb0_mtp3_ccb-NIXL.yaml#L1-L1: add the header beforemetadata.
As per coding guidelines, “Add the NVIDIA copyright header to all new files.”
📍 Affects 6 files
tests/scripts/perf/disaggregated/deepseek-v4-pro-eplb/moe_load_balancer_ctx_ep4_384.yaml#L1-L1(this comment)tests/scripts/perf/disaggregated/deepseek-v4-pro-eplb/moe_load_balancer_gen_ep16_slots384.yaml#L1-L1tests/scripts/perf/disaggregated/deepseek-v4-pro-eplb/moe_load_balancer_gen_ep16_slots416.yaml#L1-L1tests/scripts/perf/disaggregated/deepseek-v4-pro-eplb/moe_load_balancer_gen_ep32_slots384.yaml#L1-L1tests/scripts/perf/disaggregated/gb300_deepseek-v4-pro-fp4_8k1k_con666_ctx6_dep4_gen1_dep16_eplb384_mtp3_ccb-NIXL.yaml#L1-L1tests/scripts/perf/disaggregated/gb300_deepseek-v4-pro-fp4_8k1k_con8_ctx1_dep4_gen4_tep8_eplb0_mtp3_ccb-NIXL.yaml#L1-L1
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In
`@tests/scripts/perf/disaggregated/deepseek-v4-pro-eplb/moe_load_balancer_ctx_ep4_384.yaml`
at line 1, Add the repository-standard NVIDIA copyright header at the beginning
of each affected YAML file, before its existing top-level key:
tests/scripts/perf/disaggregated/deepseek-v4-pro-eplb/moe_load_balancer_ctx_ep4_384.yaml
(lines 1-1),
tests/scripts/perf/disaggregated/deepseek-v4-pro-eplb/moe_load_balancer_gen_ep16_slots384.yaml
(lines 1-1),
tests/scripts/perf/disaggregated/deepseek-v4-pro-eplb/moe_load_balancer_gen_ep16_slots416.yaml
(lines 1-1),
tests/scripts/perf/disaggregated/deepseek-v4-pro-eplb/moe_load_balancer_gen_ep32_slots384.yaml
(lines 1-1),
tests/scripts/perf/disaggregated/gb300_deepseek-v4-pro-fp4_8k1k_con666_ctx6_dep4_gen1_dep16_eplb384_mtp3_ccb-NIXL.yaml
(lines 1-1), and
tests/scripts/perf/disaggregated/gb300_deepseek-v4-pro-fp4_8k1k_con8_ctx1_dep4_gen4_tep8_eplb0_mtp3_ccb-NIXL.yaml
(lines 1-1).
Source: Coding guidelines
| @@ -0,0 +1,65 @@ | |||
| initial_global_assignments: | |||
There was a problem hiding this comment.
📐 Maintainability & Code Quality | 🟠 Major | ⚡ Quick win
Add the NVIDIA copyright header to every new YAML file.
tests/scripts/perf/disaggregated/deepseek-v4-pro-eplb/moe_load_balancer_gen_ep32_slots416.yamlLines 1-1: add the standard header.tests/scripts/perf/disaggregated/deepseek-v4-pro-eplb/moe_load_balancer_gen_ep4_slots384.yamlLines 1-1: add the standard header.tests/scripts/perf/disaggregated/deepseek-v4-pro-eplb/moe_load_balancer_gen_ep4_slots416.yamlLines 1-1: add the standard header.tests/scripts/perf/disaggregated/deepseek-v4-pro-eplb/moe_load_balancer_gen_ep8_slots384.yamlLines 1-1: add the standard header.tests/scripts/perf/disaggregated/deepseek-v4-pro-eplb/moe_load_balancer_gen_ep8_slots416.yamlLines 1-1: add the standard header.tests/scripts/perf/disaggregated/gb300_deepseek-v4-pro-fp4_8k1k_con180_ctx3_dep4_gen1_dep32_eplb384_mtp3_ccb-NIXL.yamlLines 1-1: add the standard header.tests/scripts/perf/disaggregated/gb300_deepseek-v4-pro-fp4_8k1k_con4301_ctx12_dep4_gen1_dep8_eplb384_mtp1_ccb-NIXL.yamlLines 1-1: add the standard header.
As per coding guidelines, all new files must include the NVIDIA copyright header.
📍 Affects 7 files
tests/scripts/perf/disaggregated/deepseek-v4-pro-eplb/moe_load_balancer_gen_ep32_slots416.yaml#L1-L1(this comment)tests/scripts/perf/disaggregated/deepseek-v4-pro-eplb/moe_load_balancer_gen_ep4_slots384.yaml#L1-L1tests/scripts/perf/disaggregated/deepseek-v4-pro-eplb/moe_load_balancer_gen_ep4_slots416.yaml#L1-L1tests/scripts/perf/disaggregated/deepseek-v4-pro-eplb/moe_load_balancer_gen_ep8_slots384.yaml#L1-L1tests/scripts/perf/disaggregated/deepseek-v4-pro-eplb/moe_load_balancer_gen_ep8_slots416.yaml#L1-L1tests/scripts/perf/disaggregated/gb300_deepseek-v4-pro-fp4_8k1k_con180_ctx3_dep4_gen1_dep32_eplb384_mtp3_ccb-NIXL.yaml#L1-L1tests/scripts/perf/disaggregated/gb300_deepseek-v4-pro-fp4_8k1k_con4301_ctx12_dep4_gen1_dep8_eplb384_mtp1_ccb-NIXL.yaml#L1-L1
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In
`@tests/scripts/perf/disaggregated/deepseek-v4-pro-eplb/moe_load_balancer_gen_ep32_slots416.yaml`
at line 1, Prepend the standard NVIDIA copyright header to each affected YAML
file:
tests/scripts/perf/disaggregated/deepseek-v4-pro-eplb/moe_load_balancer_gen_ep32_slots416.yaml,
moe_load_balancer_gen_ep4_slots384.yaml,
moe_load_balancer_gen_ep4_slots416.yaml,
moe_load_balancer_gen_ep8_slots384.yaml,
moe_load_balancer_gen_ep8_slots416.yaml, and the two gb300_deepseek-v4-pro YAML
files listed in the review. Keep the existing initial_global_assignments content
unchanged after the header.
Source: Coding guidelines
| benchmark: | ||
| mode: e2e | ||
| use_nv_sa_benchmark: false | ||
| multi_round: 10 | ||
| benchmark_ratio: 0.0 | ||
| streaming: true | ||
| concurrency_list: '180' | ||
| input_length: 7408 | ||
| output_length: 922 | ||
| # Random synthetic dataset — see the con8 variant for rationale. | ||
| dataset_file: <dataset_file> |
There was a problem hiding this comment.
🩺 Stability & Availability | 🟠 Major | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
printf '\n== File list ==\n'
git ls-files 'tests/scripts/perf/disaggregated/*gb300_deepseek-v4-pro-fp4*' | sed 's#^`#-` #'
printf '\n== Search for dataset_file / use_nv_sa_benchmark / placeholder token ==\n'
rg -n --hidden --no-ignore-vcs -S 'dataset_file|use_nv_sa_benchmark|<dataset_file>|nv_sa_benchmark|synthetic requests' tests/scripts/perf tests | sed -n '1,220p'
printf '\n== Relevant YAML snippets ==\n'
for f in \
tests/scripts/perf/disaggregated/gb300_deepseek-v4-pro-fp4_8k1k_con180_ctx3_dep4_gen1_dep32_eplb384_mtp3_ccb-NIXL.yaml \
tests/scripts/perf/disaggregated/gb300_deepseek-v4-pro-fp4_8k1k_con4301_ctx12_dep4_gen1_dep8_eplb384_mtp1_ccb-NIXL.yaml
do
echo "--- $f"
cat -n "$f" | sed -n '1,120p'
done
printf '\n== Candidate scripts and templates under tests/scripts/perf ==\n'
fd -t f -a . tests/scripts/perf | sed -n '1,220p'Repository: NVIDIA/TensorRT-LLM
Length of output: 1996
🏁 Script executed:
#!/bin/bash
set -euo pipefail
pwd
git ls-files 'tests/scripts/perf/disaggregated/*gb300_deepseek-v4-pro-fp4*'Repository: NVIDIA/TensorRT-LLM
Length of output: 1996
🏁 Script executed:
#!/bin/bash
set -euo pipefail
git ls-files 'tests/scripts/perf/disaggregated/*gb300_deepseek-v4-pro-fp4*' | sed -n '1,50p'Repository: NVIDIA/TensorRT-LLM
Length of output: 1996
🌐 Web query:
"tests/scripts/perf/disaggregated/gb300_deepseek-v4-pro-fp4_8k1k_con180_ctx3_dep4_gen1_dep32_eplb384_mtp3_ccb-NIXL.yaml dataset_file use_nv_sa_benchmark"
💡 Result:
The file tests/scripts/perf/disaggregated/gb300_deepseek-v4-pro-fp4_8k1k_con180_ctx3_dep4_gen1_dep32_eplb384_mtp3_ccb-NIXL.yaml is a performance benchmark configuration file used within NVIDIA's srt-slurm framework for evaluating the DeepSeek-V4-Pro model on GB300 systems [1][2]. Regarding the parameters specified in your query: 1. dataset_file: This is a standard configuration field in srt-slurm and TensorRT-LLM benchmark YAML files, serving as a placeholder (<dataset_file>) for the path to the input dataset used during the performance profiling [3][4][5]. It allows users to point to custom workloads for end-to-end (e2e) or generation-only testing [5]. 2. use_nv_sa_benchmark: This flag is used in srt-slurm and TensorRT-LLM to toggle the use of the official NVIDIA "SA" (likely referring to the internal performance benchmark suite or the "SemiAnalysis" benchmark toolchain used in specific performance sweeps) [1][6]. When set to true, the test harness invokes the specialized bench_serving.py script associated with the SA benchmark suite instead of the default benchmark command [6]. The specific YAML file you referenced is part of a series of disaggregated serving recipes for DeepSeek-V4-Pro on GB300 [1][2]. These configurations typically utilize the Dynamo frontend, Mooncake/NIXL for KV transfer, and specific expert-parallel (EP) and decode topologies (e.g., dep32, mtp3) designed to maximize throughput on GB300's architecture [1][7][8]. The "NIXL" suffix in the filename indicates it is configured to use the Nixl side-channel backend for disaggregated KV transfer [7][9].
Citations:
- 1: Add DeepSeek-V4-Pro 8k/1k SA recipes for GB300 (MTP-off + MTP-on, MXFP4) srt-slurm#192
- 2: NVIDIA/srt-slurm@c1fb698
- 3: https://github.com/NVIDIA/TensorRT-LLM/blob/main/examples/wide_ep/slurm_scripts/kimi-k2-thinking.yaml
- 4: https://github.com/NVIDIA/TensorRT-LLM/blob/7a8bd87f/examples/disaggregated/slurm/benchmark/config.yaml
- 5: https://github.com/NVIDIA/TensorRT-LLM/tree/main/examples/disaggregated/slurm/benchmark
- 6: [None][infra] Support nv sa benchmark in CI Perf Test #13004
- 7: Day 0 DeepSeek V4 Pro FP4 GB300 disaggregated SGLang benchmarks SemiAnalysisAI/InferenceX#1157
- 8: Add DSV4 GB300 wide-EP sweep configs (EP=12/16/24/32/40) SemiAnalysisAI/InferenceX#1586
- 9: https://github.com/vllm-project/vllm-project.github.io/blob/main/_posts/2026-02-13-gb300-deepseek.md
🌐 Web query:
"\"gb300_deepseek-v4-pro-fp4_8k1k_con180_ctx3_dep4_gen1_dep32_eplb384_mtp3_ccb-NIXL.yaml\""
💡 Result:
The filename gb300_deepseek-v4-pro-fp4_8k1k_con180_ctx3_dep4_gen1_dep32_eplb384_mtp3_ccb-NIXL.yaml refers to a performance benchmarking configuration file used for evaluating the DeepSeek-V4-Pro model on NVIDIA GB300 hardware [1]. These files are typically part of reproduction sweeps and performance testing suites (such as srt-slurm or TensorRT-LLM integration tests) used to optimize model serving in disaggregated architectures [1][2][3]. The naming convention follows a structured schema that encodes specific deployment and hardware parameters: - gb300: Indicates the target hardware platform (NVIDIA GB300) [1][4]. - deepseek-v4-pro: The model being benchmarked [1][5]. - fp4: The quantization precision used (MXFP4, often verified from checkpoint tensor dtypes) [1][6]. - 8k1k: Represents the input/output sequence length configuration (e.g., 8192 input tokens, 1024 output tokens) [1][7]. - con180: Likely refers to a specific concurrency level (number of requests) being tested [1]. - ctx3: Indicates the context (prefill) parallelism or pipeline configuration [1][2]. - dep4: Disaggregated expert parallelism or related parallel strategy parameter [1][3]. - gen1: Generator (decode) parallelism setting [1][2]. - dep32: Likely refers to an expert-parallelism or data-parallelism setting (e.g., Expert Parallelism size 32) [1][4]. - eplb384: Expert-parallel load-balancing or a similar resource-allocation parameter [1][3]. - mtp3: Indicates the Multi-Token Prediction (MTP) mode or depth (in this case, 3) [1][2]. - ccb-NIXL: Refers to the communication backend or connector type, in this case using the NIXL (NVIDIA Inference cross-Link) connector for disaggregated KV cache transfer [1][8]. These configurations are utilized within Slurm-managed clusters to generate pareto-optimal performance curves for model serving, often involving complex disaggregated setups where prefill and decode tasks are handled by separate nodes [1][7][9].
Citations:
- 1: Add DeepSeek-V4-Pro 8k/1k SA recipes for GB300 (MTP-off + MTP-on, MXFP4) srt-slurm#192
- 2: [TRTLLM-9965][test] add long-context disagg test for GB300/GB200 and remove config_index in yaml #10225
- 3: [None][test] local wheel installation support and add gb300 cases demo #11742
- 4: https://github.com/NVIDIA/dgxc-benchmarking/blob/main/deepseek_v3/pretrain/megatron_bridge/README.md
- 5: https://github.com/vllm-project/recipes/blob/main/models/deepseek-ai/DeepSeek-V4-Pro.yaml
- 6: https://huggingface.co/nvidia/DeepSeek-V4-Pro-NVFP4
- 7: https://pytorch.org/blog/serving-deepseek-v4-on-gb300-with-sglang-5x-higher-throughput-at-the-same-interactivity-since-day-0/
- 8: https://vllm.ai/blog/2026-02-13-gb300-deepseek
- 9: Instructions & Recipes for DeepSeek NVFP4 on GB300 sgl-project/sglang#18703
🌐 Web query:
"\"<dataset_file>\" benchmark launcher dataset_file resolved runtime use_nv_sa_benchmark false srt-slurm"
💡 Result:
The expression "<dataset_file>" appearing in your query is a placeholder value commonly used in YAML configuration files for the NVIDIA TensorRT-LLM SLURM benchmark suite [1][2][3]. When running these benchmarks, this placeholder must be replaced with the actual path to your specific dataset file to be used during the workload execution [1][4][5]. In the context of the srt-slurm framework (a tool for benchmarking LLMs on SLURM-managed clusters), configurations typically utilize specific keys in a YAML file to define how benchmarks are run [6][7]. The setting use_nv_sa_benchmark: false indicates that the benchmark suite is configured to bypass the legacy "NVIDIA SA benchmark script," allowing it to use alternative benchmark runners (such as custom scripts, sa-bench, or sglang-bench) defined within the srt-slurm workflow [2][3][6]. To resolve the runtime configuration: 1. Locate the configuration file (often named config.yaml or similar) where dataset_file: "<dataset_file>" is defined [1][2][7]. 2. Replace "<dataset_file>" with the absolute or relative filesystem path to your workload's dataset file [4][5]. 3. Ensure that the benchmark.type specified in your srt-slurm configuration matches your intended benchmark runner (e.g., sa-bench for standard throughput/latency serving) [6][7]. 4. Utilize srtctl dry-run -f <config.yaml> to validate your configuration before submitting the job to SLURM, ensuring that all paths and parameters are correctly resolved [8][9][10]. For official benchmark recipes and validated configurations, refer to the NVIDIA/srt-slurm-recipes repository, which serves as the source of truth for standard configurations across different models and GPU platforms [9].
Citations:
- 1: https://github.com/NVIDIA/TensorRT-LLM/tree/main/examples/disaggregated/slurm/benchmark
- 2: https://github.com/NVIDIA/TensorRT-LLM/blob/7a8bd87f/examples/disaggregated/slurm/benchmark/config.yaml
- 3: https://github.com/NVIDIA/TensorRT-LLM/blob/main/examples/wide_ep/slurm_scripts/kimi-k2-thinking.yaml
- 4: https://github.com/NVIDIA/TensorRT-LLM/blob/7a8bd87f/examples/disaggregated/slurm/benchmark/run_benchmark.sh
- 5: https://github.com/NVIDIA/TensorRT-LLM/blob/7a8bd87f/examples/disaggregated/slurm/benchmark/run_benchmark_aiperf.sh
- 6: https://github.com/NVIDIA/srt-slurm/blob/main/docs/config-reference.md
- 7: https://github.com/NVIDIA/srt-slurm/blob/main/docs/installation.md
- 8: https://github.com/ishandhanani/srt-slurm/blob/main/docs/cli.md
- 9: https://github.com/NVIDIA/srt-slurm-recipes
- 10: https://github.com/ishandhanani/srt-slurm/blob/main/docs/sweeps.md
Replace <dataset_file> with a concrete dataset path in both benchmark configs. use_nv_sa_benchmark: false still requires a resolvable dataset file, so the placeholder can abort benchmark startup.
📍 Affects 2 files
tests/scripts/perf/disaggregated/gb300_deepseek-v4-pro-fp4_8k1k_con180_ctx3_dep4_gen1_dep32_eplb384_mtp3_ccb-NIXL.yaml#L17-L27(this comment)tests/scripts/perf/disaggregated/gb300_deepseek-v4-pro-fp4_8k1k_con4301_ctx12_dep4_gen1_dep8_eplb384_mtp1_ccb-NIXL.yaml#L17-L27
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In
`@tests/scripts/perf/disaggregated/gb300_deepseek-v4-pro-fp4_8k1k_con180_ctx3_dep4_gen1_dep32_eplb384_mtp3_ccb-NIXL.yaml`
around lines 17 - 27, Replace the <dataset_file> placeholder with a concrete,
resolvable dataset path in the benchmark configuration at
tests/scripts/perf/disaggregated/gb300_deepseek-v4-pro-fp4_8k1k_con180_ctx3_dep4_gen1_dep32_eplb384_mtp3_ccb-NIXL.yaml
lines 17-27 and the sibling configuration at
tests/scripts/perf/disaggregated/gb300_deepseek-v4-pro-fp4_8k1k_con4301_ctx12_dep4_gen1_dep8_eplb384_mtp1_ccb-NIXL.yaml
lines 17-27. Keep the existing benchmark settings unchanged.
Source: Learnings
|
PR_Github #60338 [ skip ] triggered by Bot. Commit: |
|
PR_Github #60338 [ skip ] completed with state |
5 participants
Summary by CodeRabbit
New Performance Coverage
Tests
Add deepseek v4 pro cases on the qa side.