File tree Expand file tree Collapse file tree
Expand file tree Collapse file tree Original file line number Diff line number Diff line change @@ -62,17 +62,16 @@ The performance data includes:
6262
6363| System | #-GPUs | Precision | GBS | MBS | Sequence Length | TP | PP | CP | VP | EP | Tokens / sec / GPU | Model TFLOP / sec / GPU |
6464| --------| --------| -----------| -----| -----| -----------------| ----| ----| ----| ----| ----| -----------------------| -------------------------|
65- | DGX-GB300 | 64 | BF16 | 1280 | 4 | 4096 | 1 | 1 | 1 | n/a | 64 | 20635 | 673 |
66- | DGX-GB200 | 64 | BF16 | 1280 | 4 | 4096 | 1 | 1 | 1 | n/a | 64 | 17770 | 580 |
67- | DGX-H100 | 64 | BF16 | 1280 | 1 | 4096 | 1 | 4 | 1 | n/a | 8 | 5860 | 191 |
65+ | DGX-GB300 | 64 | MXFP8 | 1280 | 4 | 4096 | 1 | 1 | 1 | n/a | 16 | 33166 | 1081 |
66+ | DGX-GB200 | 64 | MXFP8 | 1280 | 4 | 4096 | 1 | 1 | 1 | n/a | 64 | 28947 | 943 |
6867
6968#### Model: Qwen3_30B_a3B
7069
7170| System | #-GPUs | Precision | GBS | MBS | Sequence Length | TP | PP | CP | VP | EP | Tokens / sec / GPU | Model TFLOP / sec / GPU |
7271| --------| --------| -----------| -----| -----| -----------------| ----| ----| ----| ----| ----| -----------------------| -------------------------|
7372| DGX-GB300 | 8 | MXFP8 | 512 | 8 | 4096 | 1 | 1 | 1 | n/a | 8 | 45275 | 1041 |
7473| DGX-GB200 | 8 | MXFP8 | 512 | 4 | 4096 | 1 | 1 | 1 | n/a | 8 | 40706 | 936 |
75- | DGX-H100 | 16 | FP8 | 1024 | 1 | 4096 | 1 | 1 | 1 | n/a | 16 | 8467 | 195 |
74+ | DGX-H100 | 16 | FP8 | 1024 | 1 | 4096 | 1 | 1 | 1 | n/a | 16 | 8826 | 203 |
7675
7776#### Model: Qwen3_235B_a22B
7877
You can’t perform that action at this time.
0 commit comments