Commit cf7f2be
committed
fix multi-gpu llm tests
- use MoE model (Deepseek-V2-Lite) because
vllm-project/vllm#30739 changes how vLLM handles
DP ranks - overrides dp_size=1 and dp_rank=0 if non-MoE model.
- fixes doc/source/llm/doc_code/serve/multi_gpu/dp_basic_example.py and
doc/source/llm/doc_code/serve/multi_gpu/dp_pd_example.py
Signed-off-by: Nikhil Ghosh <nikhil@anyscale.com>1 parent 85b904b commit cf7f2be
File tree
2 files changed
+3
-3
lines changed- doc/source/llm/doc_code/serve/multi_gpu
2 files changed
+3
-3
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
42 | 42 | | |
43 | 43 | | |
44 | 44 | | |
45 | | - | |
| 45 | + | |
46 | 46 | | |
47 | 47 | | |
48 | 48 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
57 | 57 | | |
58 | 58 | | |
59 | 59 | | |
60 | | - | |
| 60 | + | |
61 | 61 | | |
62 | 62 | | |
63 | 63 | | |
| |||
78 | 78 | | |
79 | 79 | | |
80 | 80 | | |
81 | | - | |
| 81 | + | |
82 | 82 | | |
83 | 83 | | |
84 | 84 | | |
| |||
0 commit comments