Skip to content

Commit 81e3def

Browse files
Fix vllm tests
Signed-off-by: Onur Yilmaz <oyilmaz@nvidia.com>
1 parent a6016be commit 81e3def

2 files changed

Lines changed: 0 additions & 6 deletions

File tree

docs/llm/nemo_2/optimized/vllm.md

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -81,7 +81,6 @@ After executing the script, it will export the model to vLLM and then initiate t
8181
- ``--swap_space``: Size (GiB) of CPU memory per GPU to use as swap space. Default is 4.
8282
- ``--cpu_offload_gb``: Size (GiB) of CPU memory to use for offloading model weights. Default is 0.
8383
- ``--enforce_eager``: Whether to enforce eager execution. Default is False.
84-
- ``--max_seq_len_to_capture``: Maximum sequence length covered by CUDA graphs. Default is 8192.
8584
- ``--triton_model_name``: Name for the service/model on Triton. (Required)
8685
- ``--triton_model_version``: Version for the service/model. Default is 1.
8786
- ``--triton_port``: Port for the Triton server to listen for requests. Default is 8000.

tests/unit_tests/export/test_vllm_exporter.py

Lines changed: 0 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -70,7 +70,6 @@ def test_export(exporter, mock_llm):
7070
swap_space=4,
7171
cpu_offload_gb=0,
7272
enforce_eager=False,
73-
max_seq_len_to_capture=8192,
7473
task="auto",
7574
)
7675

@@ -96,7 +95,6 @@ def test_export_with_lora(exporter, mock_llm):
9695
swap_space=4,
9796
cpu_offload_gb=0,
9897
enforce_eager=False,
99-
max_seq_len_to_capture=8192,
10098
task="auto",
10199
)
102100

@@ -128,7 +126,6 @@ def test_export_with_custom_params(exporter, mock_llm):
128126
swap_space=4,
129127
cpu_offload_gb=0,
130128
enforce_eager=False,
131-
max_seq_len_to_capture=8192,
132129
task="auto",
133130
)
134131

@@ -844,7 +841,6 @@ def test_export_megatron_bridge_with_all_vllm_params(exporter, mock_llm):
844841
swap_space=8,
845842
cpu_offload_gb=2,
846843
enforce_eager=True,
847-
max_seq_len_to_capture=4096,
848844
task="generate",
849845
)
850846

@@ -863,5 +859,4 @@ def test_export_megatron_bridge_with_all_vllm_params(exporter, mock_llm):
863859
assert call_kwargs["swap_space"] == 8
864860
assert call_kwargs["cpu_offload_gb"] == 2
865861
assert call_kwargs["enforce_eager"] is True
866-
assert call_kwargs["max_seq_len_to_capture"] == 4096
867862
assert call_kwargs["task"] == "generate"

0 commit comments

Comments
 (0)