Skip to content

Commit c45e5a8

Browse files
committed
update model name in sidecar test
Signed-off-by: Maya Barnea <mayab@il.ibm.com>
1 parent 7e64c19 commit c45e5a8

3 files changed

Lines changed: 6 additions & 9 deletions

File tree

test/sidecar/README.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -27,7 +27,7 @@ Send a request:
2727
$ curl http://localhost:8000/v1/completions \
2828
-H "Content-Type: application/json" \
2929
-H "x-prefiller-url: http://qwen-prefiller:8000" \
30-
-d '{"model": "Qwen/Qwen2-0.5B", "prompt": "Question: Greta worked 40 hours and was paid $12 per hour. Her friend Lisa earned $15 per hour at her job. How many hours would Lisa have to work to equal Gretas earnings for 40 hours?", "max_tokens": 200 }'
30+
-d '{"model": "Qwen/Qwen3-VL-2B-Instruct", "prompt": "Question: Greta worked 40 hours and was paid $12 per hour. Her friend Lisa earned $15 per hour at her job. How many hours would Lisa have to work to equal Gretas earnings for 40 hours?", "max_tokens": 200 }'
3131
```
3232

3333
Observe the decoder logs:

test/sidecar/req-book.txt

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,14 +1,14 @@
11
curl http://localhost:8000/v1/completions \
22
-H "Content-Type: application/json" \
33
-d '{
4-
"model": "Qwen/Qwen2-0.5B",
4+
"model": "Qwen/Qwen3-VL-2B-Instruct",
55
"prompt": "Author-contribution statements and acknowledgements in research papers should state clearly and specifically whether, and to what extent, the authors used AI technologies such as ChatGPT in the preparation of their manuscript and analysis. They should also indicate which LLMs were used. This will alert editors and reviewers to scrutinize manuscripts more carefully for potential biases, inaccuracies and improper source crediting. Likewise, scientific journals should be transparent about their use of LLMs, for example when selecting submitted manuscripts. Mention the large language model based product mentioned in the paragraph above:"
66
}'
77

88
curl http://localhost:8000/v1/chat/completions \
99
-H "Content-Type: application/json" \
1010
-d '{
11-
"model": "Qwen/Qwen2-0.5B",
11+
"model": "Qwen/Qwen3-VL-2B-Instruct",
1212
"messages": [
1313
{"role": "system", "content": "You are a helpful assistant."},
1414
{"role": "user", "content":"Author-contribution statements and acknowledgements in research papers should state clearly and specifically whether, and to what extent, the authors used AI technologies such as ChatGPT in the preparation of their manuscript and analysis. They should also indicate which LLMs were used. This will alert editors and reviewers to scrutinize manuscripts more carefully for potential biases, inaccuracies and improper source crediting. Likewise, scientific journals should be transparent about their use of LLMs, for example when selecting submitted manuscripts. Mention the large language model based product mentioned in the paragraph above:"}
@@ -36,7 +36,7 @@ curl http://localhost:8000/v1/completions \
3636
-H "Content-Type: application/json" \
3737
-H "x-prefiller-url: http://localhost:8002" \
3838
-d '{
39-
"model": "Qwen/Qwen2-0.5B",
39+
"model": "Qwen/Qwen3-VL-2B-Instruct",
4040
"prompt": "I am working on learning to run benchmarks in my openshift cluster. I was wondering if you could provide me a list of best practices when collecting metrics on the k8s platform, and furthermore, any OCP specific optimizations that are applicable here. Finally please help me construct a plan to support testing metrics collection for testing and dev environments such as minikube or kind.",
4141
"max_tokens": 200,
4242
"temperature": 0
@@ -45,7 +45,7 @@ curl http://localhost:8000/v1/completions \
4545
curl http://localhost:8000/v1/chat/completions \
4646
-H "Content-Type: application/json" \
4747
-d '{
48-
"model":"Qwen/Qwen2-0.5B",
48+
"model":"Qwen/Qwen3-VL-2B-Instruct",
4949
"messages": [
5050
{"role": "system", "content": "You are a helpful assistant."},
5151
{"role": "user", "content": "I am working on learning to run benchmarks in my openshift cluster. I was wondering if you could provide me a list of best practices when collecting metrics on the k8s platform, and furthermore, any OCP specific optimizations that are applicable here. Finally please help me construct a plan to support testing metrics collection for testing and dev environments such as minikube or kind."}
@@ -54,4 +54,4 @@ curl http://localhost:8000/v1/chat/completions \
5454
}'
5555

5656

57-
lm_eval --model local-completions --tasks gsm8k --model_args model="Qwen/Qwen2-0.5B",base_url=http://127.0.0.1:8000/v1/completions,num_concurrent=5,max_retries=3,tokenized_requests=False
57+
lm_eval --model local-completions --tasks gsm8k --model_args model="Qwen/Qwen3-VL-2B-Instruct",base_url=http://127.0.0.1:8000/v1/completions,num_concurrent=5,max_retries=3,tokenized_requests=False

test/utils/common.go

Lines changed: 0 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -5,6 +5,3 @@ const ModelName = "Qwen/Qwen3-VL-2B-Instruct"
55

66
// server pod name based onthe model name, used for pool and epp pod names
77
const ModelServerName = "vllm-qwen3-2b-instruct"
8-
9-
// "HuggingFaceTB/SmolVLM-500M-Instruct"
10-
// "Qwen/Qwen2-0.5B"

0 commit comments

Comments
 (0)