Skip to content

Commit ac47042

Browse files
authored
Update the llm backend ports (#1172)
Signed-off-by: Wang, Kai Lawrence <[email protected]>
1 parent edcd7c9 commit ac47042

File tree

3 files changed

+6
-6
lines changed

3 files changed

+6
-6
lines changed

ChatQnA/docker_compose/amd/gpu/rocm/README.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -290,7 +290,7 @@ docker compose up -d
290290
Try the command below to check whether the TGI service is ready.
291291

292292
```bash
293-
docker logs ${CONTAINER_ID} | grep Connected
293+
docker logs chatqna-tgi-server | grep Connected
294294
```
295295

296296
If the service is ready, you will get the response like below.

ChatQnA/docker_compose/intel/hpu/gaudi/README.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -314,7 +314,7 @@ For validation details, please refer to [how-to-validate_service](./how_to_valid
314314
Try the command below to check whether the LLM serving is ready.
315315
316316
```bash
317-
docker logs tgi-service | grep Connected
317+
docker logs tgi-gaudi-server | grep Connected
318318
```
319319
320320
If the service is ready, you will get the response like below.
@@ -327,15 +327,15 @@ For validation details, please refer to [how-to-validate_service](./how_to_valid
327327
328328
```bash
329329
# TGI service
330-
curl http://${host_ip}:9009/v1/chat/completions \
330+
curl http://${host_ip}:8005/v1/chat/completions \
331331
-X POST \
332332
-d '{"model": ${LLM_MODEL_ID}, "messages": [{"role": "user", "content": "What is Deep Learning?"}], "max_tokens":17}' \
333333
-H 'Content-Type: application/json'
334334
```
335335
336336
```bash
337337
# vLLM Service
338-
curl http://${host_ip}:9009/v1/chat/completions \
338+
curl http://${host_ip}:8007/v1/chat/completions \
339339
-H "Content-Type: application/json" \
340340
-d '{"model": ${LLM_MODEL_ID}, "messages": [{"role": "user", "content": "What is Deep Learning?"}]}'
341341
```

ChatQnA/docker_compose/nvidia/gpu/README.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -273,7 +273,7 @@ docker compose up -d
273273
Try the command below to check whether the TGI service is ready.
274274

275275
```bash
276-
docker logs ${CONTAINER_ID} | grep Connected
276+
docker logs tgi-server | grep Connected
277277
```
278278

279279
If the service is ready, you will get the response like below.
@@ -285,7 +285,7 @@ docker compose up -d
285285
Then try the `cURL` command below to validate TGI.
286286

287287
```bash
288-
curl http://${host_ip}:9009/v1/chat/completions \
288+
curl http://${host_ip}:8008/v1/chat/completions \
289289
-X POST \
290290
-d '{"model": "Intel/neural-chat-7b-v3-3", "messages": [{"role": "user", "content": "What is Deep Learning?"}], "max_tokens":17}' \
291291
-H 'Content-Type: application/json'

0 commit comments

Comments
 (0)