need help failed to run demo3 with L40s GPU installed well #11
Description
hello guys, follow the intruction to run cache blend demo. then I suffer below error
anyone can shed some light?
root@salab-hpedl380g11-03:~/wayne/kvcache/demo# ls
demo1-quickstart demo2-multi-node-sharing demo3-KV-blending demo4-compare-with-vllm LICENSE README.md
root@salab-hpedl380g11-03:~/wayne/kvcache/demo# cd demo3-KV-blending/
root@salab-hpedl380g11-03:~/wayne/kvcache/demo/demo3-KV-blending# ls
chat_session.py data frontend.py README.md run-server.sh.template stop-dockers.sh
root@salab-hpedl380g11-03:~/wayne/kvcache/demo/demo3-KV-blending# cp run-server.sh.template run-server.sh
root@salab-hpedl380g11-03:~/wayne/kvcache/demo/demo3-KV-blending# vim run-server.sh
root@salab-hpedl380g11-03:~/wayne/kvcache/demo/demo3-KV-blending# bash ./run-server.sh
lmcache-blend: Pulling from apostacyh/vllm
aece8493d397: Pull complete
45f7ea5367fe: Pull complete
3d97a47c3c73: Pull complete
12cd4d19752f: Pull complete
da5a484f9d74: Pull complete
05e7384ae9b3: Pull complete
b8ae4c36c787: Pull complete
5d2454ae34bc: Pull complete
c22dc14fb058: Pull complete
089112ee8c16: Pull complete
849af6542ef4: Pull complete
65d56448b8e1: Pull complete
4f4fb700ef54: Pull complete
ff72ecef08d2: Pull complete
43315d1d9bdd: Pull complete
d3145db057a2: Pull complete
21feca74d50f: Pull complete
d9b0f72b4d31: Pull complete
6e541d84b188: Pull complete
7fbf8ab87f94: Pull complete
2c3bd89084bb: Pull complete
2dccf2e467bf: Pull complete
Digest: sha256:98bb3846125f5bd505f4c61742df9ef6d65100f50fb7b21a781aef16739818fa
Status: Downloaded newer image for apostacyh/vllm:lmcache-blend
docker.io/apostacyh/vllm:lmcache-blend
docker: Error response from daemon: could not select device driver "" with capabilities: [[gpu]]
Run 'docker run --help' for more information
root@salab-hpedl380g11-03:~/wayne/kvcache/demo/demo3-KV-blending#
root@salab-hpedl380g11-03:~# nvidia-smi
Fri Apr 25 08:24:34 2025
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 550.144.03 Driver Version: 550.144.03 CUDA Version: 12.4 |
|-----------------------------------------+------------------------+----------------------+
| GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|=========================================+========================+======================|
| 0 NVIDIA L40S Off | 00000000:8A:00.0 Off | 0 |
| N/A 36C P8 24W / 350W | 1MiB / 46068MiB | 0% Default |
| | | N/A |
+-----------------------------------------+------------------------+----------------------+
+-----------------------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=========================================================================================|
| No running processes found |
+-----------------------------------------------------------------------------------------+