I used following command to deploy kimi-k2 on 8 * H20 by VLLM
CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7 vllm serve kimi-k2.5 -tp 8 --mm-encoder-tp-mode data --trust-remote-code --tool-call-parser kimi_k2 --reasoning-parser kimi_k2
but come across a unreasonable error:
I used following command to deploy kimi-k2 on 8 * H20 by VLLM
CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7 vllm serve kimi-k2.5 -tp 8 --mm-encoder-tp-mode data --trust-remote-code --tool-call-parser kimi_k2 --reasoning-parser kimi_k2
but come across a unreasonable error: