[vLLM] tool_calls array is empty when using --tool-call-parser hcx with HyperCLOVAX-SEED-Think-32B

[sangmandu]

Environment

• Docker Image : pytorch:2.4.0-py3.11-cuda12.4.1-devel-ubuntu22.04
• vLLM version: 0.9.0
• transformers: 4.53.2
• hcx-vllm-plugin: latest (installed from source)
• Model: HyperCLOVAX-SEED-Think-32B
• GPU: H100 NVL x2

Setup Steps

1. Download model from HuggingFace

pip install huggingface_hub[hf_xet] safetensors torch easydict numpy
huggingface-cli download naver-hyperclovax/HyperCLOVAX-SEED-Think-32B \
    --local-dir ./models/HyperCLOVAX-SEED-Think-32B

2. Convert model using OmniServe

Since HyperCLOVAX-SEED-Think-32B is distributed as a unified model on HuggingFace, I used OmniServe's convert_model.py to extract the LLM component for vLLM serving.

  git clone https://github.com/NAVER-Cloud-HyperCLOVA-X/OmniServe.git
  python OmniServe/convert_model.py \
      --input ./models/HyperCLOVAX-SEED-Think-32B \
      --output ./track_a \
      --track a

3. Install vLLM and hcx-vllm-plugin

  pip install transformers==4.53.2 vllm==0.9.0
  git clone https://github.com/NAVER-Cloud-HyperCLOVA-X/hcx-vllm-plugin.git
  cd hcx-vllm-plugin && pip install . && cd ..

4. Start vLLM server

  python -m vllm.entrypoints.openai.api_server \
      --model ./track_a/llm/HyperCLOVAX-SEED-Think-32B \
      --tensor-parallel-size 2 \
      --trust-remote-code \
      --enable-auto-tool-choice \
      --tool-call-parser hcx \
      --reasoning-parser hcx \
      --port 8000

Issue

The server starts successfully, and the model generates tool calls in <tool_call> format within the content. However, the tool_calls array in the API response is always empty, meaning the hcx parser is not parsing the output correctly.

Request

  curl -s http://localhost:8000/v1/chat/completions \
    -H "Content-Type: application/json" \
    -d '{
      "model": "./track_a/llm/HyperCLOVAX-SEED-Think-32B",
      "messages": [{"role": "user", "content": "Send an email to john@example.com with subject Hello"}],
      "tools": [{"type": "function", "function": {"name": "send_email", "description": "Send an email", "parameters": {"type": "object", "properties": {"to": {"type": "string"}, "subject": {"type": "string"}}, "required": ["to", "subject"]}}}],
      "max_tokens": 512
    }'

Actual Response

  {
    "choices": [{
      "message": {
        "role": "assistant",
        "content": "I'll send an email to john@example.com with the subject \"Hello\" for you.\n<tool_call>send_email\n<arg_key>to</arg_key>\n<arg_value>john@example.com</arg_value>\n<arg_key>subject</arg_key>\n<arg_value>Hello</arg_value>\n</tool_call>",
        "tool_calls": []
      }
    }]
  }

The model correctly outputs tool calls in <tool_call> format, but tool_calls array is empty.

Expected Response

  {
    "choices": [{
      "message": {
        "role": "assistant",
        "content": "I'll send an email to john@example.com with the subject \"Hello\" for you.",
        "tool_calls": [{
          "id": "...",
          "type": "function",
          "function": {
            "name": "send_email",
            "arguments": "{\"to\": \"john@example.com\", \"subject\": \"Hello\"}"
          }
        }]
      }
    }]
  }

Questions

1. I want to use HyperCLOVAX-SEED-Think-32B as a pure LLM (without OmniServe's vision encoder). Is extracting the LLM component using convert_model.py --track a the correct approach?
2. According to the technical report, HyperCLOVAX-SEED-Think-32B shows strong agent/tool-calling capabilities. But with my setup, tool calls are not being parsed correctly. What is the recommended way to serve this model with tool-calling support?