Skip to content

Conversation

@joy369
Copy link

@joy369 joy369 commented Jan 13, 2026

  • Add missing attr of PoolingParams

Hi all, I try to deploy BAAI/bge-large-en. It establishes server properly but fail to inference.

curl -X POST http://localhost:8000/v2/models/bge_emb/generate \
  -H "Content-Type: application/json" \
  -d '{
    "embedding_request": "{\"input\": \"Test\"}"
  }'

Below is the error log

(EngineCore_DPO pid=4141) Exception in thread Thread-5 (process_input_sockets):
(EngineCore_DPO pid=4141) Traceback (most recent call last):
  File "/usr/lib/python3.12/threading.py", line 1073, in _bootstrap_inner
    self.run()
  File "/usr/lib/python3.12/threading.py", line 1010, in run
    self.target(*self._args, **self._kwargs)
  File "/usr/local/lib/python3.12/dist-packages/vllm/v1/engine/core.py", line 1025, in process_input_sockets
    request = add_request_decoder.decode(data_frames)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/dist-packages/vllm/v1/serial_utils.py", line 311, in decode
    return self.decoder.decode(bufs[0])
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
msgpec.ValidationError: Expected 'bool', got 'None' - at '$[4][10]'

After assign Boolean value the model can inference properly. Thanks

- Add missing attr of PoolingParams
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant