Skip to content

[Bug]: vLLM 0.25.0 crashed #48707

Open
Open
[Bug]: vLLM 0.25.0 crashed#48707
Labels
usageHow to use vllm

Description

@snailwei
snailwei
opened on Jul 15, 2026

Your current environment

ubuntu 22.04 LTS + vLLM 0.25.0 + docker
4xA100/80G + Qwen3.6-27b + claude code

(EngineCore pid=483) ERROR 07-15 01:39:25 [core.py:1233]     outputs, model_executed = self.step_fn()
(EngineCore pid=483) ERROR 07-15 01:39:25 [core.py:1233]                               ^^^^^^^^^^^^^^
(EngineCore pid=483) ERROR 07-15 01:39:25 [core.py:1233]   File "/usr/local/lib/python3.12/dist-packages/vllm/v1/engine/core.py", line 595, in step_with_batch_queue
(EngineCore pid=483) ERROR 07-15 01:39:25 [core.py:1233]     model_output = future.result()
(EngineCore pid=483) ERROR 07-15 01:39:25 [core.py:1233]                    ^^^^^^^^^^^^^^^
(EngineCore pid=483) ERROR 07-15 01:39:25 [core.py:1233]   File "/usr/local/lib/python3.12/dist-packages/vllm/v1/executor/multiproc_executor.py", line 91, in result
(EngineCore pid=483) ERROR 07-15 01:39:25 [core.py:1233]     return super().result()
(EngineCore pid=483) ERROR 07-15 01:39:25 [core.py:1233]            ^^^^^^^^^^^^^^^^
(EngineCore pid=483) ERROR 07-15 01:39:25 [core.py:1233]   File "/usr/lib/python3.12/concurrent/futures/_base.py", line 449, in result
(EngineCore pid=483) ERROR 07-15 01:39:25 [core.py:1233]     return self.__get_result()
(EngineCore pid=483) ERROR 07-15 01:39:25 [core.py:1233]            ^^^^^^^^^^^^^^^^^^^
(EngineCore pid=483) ERROR 07-15 01:39:25 [core.py:1233]   File "/usr/lib/python3.12/concurrent/futures/_base.py", line 401, in __get_result
(EngineCore pid=483) ERROR 07-15 01:39:25 [core.py:1233]     raise self._exception
(EngineCore pid=483) ERROR 07-15 01:39:25 [core.py:1233]   File "/usr/local/lib/python3.12/dist-packages/vllm/v1/executor/multiproc_executor.py", line 95, in _wait_for_response
(EngineCore pid=483) ERROR 07-15 01:39:25 [core.py:1233]     response = self.aggregate(self.get_response())
(EngineCore pid=483) ERROR 07-15 01:39:25 [core.py:1233]                               ^^^^^^^^^^^^^^^^^^^
(EngineCore pid=483) ERROR 07-15 01:39:25 [core.py:1233]   File "/usr/local/lib/python3.12/dist-packages/vllm/v1/executor/multiproc_executor.py", line 387, in get_response
(EngineCore pid=483) ERROR 07-15 01:39:25 [core.py:1233]     status, result = mq.dequeue(timeout=dequeue_timeout)
(EngineCore pid=483) ERROR 07-15 01:39:25 [core.py:1233]                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=483) ERROR 07-15 01:39:25 [core.py:1233]   File "/usr/local/lib/python3.12/dist-packages/vllm/distributed/device_communicators/shm_broadcast.py", line 779, in dequeue
(EngineCore pid=483) ERROR 07-15 01:39:25 [core.py:1233]     with self.acquire_read(timeout, indefinite) as buf:
(EngineCore pid=483) ERROR 07-15 01:39:25 [core.py:1233]          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=483) ERROR 07-15 01:39:25 [core.py:1233]   File "/usr/lib/python3.12/contextlib.py", line 137, in __enter__
(EngineCore pid=483) ERROR 07-15 01:39:25 [core.py:1233]     return next(self.gen)
(EngineCore pid=483) ERROR 07-15 01:39:25 [core.py:1233]            ^^^^^^^^^^^^^^
(EngineCore pid=483) ERROR 07-15 01:39:25 [core.py:1233]   File "/usr/local/lib/python3.12/dist-packages/vllm/distributed/device_communicators/shm_broadcast.py", line 701, in acquire_read
(EngineCore pid=483) ERROR 07-15 01:39:25 [core.py:1233]     raise RuntimeError("cancelled")
(EngineCore pid=483) ERROR 07-15 01:39:25 [core.py:1233] RuntimeError: cancelled
(APIServer pid=11) ERROR 07-15 01:39:25 [async_llm.py:704] AsyncLLM output_handler failed.
(APIServer pid=11) ERROR 07-15 01:39:25 [async_llm.py:704] Traceback (most recent call last):
(APIServer pid=11) ERROR 07-15 01:39:25 [async_llm.py:704]   File "/usr/local/lib/python3.12/dist-packages/vllm/v1/engine/async_llm.py", line 660, in output_handler
(APIServer pid=11) ERROR 07-15 01:39:25 [async_llm.py:704]     outputs = await engine_core.get_output_async()
(APIServer pid=11) ERROR 07-15 01:39:25 [async_llm.py:704]               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=11) ERROR 07-15 01:39:25 [async_llm.py:704]   File "/usr/local/lib/python3.12/dist-packages/vllm/v1/engine/core_client.py", line 1061, in get_output_async
(APIServer pid=11) ERROR 07-15 01:39:25 [async_llm.py:704]     raise self._format_exception(outputs) from None
(APIServer pid=11) ERROR 07-15 01:39:25 [async_llm.py:704] vllm.v1.engine.exceptions.EngineDeadError: EngineCore encountered an issue. See stack trace (above) for the root cause.
(APIServer pid=11) ERROR 07-15 01:39:25 [serving.py:820] Error in chat completion stream generator.
(APIServer pid=11) ERROR 07-15 01:39:25 [serving.py:820] Traceback (most recent call last):
(APIServer pid=11) ERROR 07-15 01:39:25 [serving.py:820]   File "/usr/local/lib/python3.12/dist-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 470, in chat_completion_stream_generator
(APIServer pid=11) ERROR 07-15 01:39:25 [serving.py:820]     async for res in result_generator:
(APIServer pid=11) ERROR 07-15 01:39:25 [serving.py:820]   File "/usr/local/lib/python3.12/dist-packages/vllm/v1/engine/async_llm.py", line 579, in generate
(APIServer pid=11) ERROR 07-15 01:39:25 [serving.py:820]     out = q.get_nowait() or await q.get()
(APIServer pid=11) ERROR 07-15 01:39:25 [serving.py:820]                             ^^^^^^^^^^^^^
(APIServer pid=11) ERROR 07-15 01:39:25 [serving.py:820]   File "/usr/local/lib/python3.12/dist-packages/vllm/v1/engine/output_processor.py", line 85, in get
(APIServer pid=11) ERROR 07-15 01:39:25 [serving.py:820]     raise output
(APIServer pid=11) ERROR 07-15 01:39:25 [serving.py:820]   File "/usr/local/lib/python3.12/dist-packages/vllm/v1/engine/async_llm.py", line 660, in output_handler
(APIServer pid=11) ERROR 07-15 01:39:25 [serving.py:820]     outputs = await engine_core.get_output_async()
(APIServer pid=11) ERROR 07-15 01:39:25 [serving.py:820]               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=11) ERROR 07-15 01:39:25 [serving.py:820]   File "/usr/local/lib/python3.12/dist-packages/vllm/v1/engine/core_client.py", line 1061, in get_output_async
(APIServer pid=11) ERROR 07-15 01:39:25 [serving.py:820]     raise self._format_exception(outputs) from None
(APIServer pid=11) ERROR 07-15 01:39:25 [serving.py:820] vllm.v1.engine.exceptions.EngineDeadError: EngineCore encountered an issue. See stack trace (above) for the root cause.
(APIServer pid=11) ERROR 07-15 01:39:25 [serving.py:1007] Error in message stream converter.
(APIServer pid=11) ERROR 07-15 01:39:25 [serving.py:1007] Traceback (most recent call last):
(APIServer pid=11) ERROR 07-15 01:39:25 [serving.py:1007]   File "/usr/local/lib/python3.12/dist-packages/vllm/entrypoints/anthropic/serving.py", line 797, in message_stream_converter
(APIServer pid=11) ERROR 07-15 01:39:25 [serving.py:1007]     origin_chunk = ChatCompletionStreamResponse.model_validate_json(
(APIServer pid=11) ERROR 07-15 01:39:25 [serving.py:1007]                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=11) ERROR 07-15 01:39:25 [serving.py:1007]   File "/usr/local/lib/python3.12/dist-packages/pydantic/main.py", line 782, in model_validate_json
(APIServer pid=11) ERROR 07-15 01:39:25 [serving.py:1007]     return cls.__pydantic_validator__.validate_json(
(APIServer pid=11) ERROR 07-15 01:39:25 [serving.py:1007]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=11) ERROR 07-15 01:39:25 [serving.py:1007] pydantic_core._pydantic_core.ValidationError: 2 validation errors for ChatCompletionStreamResponse
(APIServer pid=11) ERROR 07-15 01:39:25 [serving.py:1007] model
(APIServer pid=11) ERROR 07-15 01:39:25 [serving.py:1007]   Field required [type=missing, input_value={'error': {'message': 'En...am': None, 'code': 500}}, input_type=dict]
(APIServer pid=11) ERROR 07-15 01:39:25 [serving.py:1007]     For further information visit https://errors.pydantic.dev/2.13/v/missing
(APIServer pid=11) ERROR 07-15 01:39:25 [serving.py:1007] choices
(APIServer pid=11) ERROR 07-15 01:39:25 [serving.py:1007]   Field required [type=missing, input_value={'error': {'message': 'En...am': None, 'code': 500}}, input_type=dict]
(APIServer pid=11) ERROR 07-15 01:39:25 [serving.py:1007]     For further information visit https://errors.pydantic.dev/2.13/v/missing
(APIServer pid=11) ERROR 07-15 01:39:25 [serving.py:820] Error in chat completion stream generator.
(APIServer pid=11) ERROR 07-15 01:39:25 [serving.py:820] Traceback (most recent call last):
(APIServer pid=11) ERROR 07-15 01:39:25 [serving.py:820]   File "/usr/local/lib/python3.12/dist-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 470, in chat_completion_stream_generator
(APIServer pid=11) ERROR 07-15 01:39:25 [serving.py:820]     async for res in result_generator:
(APIServer pid=11) ERROR 07-15 01:39:25 [serving.py:820]   File "/usr/local/lib/python3.12/dist-packages/vllm/v1/engine/async_llm.py", line 579, in generate
(APIServer pid=11) ERROR 07-15 01:39:25 [serving.py:820]     out = q.get_nowait() or await q.get()
(APIServer pid=11) ERROR 07-15 01:39:25 [serving.py:820]                             ^^^^^^^^^^^^^
(APIServer pid=11) ERROR 07-15 01:39:25 [serving.py:820]   File "/usr/local/lib/python3.12/dist-packages/vllm/v1/engine/output_processor.py", line 85, in get
(APIServer pid=11) ERROR 07-15 01:39:25 [serving.py:820]     raise output
(APIServer pid=11) ERROR 07-15 01:39:25 [serving.py:820]   File "/usr/local/lib/python3.12/dist-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 470, in chat_completion_stream_generator
(APIServer pid=11) ERROR 07-15 01:39:25 [serving.py:820]     async for res in result_generator:
(APIServer pid=11) ERROR 07-15 01:39:25 [serving.py:820]   File "/usr/local/lib/python3.12/dist-packages/vllm/v1/engine/async_llm.py", line 579, in generate
(APIServer pid=11) ERROR 07-15 01:39:25 [serving.py:820]     out = q.get_nowait() or await q.get()
(APIServer pid=11) ERROR 07-15 01:39:25 [serving.py:820]                             ^^^^^^^^^^^^^
(APIServer pid=11) ERROR 07-15 01:39:25 [serving.py:820]   File "/usr/local/lib/python3.12/dist-packages/vllm/v1/engine/output_processor.py", line 85, in get
(APIServer pid=11) ERROR 07-15 01:39:25 [serving.py:820]     raise output
(APIServer pid=11) ERROR 07-15 01:39:25 [serving.py:820]   File "/usr/local/lib/python3.12/dist-packages/vllm/v1/engine/async_llm.py", line 660, in output_handler
(APIServer pid=11) ERROR 07-15 01:39:25 [serving.py:820]     outputs = await engine_core.get_output_async()
(APIServer pid=11) ERROR 07-15 01:39:25 [serving.py:820]               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=11) ERROR 07-15 01:39:25 [serving.py:820]   File "/usr/local/lib/python3.12/dist-packages/vllm/v1/engine/core_client.py", line 1061, in get_output_async
(APIServer pid=11) ERROR 07-15 01:39:25 [serving.py:820]     raise self._format_exception(outputs) from None
(APIServer pid=11) ERROR 07-15 01:39:25 [serving.py:820] vllm.v1.engine.exceptions.EngineDeadError: EngineCore encountered an issue. See stack trace (above) for the root cause.
(APIServer pid=11) ERROR 07-15 01:39:25 [serving.py:1007] Error in message stream converter.
(APIServer pid=11) ERROR 07-15 01:39:25 [serving.py:1007] Traceback (most recent call last):
(APIServer pid=11) ERROR 07-15 01:39:25 [serving.py:1007]   File "/usr/local/lib/python3.12/dist-packages/vllm/entrypoints/anthropic/serving.py", line 797, in message_stream_converter
(APIServer pid=11) ERROR 07-15 01:39:25 [serving.py:1007]     origin_chunk = ChatCompletionStreamResponse.model_validate_json(
(APIServer pid=11) ERROR 07-15 01:39:25 [serving.py:1007]                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=11) ERROR 07-15 01:39:25 [serving.py:1007]   File "/usr/local/lib/python3.12/dist-packages/pydantic/main.py", line 782, in model_validate_json
(APIServer pid=11) ERROR 07-15 01:39:25 [serving.py:1007]     return cls.__pydantic_validator__.validate_json(
(APIServer pid=11) ERROR 07-15 01:39:25 [serving.py:1007]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=11) ERROR 07-15 01:39:25 [serving.py:1007] pydantic_core._pydantic_core.ValidationError: 2 validation errors for ChatCompletionStreamResponse
(APIServer pid=11) ERROR 07-15 01:39:25 [serving.py:1007] model
(APIServer pid=11) ERROR 07-15 01:39:25 [serving.py:1007]   Field required [type=missing, input_value={'error': {'message': 'En...am': None, 'code': 500}}, input_type=dict]
(APIServer pid=11) ERROR 07-15 01:39:25 [serving.py:1007]     For further information visit https://errors.pydantic.dev/2.13/v/missing
(APIServer pid=11) ERROR 07-15 01:39:25 [serving.py:1007] choices
(APIServer pid=11) ERROR 07-15 01:39:25 [serving.py:1007]   Field required [type=missing, input_value={'error': {'message': 'En...am': None, 'code': 500}}, input_type=dict]
(APIServer pid=11) ERROR 07-15 01:39:25 [serving.py:1007]     For further information visit https://errors.pydantic.dev/2.13/v/missing
(APIServer pid=11) ERROR 07-15 01:39:25 [api_router.py:73] Error in create_messages: EngineCore encountered an issue. See stack trace (above) for the root cause.
(APIServer pid=11) ERROR 07-15 01:39:25 [api_router.py:73] Traceback (most recent call last):
(APIServer pid=11) ERROR 07-15 01:39:25 [api_router.py:73]   File "/usr/local/lib/python3.12/dist-packages/vllm/entrypoints/anthropic/api_router.py", line 71, in create_messages
(APIServer pid=11) ERROR 07-15 01:39:25 [api_router.py:73]     generator = await handler.create_messages(request, raw_request)
(APIServer pid=11) ERROR 07-15 01:39:25 [api_router.py:73]                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=11) ERROR 07-15 01:39:25 [api_router.py:73]   File "/usr/local/lib/python3.12/dist-packages/vllm/entrypoints/anthropic/serving.py", line 609, in create_messages
(APIServer pid=11) ERROR 07-15 01:39:25 [api_router.py:73]     generator = await self.create_chat_completion(chat_req, raw_request)
(APIServer pid=11) ERROR 07-15 01:39:25 [api_router.py:73]                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=11) ERROR 07-15 01:39:25 [api_router.py:73]   File "/usr/local/lib/python3.12/dist-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 245, in create_chat_completion
(APIServer pid=11) ERROR 07-15 01:39:25 [api_router.py:73]     return await self._with_kv_transfer_rejection_cleanup(
(APIServer pid=11) ERROR 07-15 01:39:25 [api_router.py:73]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=11) ERROR 07-15 01:39:25 [api_router.py:73]   File "/usr/local/lib/python3.12/dist-packages/vllm/entrypoints/generate/base/serving.py", line 229, in _with_kv_transfer_rejection_cleanup
(APIServer pid=11) ERROR 07-15 01:39:25 [api_router.py:73]     return await awaitable
(APIServer pid=11) ERROR 07-15 01:39:25 [api_router.py:73]            ^^^^^^^^^^^^^^^
(APIServer pid=11) ERROR 07-15 01:39:25 [api_router.py:73]   File "/usr/local/lib/python3.12/dist-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 266, in _create_chat_completion
(APIServer pid=11) ERROR 07-15 01:39:25 [api_router.py:73]     result = await self.render_chat_request(request)
(APIServer pid=11) ERROR 07-15 01:39:25 [api_router.py:73]              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=11) ERROR 07-15 01:39:25 [api_router.py:73]   File "/usr/local/lib/python3.12/dist-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 229, in render_chat_request
(APIServer pid=11) ERROR 07-15 01:39:25 [api_router.py:73]     raise self.engine_client.dead_error
(APIServer pid=11) ERROR 07-15 01:39:25 [api_router.py:73] vllm.v1.engine.exceptions.EngineDeadError: EngineCore encountered an issue. See stack trace (above) for the root cause.
(APIServer pid=11) INFO:     172.17.0.1:33262 - "POST /v1/messages?beta=true HTTP/1.1" 500 Internal Server Error
(EngineCore pid=483) Process EngineCore:
(EngineCore pid=483) Traceback (most recent call last):
(EngineCore pid=483)   File "/usr/lib/python3.12/multiprocessing/process.py", line 314, in _bootstrap
(EngineCore pid=483)     self.run()
(EngineCore pid=483)   File "/usr/lib/python3.12/multiprocessing/process.py", line 108, in run
(EngineCore pid=483)     self._target(*self._args, **self._kwargs)
(EngineCore pid=483)   File "/usr/local/lib/python3.12/dist-packages/vllm/v1/engine/core.py", line 1235, in run_engine_core
(EngineCore pid=483)     raise e
(EngineCore pid=483)   File "/usr/local/lib/python3.12/dist-packages/vllm/v1/engine/core.py", line 1224, in run_engine_core
(EngineCore pid=483)     engine_core.run_busy_loop()
(EngineCore pid=483)   File "/usr/local/lib/python3.12/dist-packages/vllm/v1/engine/core.py", line 1265, in run_busy_loop
(EngineCore pid=483)     self._process_engine_step()
(EngineCore pid=483)   File "/usr/local/lib/python3.12/dist-packages/vllm/v1/engine/core.py", line 1304, in _process_engine_step
(EngineCore pid=483)     outputs, model_executed = self.step_fn()
(EngineCore pid=483)                               ^^^^^^^^^^^^^^
(EngineCore pid=483)   File "/usr/local/lib/python3.12/dist-packages/vllm/v1/engine/core.py", line 595, in step_with_batch_queue
(EngineCore pid=483)     model_output = future.result()
(EngineCore pid=483)                    ^^^^^^^^^^^^^^^
(EngineCore pid=483)   File "/usr/local/lib/python3.12/dist-packages/vllm/v1/executor/multiproc_executor.py", line 91, in result
(EngineCore pid=483)     return super().result()
(EngineCore pid=483)            ^^^^^^^^^^^^^^^^
(EngineCore pid=483)   File "/usr/lib/python3.12/concurrent/futures/_base.py", line 449, in result
(EngineCore pid=483)     return self.__get_result()
(EngineCore pid=483)            ^^^^^^^^^^^^^^^^^^^
(EngineCore pid=483)   File "/usr/lib/python3.12/concurrent/futures/_base.py", line 401, in __get_result
(EngineCore pid=483)     raise self._exception
(EngineCore pid=483)   File "/usr/local/lib/python3.12/dist-packages/vllm/v1/executor/multiproc_executor.py", line 95, in _wait_for_response
(EngineCore pid=483)     response = self.aggregate(self.get_response())
(EngineCore pid=483)                               ^^^^^^^^^^^^^^^^^^^
(EngineCore pid=483)   File "/usr/local/lib/python3.12/dist-packages/vllm/v1/executor/multiproc_executor.py", line 387, in get_response
(EngineCore pid=483)     status, result = mq.dequeue(timeout=dequeue_timeout)
(EngineCore pid=483)                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=483)   File "/usr/local/lib/python3.12/dist-packages/vllm/distributed/device_communicators/shm_broadcast.py", line 779, in dequeue
(EngineCore pid=483)     with self.acquire_read(timeout, indefinite) as buf:
(EngineCore pid=483)          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore pid=483)   File "/usr/lib/python3.12/contextlib.py", line 137, in __enter__
(EngineCore pid=483)     return next(self.gen)
(EngineCore pid=483)            ^^^^^^^^^^^^^^
(EngineCore pid=483)   File "/usr/local/lib/python3.12/dist-packages/vllm/distributed/device_communicators/shm_broadcast.py", line 701, in acquire_read
(EngineCore pid=483)     raise RuntimeError("cancelled")
(EngineCore pid=483) RuntimeError: cancelled
(APIServer pid=11) ERROR 07-15 01:39:26 [api_router.py:73] Error in create_messages: EngineCore encountered an issue. See stack trace (above) for the root cause.
(APIServer pid=11) ERROR 07-15 01:39:26 [api_router.py:73] Traceback (most recent call last):
(APIServer pid=11) ERROR 07-15 01:39:26 [api_router.py:73]   File "/usr/local/lib/python3.12/dist-packages/vllm/entrypoints/anthropic/api_router.py", line 71, in create_messages
(APIServer pid=11) ERROR 07-15 01:39:26 [api_router.py:73]     generator = await handler.create_messages(request, raw_request)
(APIServer pid=11) ERROR 07-15 01:39:26 [api_router.py:73]                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=11) ERROR 07-15 01:39:26 [api_router.py:73]   File "/usr/local/lib/python3.12/dist-packages/vllm/entrypoints/anthropic/serving.py", line 609, in create_messages
(APIServer pid=11) ERROR 07-15 01:39:26 [api_router.py:73]     generator = await self.create_chat_completion(chat_req, raw_request)
(APIServer pid=11) ERROR 07-15 01:39:26 [api_router.py:73]                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=11) ERROR 07-15 01:39:26 [api_router.py:73]   File "/usr/local/lib/python3.12/dist-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 245, in create_chat_completion
(APIServer pid=11) ERROR 07-15 01:39:26 [api_router.py:73]     return await self._with_kv_transfer_rejection_cleanup(
(APIServer pid=11) ERROR 07-15 01:39:26 [api_router.py:73]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=11) ERROR 07-15 01:39:26 [api_router.py:73]   File "/usr/local/lib/python3.12/dist-packages/vllm/entrypoints/generate/base/serving.py", line 229, in _with_kv_transfer_rejection_cleanup
(APIServer pid=11) ERROR 07-15 01:39:26 [api_router.py:73]     return await awaitable
(APIServer pid=11) ERROR 07-15 01:39:26 [api_router.py:73]            ^^^^^^^^^^^^^^^
(APIServer pid=11) ERROR 07-15 01:39:26 [api_router.py:73]   File "/usr/local/lib/python3.12/dist-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 266, in _create_chat_completion
(APIServer pid=11) ERROR 07-15 01:39:26 [api_router.py:73]     result = await self.render_chat_request(request)
(APIServer pid=11) ERROR 07-15 01:39:26 [api_router.py:73]              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=11) ERROR 07-15 01:39:26 [api_router.py:73]   File "/usr/local/lib/python3.12/dist-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 229, in render_chat_request
(APIServer pid=11) ERROR 07-15 01:39:26 [api_router.py:73]     raise self.engine_client.dead_error
(APIServer pid=11) ERROR 07-15 01:39:26 [api_router.py:73] vllm.v1.engine.exceptions.EngineDeadError: EngineCore encountered an issue. See stack trace (above) for the root cause.
(APIServer pid=11) INFO:     172.17.0.1:33262 - "POST /v1/messages?beta=true HTTP/1.1" 500 Internal Server Error
(APIServer pid=11) INFO:     Shutting down
(APIServer pid=11) INFO:     Waiting for application shutdown.
(APIServer pid=11) INFO:     Application shutdown complete.
(APIServer pid=11) INFO:     Finished server process [11]
(APIServer pid=11) INFO 07-15 01:39:27 [core_client.py:655] [shutdown] MPClient: start timeout=0s
(APIServer pid=11) INFO 07-15 01:39:27 [core_client.py:657] [shutdown] MPClient: stopping engine manager
(APIServer pid=11) WARNING 07-15 01:39:27 [utils.py:626] [shutdown] Process manager: force killing remaining processes count=1
(APIServer pid=11) INFO 07-15 01:39:27 [core_client.py:659] [shutdown] MPClient: engine manager stopped
(APIServer pid=11) INFO 07-15 01:39:27 [core_client.py:660] [shutdown] MPClient: cleaning up background resources
(APIServer pid=11) INFO 07-15 01:39:27 [core_client.py:662] [shutdown] MPClient: complete
/usr/lib/python3.12/multiprocessing/resource_tracker.py:279: UserWarning: resource_tracker: There appear to be 2 leaked semaphore objects to clean up at shutdown
  warnings.warn('resource_tracker: There appear to be %d '
/usr/lib/python3.12/multiprocessing/resource_tracker.py:279: UserWarning: resource_tracker: There appear to be 3 leaked shared_memory objects to clean up at shutdown
  warnings.warn('resource_tracker: There appear to be %d '

Metadata

Metadata

Assignees

No one assigned

    Labels

    usageHow to use vllm

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions