Skip to content

vLLM parser modification in the launcher and requirements #163

@diegocastanibm

Description

@diegocastanibm

@manoelmarques is benchmarking the launcher and he has good comments to take into account:

1- Parser deprecation
Just to let you know in newer vllms the import for the parser is:
from vllm.utils.argparse_utils import FlexibleArgumentParser
That location has always been valid. The definition has been removed from vllm.utils

2- Requirements
Also, for my dockerfile and in general, the requirements.txt is way too big. SInce I am basing it from a vllm image, I already have everything needed. The only thing I install in the image additionally is uvicorn. I will be testing it and see if it is all that is needed. So far it starts the uvicorn server fine in a pod
Installing the whole requirements.txt using a base vllm image ended up making the base vllm crash with missing C functions and I believe it is because the requirements.txt ended up installing some pytorch version that conflicted with the binaries from the base vllm.

Metadata

Metadata

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions