Skip to content

[Feat] Support command and args on servingEngineSpec.modelSpec - #1071

Open
dundysm wants to merge 1 commit into
vllm-project:mainfrom
dundysm:feat/modelSpec-command-args
Open

[Feat] Support command and args on servingEngineSpec.modelSpec#1071
dundysm wants to merge 1 commit into
vllm-project:mainfrom
dundysm:feat/modelSpec-command-args

Conversation

@dundysm

@dundysm dundysm commented Sep 1, 2026

Copy link
Copy Markdown

The main vLLM container currently hardcodes vllm serve. This adds optional command and args on each servingEngineSpec.modelSpec[] entry so startup can be wrapped the same way init containers already can (the Prometheus/LMCache mkdir case from the issue).

When command is unset, the default vllm serve invocation is unchanged. vllmConfig.extraArgs still applies only to that default path.

Fixes #682

  • Make sure the code changes pass the pre-commit checks. Helm-only change; ran helm lint and helm template.
  • Sign-off your commit by using -s when doing git commit
  • Try to classify PRs ([Feat])

Testing

  • helm lint helm/ — pass (chart dependency warning only)
  • helm template with chart defaults — main container command stays vllm serve
  • helm template with modelSpec.command / modelSpec.args — custom sh -c wrapper, default serve is not in command
  • Added a helm unittest case in helm/tests/deployment-vllm-multi_test.yaml

The main vLLM container currently hardcodes `vllm serve`. Optional
command and args on each modelSpec let users wrap startup (for example
mkdir before exec) the same way init containers already can.

Fixes vllm-project#682

Signed-off-by: Dundy Pasupuleti <pasupuletidundy@gmail.com>

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces support for configuring custom commands and arguments for the main vLLM container in the multi-deployment Helm chart. It updates the deployment template to conditionally render command and args if specified in the model configuration, adds corresponding schema definitions in values.schema.json, provides commented-out examples in values.yaml, and includes a unit test to verify the new behavior. There are no review comments, and I have no feedback to provide.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feature: Support command and args in the servingEngineSpec.modelSpec[]

1 participant