Summary
The Responses API vLLM backend defines tensor parallelism with:
TP = os.environ.get("TP", 2)
When TP is unset, the fallback is integer 2. When a user configures the environment variable, os.environ.get() returns a string, and that string is passed directly to:
LLM(..., tensor_parallel_size=TP, ...)
tensor_parallel_size is an integer configuration value, so the explicit environment override changes the value's type instead of only changing its size.
Impact
The documented/configurable TP path can fail during vLLM engine setup when users set values such as TP=4, even though the default path works.
Proposed resolution
Parse the environment variable as an integer:
TP = int(os.environ.get("TP", 2))
Add a regression that imports the backend with TP=4 and verifies load_model() passes integer 4 to LLM.
Summary
The Responses API vLLM backend defines tensor parallelism with:
When
TPis unset, the fallback is integer2. When a user configures the environment variable,os.environ.get()returns a string, and that string is passed directly to:tensor_parallel_sizeis an integer configuration value, so the explicit environment override changes the value's type instead of only changing its size.Impact
The documented/configurable TP path can fail during vLLM engine setup when users set values such as
TP=4, even though the default path works.Proposed resolution
Parse the environment variable as an integer:
Add a regression that imports the backend with
TP=4and verifiesload_model()passes integer4toLLM.