[Description (from Claude)]
vllm_isolated_venv mismatch between harness preset and model preset provider kind
Bug
When launching with --harness default, the job is correctly set up with vllm_isolated_venv=True because the default preset specifies kind=ProviderKind.VLLM_SERVER (presets.py:74). This causes vLLM to be installed in /opt/vllm-venv and stripped from the main /opt/venv during the install step (launcher.py:704-720).
However, get_provider_extras is called with the model spec rather than the harness provider kind (job_assembler.py:416):
provider_extras = get_provider_extras(exp.model_spec)
For model presets with kind=ProviderKind.VLLM (e.g. olmo-3-1025-7b), this correctly adds "vllm" to extras, which combined with vllm_isolated_venv=True triggers the isolated install. But at runtime, the model preset's own kind=vllm can take precedence, causing the runner to try to import vLLM from the main venv where it is no longer present.
Root cause
vllm_isolated_venv is determined by the harness provider kind (job_assembler.py:409):
vllm_isolated_venv = provider_kind == "vllm_server"
But the runtime provider kind can diverge from the install-time assumption when the model preset specifies a different kind than the harness preset.
Steps to reproduce
- Use a model preset with
kind=vllm (e.g. olmo-3-1025-7b)
- Launch with
--harness default
- Job installs vLLM into
/opt/vllm-venv only
- At runtime the
vllm provider is selected and fails to import from the main venv
Expected behavior
The provider kind used at install time and runtime should be consistent. Either the harness preset's provider kind should fully override the model preset at runtime, or vllm_isolated_venv should only be set when the runtime provider is confirmed to be vllm_server.
[Description (from Claude)]
vllm_isolated_venvmismatch between harness preset and model preset provider kindBug
When launching with
--harness default, the job is correctly set up withvllm_isolated_venv=Truebecause thedefaultpreset specifieskind=ProviderKind.VLLM_SERVER(presets.py:74). This causes vLLM to be installed in/opt/vllm-venvand stripped from the main/opt/venvduring the install step (launcher.py:704-720).However,
get_provider_extrasis called with the model spec rather than the harness provider kind (job_assembler.py:416):For model presets with
kind=ProviderKind.VLLM(e.g.olmo-3-1025-7b), this correctly adds"vllm"to extras, which combined withvllm_isolated_venv=Truetriggers the isolated install. But at runtime, the model preset's ownkind=vllmcan take precedence, causing the runner to try to import vLLM from the main venv where it is no longer present.Root cause
vllm_isolated_venvis determined by the harness provider kind (job_assembler.py:409):But the runtime provider kind can diverge from the install-time assumption when the model preset specifies a different
kindthan the harness preset.Steps to reproduce
kind=vllm(e.g.olmo-3-1025-7b)--harness default/opt/vllm-venvonlyvllmprovider is selected and fails to import from the main venvExpected behavior
The provider kind used at install time and runtime should be consistent. Either the harness preset's provider kind should fully override the model preset at runtime, or
vllm_isolated_venvshould only be set when the runtime provider is confirmed to bevllm_server.