Skip to content

[BUG] vLLM venv mismatch between harness preset and model preset #123

@rlebras

Description

@rlebras

[Description (from Claude)]

vllm_isolated_venv mismatch between harness preset and model preset provider kind

Bug

When launching with --harness default, the job is correctly set up with vllm_isolated_venv=True because the default preset specifies kind=ProviderKind.VLLM_SERVER (presets.py:74). This causes vLLM to be installed in /opt/vllm-venv and stripped from the main /opt/venv during the install step (launcher.py:704-720).

However, get_provider_extras is called with the model spec rather than the harness provider kind (job_assembler.py:416):

provider_extras = get_provider_extras(exp.model_spec)

For model presets with kind=ProviderKind.VLLM (e.g. olmo-3-1025-7b), this correctly adds "vllm" to extras, which combined with vllm_isolated_venv=True triggers the isolated install. But at runtime, the model preset's own kind=vllm can take precedence, causing the runner to try to import vLLM from the main venv where it is no longer present.

Root cause

vllm_isolated_venv is determined by the harness provider kind (job_assembler.py:409):

vllm_isolated_venv = provider_kind == "vllm_server"

But the runtime provider kind can diverge from the install-time assumption when the model preset specifies a different kind than the harness preset.

Steps to reproduce

  1. Use a model preset with kind=vllm (e.g. olmo-3-1025-7b)
  2. Launch with --harness default
  3. Job installs vLLM into /opt/vllm-venv only
  4. At runtime the vllm provider is selected and fails to import from the main venv

Expected behavior

The provider kind used at install time and runtime should be consistent. Either the harness preset's provider kind should fully override the model preset at runtime, or vllm_isolated_venv should only be set when the runtime provider is confirmed to be vllm_server.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions