coder-vllm-bootstrap

Tiny Python helper that provisions a Coder vLLM workspace and hands back an OpenAI-compatible endpoint + auth headers. Designed to drop into scripts, notebooks, evals, or services so you don't have to think about workspace lifecycle, token minting, or readiness probes.

from openai import OpenAI
from coder_vllm_bootstrap import bootstrap

endpoint, headers = bootstrap("Qwen/Qwen2.5-7B-Instruct")
client = OpenAI(base_url=endpoint, api_key="not-needed", default_headers=headers)
print(client.chat.completions.create(
    model="Qwen/Qwen2.5-7B-Instruct",
    messages=[{"role": "user", "content": "hi"}],
).choices[0].message.content)

What bootstrap() does:

1. Mints a fresh 1h Coder API token via the local coder CLI.
2. Resolves the current Coder username (used in the workspace subdomain).
3. Ensures the named workspace exists (creates with the configured template if missing, starts if stopped, no-op if running).
4. Polls the public /v1/models endpoint until vLLM responds.
5. Returns (endpoint, headers) ready to plug into any OpenAI-compatible client.

Requirements

• Python 3.9+
• The coder CLI on PATH, authenticated to your Coder server (coder login <url>).
• A Coder server with a vLLM-capable template (the matching Terraform template ships parameter names model, extra_args, vram_budget_gb).

Install

pip install coder-vllm-bootstrap

Configuration

Configuration is via environment variables. The library does not load .env files itself — use python-dotenv or your shell of choice.

Required:

Var	Description
CODER_URL	Base URL of your Coder server (any deployment).
CODER_WORKSPACE	Workspace name to provision / reuse.

Optional (defaults shown):

Var	Default	Effect
CODER_TEMPLATE	vllm	Coder template name used when creating.
CODER_APP_NAME	vllm	App subdomain prefix in the workspace URL: <APP>--<workspace>--<owner>.<host>.
BOOTSTRAP_TOKEN_NAME	vllm-bootstrap	Prefix for the minted Coder API token (timestamp suffix is appended).
VRAM_BUDGET_GB	60	Forwarded to the template's vram_budget_gb parameter.
VLLM_EXTRA_ARGS	--max-model-len 8192 --max-num-seqs 8	Forwarded to the template's extra_args parameter.

Usage with python-dotenv

from dotenv import load_dotenv
from coder_vllm_bootstrap import bootstrap

load_dotenv()  # reads .env from cwd
endpoint, headers = bootstrap("google/gemma-4-E4B-it")

A sample .env:

CODER_URL=https://coder.example.com
CODER_WORKSPACE=my-vllm
VRAM_BUDGET_GB=16
VLLM_EXTRA_ARGS=--max-model-len 8192 --max-num-seqs 8 --quantization fp8

Behavior notes

• Tokens are short-lived (1h) and minted fresh on every bootstrap() call. They auto-expire — no cleanup needed.
• Workspace creation can take 1–10 minutes depending on model size and quantization. The _wait_for_vllm poll has a 10-minute deadline.
• This library does not manage GPU contention. If your Coder host runs one workspace at a time per GPU, you're responsible for stopping conflicting workspaces before calling bootstrap().
• The workspace-listing filter checks both name and template_name, so two workspaces with the same name but different templates won't collide.

Assumptions about the template

The Coder template referenced by CODER_TEMPLATE must:

• Accept three workspace-creation parameters: model, extra_args, vram_budget_gb.
• Publish a coder_app whose slug matches CODER_APP_NAME, exposing the OpenAI-compatible API on the workspace's wildcard subdomain.

A reference template that satisfies both is here.

License

MIT — see LICENSE.