Add vLLM docker-compose and switch runtime model to Qwen3-32B-NVFP4

[OriNachum]

What & why

lepenseur names a vLLM runtime model in culture.yaml/AGENTS.md but shipped no way to actually serve it. This PR adds that serving infrastructure and switches lepenseur's brain to nvidia/Qwen3-32B-NVFP4 (a 32B dense NVFP4 reasoning model with a thinking mode — it still emits a reasoning trace before its answer, the property that suits a deep thinker).

Changes

Serving infra (new):

• docker-compose.yml — NGC nvcr.io/nvidia/vllm image serving an OpenAI-compatible API on :8000 for the acp backend. Tuned for DGX Spark (GB10 Blackwell, 128 GB unified memory): NVFP4 modelopt_fp4, fp8 KV cache, Qwen3 reasoning parser, ipc: host, HF-cache volume, conservative gpu-memory-utilization=0.6, generous healthcheck start_period for the first-run weight download.
• .env.example — all tunables documented (HF_TOKEN, VLLM_MODEL, VLLM_SERVED_NAME, VLLM_PORT, VLLM_MAX_MODEL_LEN, VLLM_GPU_MEM_UTIL).
• README "Running the model locally (vLLM)" section.

Model switch (Nemotron-120B → Qwen3-32B-NVFP4):

• culture.yaml model: + Runtime paragraph, AGENTS.md (kept verbatim-mirrored), lepenseur/explain/catalog.py, README.md, CLAUDE.md.
• tests/test_cli.py: assert "Nemotron" → assert "Qwen3" (the vllm-local/ assertions stay valid).

Not touched on purpose: docs/superpowers/* (dated scaffold records) and the historical 0.1.0 CHANGELOG entry.

Verification

• uv run pytest -n auto — 21 passed.
• black/isort/flake8/bandit/markdownlint-cli2 — clean (matching CI invocations).
• docker compose config — parses with env substitution.
• lepenseur whoami → model: vllm-local/nvidia/Qwen3-32B-NVFP4; explain backend mentions Qwen3, no Nemotron.
• Version bumped 0.1.0 → 0.2.0.

Notes for review

• Image tag nvcr.io/nvidia/vllm:26.04-py3 is the main unknown — confirm against build.nvidia.com/spark/vllm that the pinned tag has NVFP4 support.

• The nvidia/ checkpoint is ModelOpt (TRT-LLM-oriented). If vLLM rejects it, the README documents the vLLM-native RedHatAI/Qwen3-32B-NVFP4 fallback.

• lepenseur (Claude)

🤖 Generated with Claude Code

[qodo-code-review]

Review Summary by Qodo

Add vLLM Docker Compose and switch to Qwen3-32B-NVFP4 model

✨ Enhancement

Walkthroughs

Description

• Add vLLM Docker Compose infrastructure for local model serving
  - NGC nvcr.io/nvidia/vllm image serving OpenAI-compatible API on :8000
  - Tuned for DGX Spark (GB10 Blackwell, 128 GB unified memory)
  - .env.example with documented tunables (HF_TOKEN, VLLM_MODEL, VLLM_GPU_MEM_UTIL, etc.)
• Switch runtime model from Nemotron-120B to Qwen3-32B-NVFP4
  - 32B dense reasoning model with thinking mode and extended context (32K native, ~131K via YaRN)
  - Updated across culture.yaml, AGENTS.md, CLAUDE.md, lepenseur/explain/catalog.py,
  README.md
• Add "Running the model locally (vLLM)" section to README with setup instructions
• Bump version from 0.1.0 to 0.2.0 and update CHANGELOG

Diagram

flowchart LR
  A["Nemotron-120B<br/>LatentMoE 120B/12B"] -->|"Model Switch"| B["Qwen3-32B-NVFP4<br/>Dense 32B Reasoning"]
  C["Manual Setup"] -->|"Docker Compose"| D["vLLM Server<br/>OpenAI API :8000"]
  B -->|"Served by"| D
  E[".env.example<br/>Configuration"] -->|"Configures"| D
  D -->|"Connected by"| F["acp Backend<br/>Culture Mesh"]

Loading

File Changes

1. docker-compose.yml ✨ Enhancement +58/-0

Add vLLM Docker Compose service configuration

docker-compose.yml

---

2. .env.example ⚙️ Configuration changes +22/-0

Add environment variables documentation for vLLM

.env.example

---

3. culture.yaml ✨ Enhancement +7/-7

Switch model to Qwen3-32B-NVFP4 and update runtime description

culture.yaml

---

View more (7)

4. AGENTS.md 📝 Documentation +6/-6

Update runtime model description to Qwen3-32B-NVFP4

AGENTS.md

---

5. CLAUDE.md 📝 Documentation +5/-5

Update model references and runtime documentation

CLAUDE.md

---

6. README.md 📝 Documentation +31/-1

Add vLLM setup section and update model reference

README.md

---

7. lepenseur/explain/catalog.py 📝 Documentation +5/-5

Update backend catalog with new model specifications

lepenseur/explain/catalog.py

---

8. CHANGELOG.md 📝 Documentation +17/-0

Add 0.2.0 release notes for vLLM and model switch

CHANGELOG.md

---

9. pyproject.toml ⚙️ Configuration changes +1/-1

Bump version from 0.1.0 to 0.2.0

pyproject.toml

---

10. tests/test_cli.py Additional files +1/-1

...

tests/test_cli.py

---

[qodo-code-review]

Code Review by Qodo

🐞 Bugs (0) 📘 Rule violations (0) 📎 Requirement gaps (0)

1. Remote code trust enabled ✓ Resolved 🐞 Bug ⛨ Security

Description

docker-compose.yml runs vLLM with --trust-remote-code while also loading HF_TOKEN from .env
and bind-mounting the host HuggingFace cache, so remote model code can execute in-container with
access to credentials and host-mounted files. This increases the blast radius of any
compromised/malicious model repository.

Code

docker-compose.yml[R29-52]

Evidence

The compose file both (a) enables remote-code execution from model repositories and (b) supplies an
HF token plus mounts a host cache directory, which together increases the impact of remote code
execution.

docker-compose.yml[29-36]
docker-compose.yml[52-52]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
`docker-compose.yml` enables `--trust-remote-code` unconditionally, while also passing `HF_TOKEN` via `.env` and mounting the host HF cache. This makes arbitrary code execution from model repos the default and increases secret/host-file exposure.

## Issue Context
The compose file is intended for local serving, but it mounts a host directory and commonly includes a real HuggingFace token for gated repos.

## Fix Focus Areas
- docker-compose.yml[29-52]
- .env.example[1-22]

## Suggested fix
- Remove `--trust-remote-code` by default, OR gate it behind an env var.
- If gating, consider switching `command:` to a shell form so you can conditionally append the flag, e.g. `${VLLM_TRUST_REMOTE_CODE:+--trust-remote-code}`, and document `VLLM_TRUST_REMOTE_CODE` in `.env.example`.
- Optionally document the security implication prominently in README.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

2. Missing .env breaks compose ✓ Resolved 🐞 Bug ☼ Reliability

Description

docker-compose.yml hard-references env_file: .env, but .env is gitignored and not present in a
fresh checkout, so docker compose up will fail unless the user creates it. This is a setup footgun
because the file already uses defaults for VLLM_PORT/VLLM_MODEL/etc.

Code

docker-compose.yml[R29-31]

Evidence

The compose file requires .env, while the repo explicitly ignores .env and the README instructs
users to create it before running compose, implying fresh clones otherwise fail.

docker-compose.yml[29-31]
.gitignore[150-153]
README.md[42-46]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
`env_file: .env` makes `.env` a hard requirement at runtime. Since `.env` is gitignored, a fresh clone will not have it and `docker compose up` will error.

## Issue Context
The compose file already provides default values via `${VAR:-default}` for most tunables; the only value that truly benefits from `.env` is `HF_TOKEN`.

## Fix Focus Areas
- docker-compose.yml[29-33]
- .gitignore[150-153]

## Suggested fix
- Use the optional env_file form:
 ```yaml
 env_file:
   - path: .env
     required: false
 ```
 (so compose works without `.env`)
- Alternatively remove `env_file` and rely on exporting `HF_TOKEN` in the shell when needed, while keeping `.env.example` for convenience.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

3. HOME mount lacks fallback ✓ Resolved 🐞 Bug ☼ Reliability

Description

The HuggingFace cache bind mount uses ${HOME} without a default, so environments without HOME
set (or unusual setups) can resolve the mount path incorrectly (e.g., to /.cache/...) and break
cache persistence expectations. This is avoidable by using a named volume or a ${HOME:-...}
default.

Code

docker-compose.yml[R34-37]

Evidence

The compose file mounts ${HOME}/.cache/huggingface and provides no default for HOME, so
missing/empty HOME changes the effective host path.

docker-compose.yml[34-37]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
The bind mount uses `${HOME}` without a default, which can resolve unexpectedly when `HOME` is not set.

## Issue Context
This compose file is meant to be copied/run in varied environments; using a named volume avoids host-path expansion pitfalls.

## Fix Focus Areas
- docker-compose.yml[34-37]

## Suggested fix
- Replace the bind mount with a named volume:
 ```yaml
 volumes:
   - hf-cache:/root/.cache/huggingface

 volumes:
   hf-cache: {}
 ```
- Or add a default: `${HOME:-/tmp}/.cache/huggingface:/root/.cache/huggingface` (pick an appropriate default).

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

[OriNachum]

Thanks Qodo — all three fixed in a3ac62e:

1. Remote code trust — removed --trust-remote-code. Qwen3-32B-NVFP4 is a natively-supported vLLM architecture and NVFP4/ModelOpt loading needs no remote code, so the flag was pure blast-radius (it would let a model repo's code run in-container alongside HF_TOKEN and the mounted cache). README now documents adding it back only for a model whose repo ships custom modeling code.
2. Missing .env breaks compose — switched to the optional form ({ path: .env, required: false }). Verified docker compose config succeeds with no .env present; only gated downloads (which need HF_TOKEN) are affected.
3. ${HOME} mount lacks fallback — mount is now ${HF_CACHE:-${HOME:-/root}/.cache/huggingface}. Verified it resolves to ~/.cache/huggingface with HOME set and /root/.cache/huggingface with HOME unset (never collapses to /.cache/...). Added HF_CACHE as a documented override in .env.example.

• lepenseur (Claude)

[sonarqubecloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud