lobes never advertises a capability it cannot serve (#92 · #91 · #95 · #97 · #96 · #74 · #69)

.devague/current

@@ -1 +1 @@
-lobes-preserves-qwen-thinking-traces-across-multi
+lobes-never-advertises-a-capability-it-cannot-serv

.devague/current_plan

@@ -1 +1 @@
-lobes-preserves-qwen-thinking-traces-across-multi
+lobes-never-advertises-a-capability-it-cannot-serv

CHANGELOG.md

@@ -4,6 +4,36 @@ All notable changes to this project are documented here. The format is based on
 [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and this project
 adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [0.40.0] - 2026-07-09
+
+### Added
+
+- `scripts/live-check.sh` + `tests/test_live_capabilities.py` — a LOCAL, single-trigger, unattended pre-PR gate that dials every advertised role endpoint+path, every id in `/v1/models`, checks CLI/gateway agreement, reproduces Colleague's role-discovery path, and fails on deployed-gateway version skew. It FAILS rather than skips when armed. A 429 (pressure shed) and a 503 carrying `Retry-After` count as reachable; only a 404, a connection failure, a `Retry-After`-less 503, or a bare 5xx are faults.
+- `lobes/gateway/_readiness.py` — a bounded background probe of each backend's `/health`, mirroring `PressureCache`. Tri-state (`True` healthy / `False` reached-but-unhealthy / `None` unreachable), daemon thread, socket-free `.current()`, and a probe that degrades to `None` on `OSError`, `http.client.HTTPException` and `ValueError`.
+- `GET /health` now reports `{"version": ...}`, and `lobes doctor` gains a `gateway_version_match` check that fails on skew between the deployed gateway and the CLI wheel (issue #99).
+- `lobes/gateway/_routing.py::is_unknown_model` — a pure predicate separating "unknown model id" from "unspecified model".
+- Ground-truth perception probes for the `senses` role: a stdlib-generated solid-colour PNG whose colour the model must name, and a Chatterbox-synthesized word the model must transcribe. Both carry negative controls; the old placeholder-media tests are relabelled as wire checks.
+
+### Changed
+
+- **No cross-backend failover.** `order_backends` now returns at most one backend. A request naming the cortex model can never be answered by the Gemma backend, which protects the `final_authority` role contract from #81.
+- `GET /v1/models` and `GET /capabilities.ready` are backed by the live readiness cache rather than by configuration. A wired-but-dead backend is no longer advertised.
+- `RoleInfo.ready` is no longer an alias of `loaded` for `cortex`/`senses`/`embedder`/`reranker`. `build_role_registry` self-enforces the invariant: a supplied `backend_ready` map is authoritative, and a present `None`, a present `False`, and a missing key all mean not-ready.
+- `lobes capabilities` / `lobes endpoint` now render the running gateway's `GET /capabilities` when it is reachable, falling back to an offline `.env`-derived view tagged `"source": "offline"` with `ready=false` on every role. The CLI and the gateway can no longer disagree, because there is now one derivation instead of two.
+- A backend is wired only when its `*_BASE_URL` is set; the `or *_SERVED_NAME` clause is gone (issue #97).
+- `senses` is documented as vision-only intake. The checkpoint declares audio support but vLLM's `gemma4_unified` path does not serve it (issue #101); `stt` remains the supported speech path.
+- `docs/gemma4-mtp-draft.md` carries a superseded banner: DSpark does not load on vLLM 0.23 (#75). Issue #69's disabled-experiment-entry criterion is closed answered-negative.
+- README's quickstart no longer claims `lobes init` scaffolds the single-model deployment; the fleet duo has been the default since #69.
+
+### Fixed
+
+- **#91** — a dead cortex backend no longer surfaces as a terminal `404 model does not exist`. `handle_post` rewrote the model id once, before the failover loop, then retried the same body against a backend serving a different model, which correctly 404'd; the `4xx = client error, no failover` rule relayed that as terminal and killed multi-step agent loops. A dead, unreachable or warming owner now yields **503 + `Retry-After`** with `type: backend_unavailable`.
+- **#92 / #95** — the gateway never advertises an origin built from its internal listen port. Precedence is `GATEWAY_PUBLIC_URL` (an operator override for a tunnel or Host-rewriting proxy) > the request `Host` header > an empty endpoint. `GATEWAY_PUBLIC_URL` is deliberately NOT defaulted: a defaulted `public_url` outranks `Host` and would advertise loopback to every remote client.
+- **#96** — `AUDIO_URL` now reaches the gateway from the base fleet compose, so `stt`/`tts` stop advertising `ready=true` on a path that 404s when the audio overlay is not composed in.
+- **#97** — `GET /v1/models` no longer advertises phantom backends wired from a `*_SERVED_NAME` alone against a `default_url` naming a container that need not exist.
+- An unknown model id returns `404 model_not_found` instead of being silently served by the default backend under a different model's weights. Unknown-ness is decided against the routing table, never against the readiness-filtered `/v1/models` list, so a wired-but-dead backend still yields 503 rather than 404.
+- `test_live_main_text_returns_nonempty_content` no longer fails on a thinking model: `max_tokens=16` was consumed entirely by the reasoning trace, leaving `content=None` and `finish_reason=length` (issue #93).
+
 ## [0.39.0] - 2026-07-07
 
 ### Added

CLAUDE.md

@@ -58,8 +58,11 @@ output quality is expected and opt-in, not guaranteed by the diagnostic. See
 Beyond `cortex`, the **fleet** exposes SIX first-class, Colleague-facing
 **roles** (issue #81) — the primary contract callers should address, not raw
 model ids: `cortex` (the 27B primary — reasoning/deciding/final authority),
-`senses` (the Gemma 4 12B multimodal gear — vision+audio intake/perception;
-never decides or takes repo actions), `embedder` (`Qwen/Qwen3-Embedding-0.6B` →
+`senses` (the Gemma 4 12B multimodal gear — vision intake/perception; never
+decides or takes repo actions; the checkpoint declares audio support but it is
+**not currently served** on this vLLM path — issue #101 — so `senses` is
+vision-only in practice, and the purpose-built `stt` role, below, is the
+supported path for speech), `embedder` (`Qwen/Qwen3-Embedding-0.6B` →
 `POST /v1/embeddings`), `reranker` (`Qwen/Qwen3-Reranker-0.6B` → `POST
 /v1/rerank` + `/v1/score`), and the opt-in audio overlay's `stt`/`tts`. Roles
 are routed by **task family** (`generate` / `embed` / `score` / `rerank`) and

README.md

@@ -50,17 +50,24 @@ CLIs in loops, so safe-by-default is mandatory.
 ## Running the model locally (vLLM)
 
 `lobes init` scaffolds a deployment directory (default `$HOME/.lobes`) from the
-packaged templates: a `docker-compose.yml` that stands up the vLLM model as an
-OpenAI-compatible server on `:8000`, plus a `.env`. Tuned for DGX Spark (GB10
-Grace Blackwell, 128 GB unified memory) per
+packaged templates. **Since issue #69, bare `lobes init` (no flags) scaffolds
+the fleet duo by default** — see ["Running the model behind a gateway
+(fleet)"](#running-the-model-behind-a-gateway-fleet) below; that is almost
+certainly what you want. This section instead walks through the **legacy
+single-model** scaffold: one vLLM server, no gateway, opted into with
+`--single` (alias `--legacy`) — a `docker-compose.yml` that stands up the
+vLLM model directly as an OpenAI-compatible server on `:8000` (the
+*container's own* port; there is no gateway in front of it in this mode —
+see the port note in the fleet section below), plus a `.env`. Tuned for DGX
+Spark (GB10 Grace Blackwell, 128 GB unified memory) per
 [build.nvidia.com/spark/vllm](https://build.nvidia.com/spark/vllm).
 
 Prerequisites: the [NVIDIA Container Toolkit](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html),
 and `docker login nvcr.io` with an [NGC API key](https://org.ngc.nvidia.com/setup/api-key)
 to pull the `nvcr.io/nvidia/vllm` image.
 
 ```bash
-lobes init --apply          # writes $HOME/.lobes/{docker-compose.yml,.env}
+lobes init --single --apply # writes $HOME/.lobes/{docker-compose.yml,.env} (legacy single-model)
 # edit $HOME/.lobes/.env to set HF_TOKEN if the model repo is gated
 lobes serve --apply         # first run downloads ~28 GB of weights (the 27B primary)
 lobes status                # waits/reports until /health is up
@@ -178,19 +185,37 @@ tunnel` for the full flow.
 
 ## Running the model behind a gateway (fleet)
 
-`lobes init --fleet` scaffolds a **multi-container** deployment instead of one: the
-always-warm Qwen generate primary, two tiny co-resident **embedding** and
-**reranker** gears, and a single stdlib **gateway** that fronts them on the host
-port the acp `vllm-local` provider already expects. The gateway routes each
-request by its `model` field — to the primary, the embedder, or the reranker by
-task family (generate / embed / score / rerank) — and defaults an unknown/missing
-name to the primary, so existing single-model clients keep working unchanged. The
-same front fans `/v1/audio/*` out to the `--audio` overlay, and a warm *generate*
-fallback can be wired in later (the gateway adds it, with failover, only when one
-is configured).
+**This is the default scaffold since issue #69** — plain `lobes init --apply`
+(no flags) already gives you this; `--fleet` is a back-compat no-op kept only
+so old scripts that passed it keep working, and `--single` (alias `--legacy`)
+is what opts you *out*, back to the one-container deployment the previous
+section describes. `lobes init` scaffolds a **multi-container** deployment:
+the always-warm Qwen generate primary, the Gemma 4 12B multimodal gear, two
+tiny co-resident **embedding** and **reranker** gears, and a single stdlib
+**gateway** that fronts them all on the host port the acp `vllm-local`
+provider already expects — `VLLM_PORT` (packaged default `:8000`; `:8001` on
+the reference DGX Spark deployment, deliberately set apart from the
+single-model story's default). The gateway routes each request by its
+`model` field — to the primary, the multimodal gear, the embedder, or the
+reranker by task family (generate / embed / score / rerank) — and defaults
+an unknown/missing name to the primary, so existing single-model clients
+keep working unchanged. The same front fans `/v1/audio/*` out to the
+`--audio` overlay, and a warm *generate* fallback can be wired in later (the
+gateway adds it, with failover, only when one is configured).
+
+> **`:8000` means two different things depending on scaffold — this
+> ambiguity is what issue #92 was about.** In the legacy single-model
+> section above, `:8000` is the vLLM **container's own** port, published
+> straight to the host with no gateway in front of it. Here, `:8000` (or
+> whatever `VLLM_PORT` is set to) is the **gateway's** published port,
+> fronting several backends by `model` field. They happen to share the same
+> default number, but a client must dial whichever origin its deployment
+> actually publishes — never assume a bare port number implies a particular
+> topology. `lobes doctor` and `GET /capabilities` are the source of truth
+> for what a given deployment actually serves and where.
 
 ```bash
-lobes init --fleet --apply        # $HOME/.lobes/{docker-compose.yml,.env,Dockerfile.gateway}
+lobes init --apply                # $HOME/.lobes/{docker-compose.yml,.env,Dockerfile.gateway} — the default
 docker login nvcr.io              # NGC API key for the vLLM image
 lobes fleet up --apply            # builds the gateway image + starts the backend
 lobes fleet status                # container states + gateway /health + /v1/models
@@ -204,7 +229,7 @@ curl -s http://localhost:8000/v1/chat/completions -d '{"model":"sakamakismile/Qw
 
 The fleet runs a **default-on `cortex` + `senses` duo** (the `main` + `multimodal`
 backends) — the 27B Qwen text generate primary served at **128K** (`cortex`, util
-`0.30`) and the Gemma 4 12B vision+audio gear served at **32K** (`senses`, util
+`0.30`) and the Gemma 4 12B vision gear served at **32K** (`senses`, util
 `0.14` — provisional pending live validation; `coolthor/gemma-4-12B-it-NVFP4A16`,
 native MTP default-on — the coder fine-tune, `sakamakismile/gemma-4-12B-coder-…`,
 is kept as an opt-in `multimodal-coder` gear; see

docs/colleague-stack.md

@@ -27,12 +27,23 @@ below.
 | Role | Backend / service | Endpoint path | What it's for |
 |---|---|---|---|
 | `cortex` | `primary` (generate) | `POST /v1/chat/completions` | Reasoning, deciding, planning, tool use, repo actions — the final authority. |
-| `senses` | `multimodal` (generate) | `POST /v1/chat/completions` | Intake/perception (text+image+audio) and speaking back to the user. Does **not** decide or act. |
+| `senses` | `multimodal` (generate) | `POST /v1/chat/completions` | Intake/perception (text+image) and speaking back to the user. Does **not** decide or act. |
 | `embedder` | `embed` (pooling) | `POST /v1/embeddings` | Dense text embeddings for memory/retrieval. |
 | `reranker` | `rerank` (pooling) | `POST /v1/rerank` (+ `/v1/score`) | Reordering/scoring retrieved candidates. |
 | `stt` | Parakeet (audio overlay, opt-in) | `POST /v1/audio/transcriptions` | Speech-to-text. |
 | `tts` | Chatterbox (audio overlay, opt-in) | `POST /v1/audio/speech` | Text-to-speech. |
 
+> **`senses` is vision-only intake — audio is not currently served (issue
+> #101).** The `coolthor/gemma-4-12B-it-NVFP4A16` checkpoint behind `senses`
+> declares an `audio_config` in its own model config, but on this vLLM serving
+> path (`gemma4_unified`) an `input_audio` content part is silently **dropped**
+> rather than rejected: a caller gets `200 OK` and a fluent answer that ignored
+> the audio. Live evidence and the tracking issue are in
+> [`docs/gemma-4-12b-nvfp4.md`](gemma-4-12b-nvfp4.md#live-validation-status-71).
+> For speech, use the purpose-built **`stt`** role (Parakeet, `POST
+> /v1/audio/transcriptions`) instead — it remains first-class and is
+> unaffected by this gap.
+
 `cortex`, `senses`, `embedder`, and `reranker` are always enumerated (present
 with `loaded=false` if their gear isn't wired in this deployment); `stt`/`tts`
 require `lobes init --fleet --audio`. **`brain` is not a valid role name** —
@@ -313,7 +324,7 @@ scaffold did solo, in exchange for co-residency:
 | **Fleet duo, pre-rebalance** | 64K, util 0.30 | 128K (`MULTIMODAL_MAX_MODEL_LEN=131072`, util 0.22) |
 | **Fleet duo, current (this doc)** | **128K** (`PRIMARY_MAX_MODEL_LEN=131072`, util 0.30 — util-bound, not context-bound) | **32K** (`MULTIMODAL_MAX_MODEL_LEN=32768`, util 0.14) |
 
-The pre-rebalance duo gave the vision/audio gear its full native 128K at the
+The pre-rebalance duo gave the vision gear its full native 128K at the
 cost of trimming `cortex` to 64K; the current default flips that trade-off —
 `cortex` (the final-authority reasoning role) now gets its full native 128K,
 and `senses` (intake/perception) is trimmed to 32K, which is ample for the

docs/gateway-fleet.md

@@ -347,6 +347,17 @@ is required (pinning keeps the image reproducible); from-source/dev boxes that r
 ahead of a PyPI release point `MODEL_GEAR_VERSION` at a published TestPyPI `.devN`
 build.
 
+**That pin is written once and nothing re-pins it (issue #99).** `lobes init`
+sets `MODEL_GEAR_VERSION` at scaffold time; no other verb touches it
+afterwards, so a long-lived deployment can silently keep running an old
+gateway image long after the host's own `lobes` binary — and PyPI — have
+moved on. `GET /health` reports the deployed gateway's own `lobes-cli`
+`version` (additive field, issue #99) precisely so this is detectable without
+docker: `lobes doctor`'s `gateway_version_match` check compares it against the
+CLI's own version and fails the run on a real mismatch (remediation: bump
+`MODEL_GEAR_VERSION` in `.env` and `docker compose up -d --build gateway`) —
+see [Verbs](#verbs) below.
+
 ### Auth (known limitation)
 
 The gateway is a **pass-through** and is **not auth-aware** — it does not inspect