CLAUDE.md

CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

What this project is

lobes is the tooling that runs, assesses, and switches the local, OpenAI-compatible vLLM model the Culture mesh consumes. The binary is lobes (lobes switch, lobes assess, lobes serve, …; model is a deprecated alias).

lobes is one identity — the tool and the deployed agent:

• lobes is the repo and the tool. It is a normal CLI/PyPI sibling (Python package lobes, binary lobes, distributed as lobes-cli).
• lobes is also the agent deployed on the model it serves. AGENTS.md + culture.yaml are that agent's runtime identity (the acp system prompt and the suffix: lobes / backend: acp / model: vllm-local/... declaration). Same name, one identity: the gear runs the model and the agent rides on it. (It used to be a separate agent, lepenseur; that name is retired.)

The served model is vllm-local/unsloth/Qwen3.6-27B-NVFP4 (a Qwen3.6 27B with hybrid Mamba/linear-attention layers; MULTIMODAL — image and video intake through its own ViT; a self-hosted MTP draft head baked into the checkpoint, so vLLM speculative decoding (Multi-Token Prediction) works with no external draft repo; compressed-tensors NVFP4 (mixed precision: fp8 attention/lm_head + nvfp4 MLP, ViT left unquantized), 256K native; thinking mode with a reasoning trace). This is the cortex role — the fleet's reasoning/deciding/final-authority lobe (issue #81), and since the 2026-07-31 promotion it is the first role that can both see an image and decide: the role contract forbids senses from final_decision/repo_action and worker from final_decision/security_decision, so before this every visual decision had to be handed to a role barred from making it.

Served context depends on deployment shape: the legacy single-model scaffold (lobes serve, no fleet) serves the full 256K solo; the spark-lobe shape (what the DGX Spark runs — senses dropped to a peer) serves cortex at the full 256K at gpu_mem_util=0.44, MEASURED 2026-07-31; the machine-as-brain fleet duo declares 128K (PRIMARY_MAX_MODEL_LEN=131072) so cortex can co-reside with a local multimodal gear — that duo budget is inherited from the previous text-only checkpoint and has not been booted with a ViT (see lobes/profiles/builtin/spark.toml). See docs/colleague-stack.md#migration-before--after. lobes runs it; the acp vllm-local provider connects the lobes agent to it.

Two 27B checkpoints remain as candidates, kept not deleted (cite-don't-delete):

• sakamakismile/Qwen3.6-27B-Text-NVFP4-MTP — the previous default primary, demoted 2026-07-31. Its export dropped the ViT (hence its --language-model-only and --tokenizer=mmangkad/… flags, both now gone from the lane), so it is the remaining text-only 27B — the pick for a deployment wanting a smaller weight footprint and no vision. Every pre-0.54.9 evidence transcript was measured against it.
• mmangkad/Qwen3.6-27B-NVFP4 — the archived original primary. It used to be justified as the tokenizer source the MTP primary served with and the only vision-capable 27B; both rationales are now obsolete — the promoted primary ships its own tokenizer and its own ViT. It is kept as a plain candidate.

The nvidia/Qwen3-32B-NVFP4 dense model also remains a supported candidate — see docs/qwen3-32b-nvfp4.md and lobes overview --list.

Swapping the served checkpoint breaks every consumer that pins a raw model id — and as of the 2026-07-31 audit, none of them address the fleet by role name. Read docs/model-switch-playbook.md before the next swap; it records the ordering that matters (benchmark the incumbent first — that baseline is unrecoverable afterwards) and two measurement traps that produced wrong answers on this one.

Thinking continuity — preserve_thinking (issue #93). The cortex/main vLLM service adds --default-chat-template-kwargs '{"preserve_thinking": true}' next to --reasoning-parser=qwen3, so the served Qwen3.6 chat template retains all historical <think> blocks across a multi-turn conversation by default (the template otherwise keeps only the reasoning after the last user turn). It is default-on but per-request overridable — a caller's own chat_template_kwargs wins over the server default, so lobes route's terse routing path still forces enable_thinking=false and gets a thinking-free reply. Scoped to the cortex/main generate lane only — the embed/rerank/senses lanes are untouched. A read-only preserve-thinking diagnostic (a two-turn prompt-token-count delta) proves the input-side round-trip is live; the continuity benefit to output quality is expected and opt-in, not guaranteed by the diagnostic. See docs/qwen3.6-27b-text-nvfp4-mtp.md for the flag and diagnostic detail.

Strict tool calling with thinking (colleague#320). Unconstrained thinking-mode generation can drift off the qwen3_coder tool-call template and get "salvaged" by vLLM's parser into a mangled call; strict: true on a tool schema arms xgrammar structural-tag constrained decoding to make that impossible, but the served build hardcodes reasoning=False at its structural-tag call site, which breaks the grammar for a thinking model (</think> rejected → 500). The fix is the qwen3_coder_thinking tool-parser plugin (lobes/vllm_plugins/, loaded via vLLM's own --tool-parser-plugin file-path surface, cortex/main lane only — mirrors the preserve_thinking #93 scoping) — it derives the grammar's reasoning flag from the request's own enable_thinking instead of the hardcoded False. A separate, default-off gateway knob (GATEWAY_FORCE_STRICT_TOOLS) opts existing callers into strict schemas without a client-side change, with a retry-without-strict fallback on a grammar-compile failure. See docs/qwen3.6-27b-text-nvfp4-mtp.md, docs/openai-api.md, and docs/gateway-fleet.md for the mechanism, scope, and knob detail.

Gemma 4 tool calling — the gemma4 parser PAIR (2026-07-17). All three Gemma 4 lanes (senses, the opt-in coder candidate, and muse) serve tool calls with a matched pair: --tool-call-parser=gemma4 (Gemma4EngineToolParser) plus --reasoning-parser=gemma4 (Gemma4ParserReasoningAdapter) — mirroring how the cortex lane has always paired --reasoning-parser=qwen3 with its qwen3_coder tool parser. lobes previously wired neither half, and both failures were silent:

• Wrong tool parser. Gemma 4 emits <|tool_call>call:name{...}<tool_call|>, whose delimiters are special tokens. The generic pythonic parser runs with skip_special_tokens=True, never sees them, matches nothing, and vLLM relays the model's well-formed call as ordinary assistant content with tool_calls: null / finish_reason: "stop" — callers get prose shaped like a tool call and no callable one. pythonic was a never-validated guess (its own _parser.py comment flagged it "risk r2, pending live validation"); the check finally ran on the live 31B and disproved it.
• Missing reasoning parser. The tool parser forces skip_special_tokens=False (that is how it sees <|tool_call>), which also exposes Gemma's <|channel>thought markers — so the tool parser alone leaks them into content. The reasoning parser is what consumes them. Enable both or neither.

Validated on the 31B muse lane only (physical Thor, docs/evidence/2026-07-17-accept-muse-tool-calling-thor.txt); the 12B lanes inherit the family rule and are UNVALIDATED (#108). Note GATEWAY_FORCE_STRICT_TOOLS deliberately does not arm the muse lane — measured live, strict never engages xgrammar there at all (a schema xgrammar cannot compile is still served 200), so arming it would advertise a grammar-constrained lane that isn't one. See docs/gemma-4-31b-nvfp4.md#tool-calling, which also records two rationales for that exclusion that turned out to be wrong (the supports_required_and_named flag, which cortex's own parser shares; and an EngineCore-crash risk that did not reproduce).

Colleague roles: cortex / senses / muse / worker / embedder / reranker / stt / tts

Beyond cortex, the fleet exposes EIGHT first-class, Colleague-facing roles (issue #81; worker joined as the eighth — thor-worker-lobe plan) — the primary contract callers should address, not raw model ids: cortex (the 27B primary — reasoning/deciding/final authority), senses (the Gemma 4 12B multimodal gear — vision intake/perception; never decides or takes repo actions; the checkpoint declares audio support but it is not currently served on this vLLM path — issue #101 — so senses is vision-only in practice, and the purpose-built stt role, below, is the supported path for speech), muse (the opt-in-hosted creative/ideation lobe — currently DORMANT/unhosted mesh-wide, see the paragraph below), worker (the opt-in-hosted fast ground-work DOER, and the first non-cortex role allowed to act on the repo — see the paragraph below), embedder (Qwen/Qwen3-Embedding-0.6B → POST /v1/embeddings), reranker (Qwen/Qwen3-Reranker-0.6B → POST /v1/rerank + /v1/score), and the opt-in audio overlay's stt/tts. Roles are routed by task family (generate / embed / score / rerank) and discoverable via lobes capabilities / lobes endpoint <role> / gateway GET /capabilities — a JSON contract keyed by role (model / runtime / endpoint / path / context / quant / mtp / responsibilities / forbidden_responsibilities / ready / loaded); see docs/colleague-stack.md for the full contract. cortex/senses/embedder/reranker are default-on and co-reside on the DGX Spark GB10: cortex serves its full 128K native context at util 0.30, senses is trimmed to 32K at util 0.14, and the two ~0.6B pooling gears run at *_GPU_MEM_UTIL=0.06 each — default budget 0.30 + 0.14 + 0.06 + 0.06 = 0.56 on the 128 GB GB10. These are the machine-as-brain (default) values — one box hosting every role it can serve; a mesh-brain deployment shape (below) drops one heavy lobe to a peer box and reclaims its budget instead of merely co-residing it. The 4B minor (back-compat cheap, COMPOSE_PROFILES=minor, util 0.10) and the legacy 14B Qwen (COMPOSE_PROFILES=middle, util 0.12) are opt-in gears and are not first-class Colleague roles. Callers address the generate lane by capability-tier alias — model=main|minor|multimodal|worker|muse (back-compat: hard|cheap|normal; capability order minor < multimodal < worker < muse < main), or the Colleague-role names model=cortex|senses layered on top of main/multimodal (muse's and worker's role names ARE their tier/backend names); normal/multimodal maps to the Gemma gear, not the demoted 14B. A swap/iowait pressure policy SHEDS full-tier cortex, senses, worker, and muse requests with HTTP 429 + Retry-After rather than substituting a different model (swap > 75 % or iowait > 50 % → busy — the former degrade-to-minor substitution path was removed outright, so there is no cheaper-rung fallback for any of the four); an explicit minor request is the servable floor and is always served regardless of pressure. lobes status --pressure shows the current busy/warm state. Start/stop one role at a time with lobes up <role> (or the six-default-role bundle, lobes up colleague-stack — muse and worker are deliberately excluded from the bundle, both being opt-in-hosted; lobes up muse works on a muse-hosting deployment and errors helpfully when COMPOSE_PROFILES doesn't include muse, and lobes up worker is landing alongside the worker-hosting shape, below, to mirror that exact mechanic); measure per-role runtime with lobes measure (muse and worker ride the llm family) and compare fleet profiles with lobes benchmark --profile {cortex-only, cortex+senses,senses-direct,qwen-nvfp4-vs-bf16,all}. LoRA adapter training targets the 4B bf16 minor only — the 14B NVFP4 is inference-only, and there is no lobes train verb. See docs/qwen3-embedding-0.6b.md, docs/qwen3-reranker-0.6b.md, docs/gemma-4-12b-nvfp4.md, docs/gemma-4-31b-nvfp4.md, docs/qwen3.6-35b-a3b-nvfp4.md, docs/gateway-fleet.md, and docs/colleague-stack.md (the eight-role contract).

muse — the seventh role, currently DORMANT/unhosted mesh-wide. Checkpoint: nvidia/Gemma-4-31B-IT-NVFP4 (Gemma 4 31B IT, NVIDIA's official modelopt NVFP4 export; 256K native; plain-gemma4 line, gemma4 tool parser; native MTP declared via the google/gemma-4-31B-it-assistant draft — UNMEASURED on this target). Responsibilities: creative generation, long-form writing, ideation, style variation, a divergent second opinion, tool_use — muse proposes, cortex decides (forbidden: final_decision / repo_action / security_decision, so muse calls tools to RESEARCH a proposal, never to enact one). Alias model=muse. It is an opt-in core role (OPT_IN_CORE_ROLES): machine-as-brain NEVER hosts it — a 31B cannot co-reside with the cortex+senses duo on a 128 GB box — so only an explicit muse-hosting shape (thor-muse, below) serves it; every non-hosting shape renders nothing for muse — the card's own declaration passes through, so machine-as-brain stays a byte-identical no-op and only base.toml's veto emits MUSE_FEASIBLE=false — and on a stale/pre-muse .env an unwired muse defaults to infeasible (OPT_IN_BACKENDS — model=muse 404s role_infeasible, referable/proxyable, never a silent fallback to cortex). Under pressure muse sheds (429) exactly like cortex/senses/worker. Operator decision (thor-worker-lobe plan): no box in the mesh currently hosts muse — the Jetson AGX Thor, the one box that ran thor-muse (DECLARED/UNVALIDATED: the 2026-07-17 live boot measured the budget, util 0.55 at the full 262144 window, but the acceptance transcript never landed, issue #108), moved to hosting worker instead (below). The deployed Thor declares no MUSE_PEER_ORIGIN, so model=muse now 404s role_infeasible with no hosted_by referral anywhere in the mesh. The muse role, its catalog entry, and the thor-muse shape all stay in-tree (cite-don't-delete) — dormant, not deleted — and the tier vocabulary above still ranks worker < muse. See docs/gemma-4-31b-nvfp4.md.

worker — the eighth role (opt-in hosting), the fast ground-work DOER. Checkpoint: unsloth/Qwen3.6-35B-A3B-NVFP4 (Qwen3.6 35B-A3B, an MoE with ~3B active parameters per token, 262144 native context, compressed-tensors NVFP4, and — unlike the catalog's other 35B-A3B candidate, mmangkad/Qwen3.6-35B-A3B-NVFP4, whose MTP never loads — ships its OWN self-hosted MTP draft module baked into the checkpoint). Served MULTIMODAL (image+video intake via the checkpoint's own ViT, no audio; no --language-model-only) — a "seeing doer", distinct from senses, which perceives but must never act. Responsibilities: execution, ground_work, bulk_transform, drafting, image_understanding, video_understanding, tool_use, and repo_action — worker is the FIRST role besides cortex permitted to act on the repo, under cortex's direction (forbidden: final_decision, security_decision — worker never makes the final call or a security decision on its own authority). Alias model=worker. It is the second opt-in core role (OPT_IN_CORE_ROLES = ("muse", "worker")): machine-as-brain NEVER hosts it, the gateway wires its backend only behind WORKER_BASE_URL, and an unwired worker defaults to infeasible (OPT_IN_BACKENDS — model=worker 404s role_infeasible, never a silent fallback), mirroring muse's mechanics exactly (WORKER_FEASIBLE / WORKER_PEER_ORIGIN / WORKER_PEER_PROXY / WORKER_PEER_API_KEY; base.toml vetoes it on an unrecognised card just like muse). Under pressure worker sheds (429) exactly like cortex/senses/muse.

The thor-worker deployment shape has LANDED and is VALIDATED on the physical Jetson AGX Thor (sm_110), 2026-07-31 — docs/evidence/2026-07-31-accept-worker-thor.txt. It drops BOTH cortex and senses and hosts worker + the pooling gears + the audio overlay, mirroring thor-muse's structure. Its budget is measured, not declared: gpu_mem_util=0.45 at the full max_model_len=262144 (no trim), weights 24.81 GiB loaded in ~31 s, KV pool 41.78 GiB, ~50.8 tok/s decode single-stream, MTP self-draft acceptance 89.1%.

Concurrency numbers in this repo are KV-pool CEILINGS, not measured throughput. KV pool / max_model_len gives 14.07× for this lane — how many full-context requests the cache could hold. Two independent consumers measured the real behaviour on 2026-07-31: usable concurrency saturates near width 8–9, and per-stream decode falls to ~30 tok/s at high width (embodiment; colleague#361 measured 29.8 tok/s/stream and 268.1 tok/s aggregate at width 14 — a 5.5% aggregate gain for a 75% load increase over width 8). Multiplying a single-stream tok/s by the ceiling is the misreading to avoid. The same caveat applies to every Nx concurrency figure below. The 0.45 hypothesis booted first try — the MoE's ~3B active-parameter footprint is why. --moe-backend is deliberately NOT forced on sm_110: every forced value was refused live (flashinfer_b12x/flashinfer_cutlass are sm_121a/Spark-only, marlin rejects the unquantized MTP experts, triton rejects NVFP4 MoE) — only auto-select boots, so the committed compose omits the flag and WORKER_MOE_BACKEND is an opt-in hand-pin. See docs/qwen3.6-35b-a3b-nvfp4.md.

The Spark reaches it by proxy (2026-07-31): WORKER_PEER_ORIGIN + WORKER_PEER_PROXY on the spark-lobe box forward model=worker to the Thor and relay the answer with X-Lobes-Proxied-By, image input included — so callers address worker on their local gateway and never dial the Thor directly. That required a fix in 0.54.8: worker was wired into _config.py's peer dicts in 0.54.6 but missing from server.py's _PEER_SERVED_NAME_ENV / _PEER_ROLE_HINT, so peer_specs_from_table silently dropped it and WORKER_PEER_PROXY=true did nothing. See docs/evidence/2026-07-31-accept-worker-proxy-spark.txt.

An opt-in realtime audio overlay (lobes init --fleet --audio) adds an OpenAI /v1/audio/* facade — a realtime bridge container (shipped in the wheel as lobes.realtime) that the gateway fans /v1/audio/* out to — backed by two fixed GPU sidecars: Parakeet STT (nvidia/parakeet-tdt-0.6b-v2, NeMo ASR → POST /v1/audio/transcriptions) and Chatterbox TTS (Resemble AI, 0.5B, Apache-2.0 → POST /v1/audio/speech, 24 kHz, zero-shot voice cloning; it replaced the retired Magpie NIM — no NGC key). These two are hardcoded, not in the switchable catalog (lobes/catalog.py).

GET /v1/realtime — the server_vad WebSocket session (issue #149). Beyond the batch facade, one WebSocket connection replaces separate STT calls with a persistent session: stream PCM16 mono little-endian in (24000 Hz default, 16000 Hz accepted — the server resamples down to 16 kHz itself) and receive session.created / input_audio_buffer.speech_started/...speech_stopped / conversation.item.input_audio_transcription.completed / error events back on the SAME connection (event schema + config parsing in lobes/realtime/_session.py; the server_vad segmenter — a pure state machine, Silero injected as a callable — in lobes/realtime/_segmenter.py; the thin FastAPI route wiring both to real Silero + scipy in lobes/realtime/app.py). A never-silent turn force-commits at VAD_MAX_TURN_MS (default 30s) with reason="max_turn" — a normal boundary event, never an error (true of the server's behaviour from the start, but not observable by any client until #151 put reason on the wire). Sessions are ephemeral (no resume; any disconnect tears the session down and a reconnect gets a brand-new session id) and reached through the gateway (101-upgrade + byte tunnel, lobes/gateway/_realtime.py) under the same opt-in GATEWAY_API_KEY bearer gate; a declared-off stt lane 404s role_infeasible and the session is never proxied cross-box (the #129 proxy-lobes forwarder is POST-only). This redeems two in-tree IOUs — app.py's own "PR2 adds the /v1/realtime WebSocket route" docstring promise, and realtime-pipeline.md's former "planned for a later release" boundary claim — against the #149 baseline probe (the deployed facade served four batch routes and no WebSocket, forcing reachy-mini-cli's client-side energy-threshold endpointing). The session mechanism is VALIDATED live on the DGX Spark GB10, 2026-07-21 (docs/evidence/2026-07-21-accept-realtime-spark.txt): a full session through the gateway tunnel against real Silero + Parakeet, at both 24000 Hz and the 16000 Hz passthrough. The live run is also what caught the tunnel's leftover-direction bug — the bridge's first frame was being sent back upstream, killing every session the instant it opened, and the unit test had asserted that wrong direction as correct. That transcript predates the #151 wire change below, so it proves the tunnel, the VAD and the STT forward — not the wire the server speaks today. Still UNVALIDATED, and NOT retired by #151: a real microphone (the runs used synthesized audio), the VAD-unavailable path, concurrent sessions, and the max-turn cap.

Voice-to-voice on the same socket (issue #151) — a coordinated wire break plus an opt-in conversation surface. Two changes land together, and the first is a break: raw binary audio frames are gone. Audio now travels as OpenAI-shaped base64 JSON events in BOTH directions — input_audio_buffer.append inbound, response.audio.delta outbound (codec: lobes/realtime/_wire.py). A binary frame now yields the new named invalid_wire_event error instead of being read as audio. The deployed reachy-mini-cli speaks the old wire and cannot stream until it adapts (tracked in reachy-mini-cli#115); that is a recorded, operator-accepted decision (frame decision c40), not a regression. Second, conversation is opt-in: a session answers nothing until the client sends response.create, and one that never does emits exactly the #149 transcription-only sequence (the ears-only contract reachy depends on — a structural property of _conversation.py, where every floor call sits behind if self.armed). Once armed, a committed turn is generated through the gateway's own /v1/chat/completions (voice lane defaults to multimodal, ~1 s to a short reply; OPENAI_MODEL overrides; a lane this box does not host surfaces as generate_failed with hosted_by, never a silent fallback), synthesized by Chatterbox and streamed back as 24 kHz deltas that never resample. Speaking during playback is meant to interrupt: generate and TTS are both cancelled, the undelivered remainder is never sent, and response.interrupted goes out — only the plausibly-heard prefix enters history. That contract is implemented in _floor.py and offline-proven, but the 2026-07-22 live run found it inert: the route delivered audio as fast as the socket drained (measured 2–4 ms for 7.5–8.5 s of speech), so the floor left SPEAKING before the user had heard two words and a barge-in landed while it was already LISTENING — a new turn opened and response.interrupted never fired. 0.54.1 paces delivery to the playhead (delivery_pause_ms, ≤ 400 ms lead) to close that gap; barge-in itself is still UNVALIDATED live (#108). New machinery: _floor.py (explicit floor/turn state machine, per-stage 60 s deadlines → response_timeout rather than a wedged session), _turn.py (generate payload shaping), per-session in-memory history + system prompt, and per-lane TTS pools so a spoken reply never queues behind batch TTS. New knobs: BARGE_IN_WINDOW_MS (750; a guard window, not a delay), BARGE_IN_MODEL (threaded but unconsumed — window-only barge-in ships), TTS_VOICE_CONCURRENCY (1), DEFAULT_SYSTEM_PROMPT. Boundary events now carry at_ms and reason, so VAD tuning is observable live. A local-only Astro harness under site/ drives all of it from a browser (real mic, live event stream, audio out) through a local credential-injecting WebSocket proxy reached via ssh -L — never deployed; CI only builds it. PARTIALLY validated (#108): every decision lives in stdlib modules the offline suite covers and app.py stays a pragma: no cover shell. The live acceptance transcript has now landed (docs/evidence/2026-07-22-accept-realtime-voice-to-voice-spark.txt, 2026-07-22, DGX Spark GB10) and it is split — voice-to-voice works end-to-end (armed session → generate → Chatterbox reply streamed back as deltas), and barge-in does not, for the delivery-pacing reason above. Treat that transcript as evidence for the conversation loop only; it predates the 0.54.1 pacing fix, so it is not evidence for barge-in either way. Still UNVALIDATED: barge-in (re-run needed against 0.54.1), a real microphone (the run used synthesized audio), concurrent sessions, and the VAD-unavailable path. On muting, note the deliberately narrow rule (deviation d1): automatic mute-during-playback stays FORBIDDEN — it is the AEC substitute that makes barge-in impossible, since you cannot interrupt a machine that has stopped listening — while user-initiated mute/mic-off is allowed, because AEC is genuinely owned at the client edge (Reachy firmware, browser echoCancellation).

See docs/realtime-pipeline.md, docs/parakeet-stt.md, docs/chatterbox-tts.md, docs/gateway-fleet.md (the realtime lane), and docs/openai-api.md (the full OpenAI-compatible endpoint surface, including /v1/realtime). lobes explain realtime / api are the in-CLI versions.

Machine profiles and supported hardware

lobes runs the fleet with knob values tuned to the hardware it lands on. Machine profiles — built-in TOML declarations in lobes/profiles/builtin/ — declare per-role models, context lengths, GPU memory budgets, attention backends, and vLLM knobs. lobes init auto-detects the card via nvidia-smi + hostname, resolves a profile by name, and renders it to env vars the compose template substitutes at startup.

Validated support:

card	profile	status	validation
DGX Spark (Grace Blackwell, 128 GB unified)	spark	load-tested	2026-06-03 — fleet duo (cortex 128K + senses 32K) serves at ~7.8–8.0 tok/s decode (27B primary, util 0.30) with FlashInfer attention. The correctness probes postdate that run and are unverified on the GB10 (rerank ordering: issue #106). See docs/tuning-profiles.md.
Jetson AGX Thor (Blackwell-class sm_110, 128 GB unified)	thor	load-tested	2026-07-13 — the three correctness probes pass (cortex known-answer, embed ranking, rerank ordering) with four validated divergences: cortex kv_cache_dtype=auto (uncalibrated-fp8 exposure, #109), embedder/reranker attention_backend=TRITON_ATTN (FLASH_ATTN pooling broken on sm_110, #105), reranker enforce_eager=true (CUDA graphs unstable on sm_110). Concurrent first boot can fail on a memory race — see the boot-ordering caveat in docs/machine-profiles.md.
unknown card	base	conservative fallback	— small 4B model, no 27B, no multimodal (senses disabled) to avoid OOM on first boot. Resolved when card detection returns UNKNOWN. See issue #107 (broader tuned-small-model work, future).

Custom profiles: operator-defined TOML files in <deploy-dir>/profiles/<name>.toml override built-ins by name. See docs/machine-profiles.md#writing-your-own-profile for the format, and lobes explain profiles for the brief reference.

See also: docs/machine-profiles.md (the deep reference: detection flow, knob meanings, Thor's validated divergences, custom profiles, goldens contract); lobes explain profiles / lobes explain tuning (in-CLI).

Deployment shapes

Orthogonal to the machine-profile axis above (how a role is tuned on a card) is the deployment-shape axis (issue #113): which of the eight Colleague roles a box hosts at all, composed as pure data over the card profile at render time (lobes/profiles/shapes.py, shape_render.py). machine-as-brain (the default — bare lobes init, unchanged, zero new decisions) hosts every role a card can serve; the four core roles stay default-on by machine-as-brain, not unconditionally — a mesh-brain shape drops one heavy generate lobe to a peer box via a generated docker-compose.shape.yml override (the base fleet template itself stays unconditional). Two mesh-lobe shapes are validated live (2026-07-14): spark-lobe (DGX Spark GB10 — drops senses, cortex reclaims to gpu_mem_util=0.44 / max_model_len=262144, measured KV pool 888,946 tokens / 3.39× concurrency at full 256K) and thor-lobe (Jetson AGX Thor — drops cortex, senses reclaims to gpu_mem_util=0.30 / max_model_len=131072, measured KV pool 1,418,554 tokens / 10.82× concurrency at 131072). Both reclaim values are measured, not computed — the naive reclaim-sum/solo-default was refused by vLLM on the live, unified-memory box in each case. A fourth built-in shape, orin-small (mesh-brain end-state, issue #112, t2), drops BOTH heavy lobes and hosts the opt-in minor gear (vllm-minor) instead, alongside the pooling gears and audio overlay — it ships as declared, UNVALIDATED data only (the #108 rule: no physical Jetson AGX Orin has booted it, so no doc, support table, or lobes capabilities output may claim it validated). A fifth built-in shape, thor-muse, likewise drops BOTH heavy default lobes and instead hosts the opt-in muse lobe (vllm-muse, Gemma 4 31B) plus the pooling gears and audio overlay — muse is an opt-in core role (OPT_IN_CORE_ROLES): hostable only by an explicit shape, with the full muse declaration (model + budget knobs) in the shape's own overrides, the card profiles silent on it, and base.toml vetoing it; thor-muse too is declared, UNVALIDATED — its budget values (gpu_mem_util=0.55, max_model_len=262144 — the full 256K window) were measured by the 2026-07-17 live boot on a physical Thor (the 0.40 hypothesis was refused), and it stays UNVALIDATED until the acceptance transcript lands under docs/evidence/ (#108). thor-muse is now DORMANT/unhosted — the physical Thor moved to hosting worker instead (below) and no box declares MUSE_PEER_ORIGIN — but the shape file, its TOML, and its goldens stay in-tree unchanged (cite-don't-delete); nothing here or in lobes capabilities claims a box currently renders it. worker — the eighth Colleague role — introduced a second opt-in core role on the same mechanism: the thor-worker shape drops BOTH cortex and senses and instead hosts the opt-in worker lobe (vllm-worker, Qwen3.6-35B-A3B) plus the pooling gears and audio overlay, mirroring thor-muse's structure exactly (OPT_IN_CORE_ROLES = ("muse", "worker"), base.toml veto, shape-owned override declaration). It has LANDED and is VALIDATED on the physical Jetson AGX Thor, 2026-07-31, with measured values — gpu_mem_util=0.45 at the full max_model_len=262144, KV pool 41.78 GiB = 14.07x concurrency, MTP acceptance 89.1% (docs/evidence/2026-07-31-accept-worker-thor.txt); the 0.45 hypothesis booted first try, unlike thor-muse's refused 0.40. Select with lobes init --shape <machine-as-brain|spark-lobe|thor-lobe|orin-small|thor-muse|thor-worker> (dry-run by default, --apply to commit, byte-for-byte restorable by re-running with the previous shape). A dropped role is flagged feasible:false on both lobes capabilities and GET /capabilities, omitted from /v1/models, and 404s role_infeasible on every alias — never half-served. Opt-in honest referral (issue #112, t3): declaring a peer origin per dropped role (PRIMARY_PEER_ORIGIN / MULTIMODAL_PEER_ORIGIN / MUSE_PEER_ORIGIN / WORKER_PEER_ORIGIN / EMBED_PEER_ORIGIN / RERANK_PEER_ORIGIN — plus, since #129, the first-class audio lanes STT_PEER_ORIGIN / TTS_PEER_ORIGIN, declared off per-lane with STT_FEASIBLE/TTS_FEASIBLE=false — always operator-typed, never derived, per #92) makes both capabilities surfaces and the role_infeasible 404 body name the hosting peer (hosted_by); by default this is annotation only — the gateway does not forward a request to a peer on the origin declaration alone, and with no peer config every response is byte-identical to the pre-referral contract. A box can opt into actually following its own referral — see proxy-lobes, next.

The mesh-brain end-state (issue #112) — one heavy lobe per box, cheap gears co-reside, the brain stays whole across the mesh — has landed on top of the near-term work above, recording four decisions: (1) cross-box reachability is direct addressing + opt-in honest referral by default, now with an opt-in proxy extension (below); (2) the cheap gears (embedder/reranker/stt/tts) co-reside on every box that wants them — no gear is forced to move; (3) the reference shape assignment is Spark GB10 = cortex via spark-lobe, Thor 128GB = senses via thor-lobe, Orin 64GB = small-model lobes via orin-small; and (4) the shape axis is mixable — specialized, multi-role, and mixed boxes (local or cloud) compose into one brain, with machine-as-brain staying the default and one-box users unaffected. The referral surface is live-validated cross-box on the physical Thor (docs/evidence/2026-07-14-accept-referral-thor.txt); physical Orin validation remains open. See docs/deployment-shapes.md (the deep reference) and lobes explain shapes (in-CLI).

Proxy-lobes (issues #115/#127, phase 1 — landed on top of referral). A dropped role can go beyond referral-only to a third state, proxy: this box forwards the request to its declared peer instead of 404ing, so the caller never has to know it moved. Two knobs, both opt-in and both required together: <PREFIX>_PEER_PROXY=true (arms the forward; inert without a declared <PREFIX>_PEER_ORIGIN) and <PREFIX>_PEER_API_KEY (the outbound credential — always a copy of the peer's own inbound GATEWAY_API_KEY, never a value minted per pairing, so key material scales O(machines), not O(pairs)). The caller's own Authorization (validated by this box's own opt-in GATEWAY_API_KEY inbound gate) is stripped before every forward and never reaches a peer. Proxying is single-hop — a request that arrives already marked X-Lobes-Proxied is refused (508 proxy_loop) rather than re-forwarded — and every proxied answer carries X-Lobes-Proxied-By: <peer origin> so a caller can always tell a forwarded answer from a locally-served one. Peer origins are assumed reachable over a private/tailnet transport, never the public internet (no TLS termination happens at this layer). With no <PREFIX>_PEER_PROXY set anywhere — every pre-#115 deployment — every response stays byte-identical to the pre-proxy contract.

Live as of 2026-07-31: the DGX Spark (spark-lobe) proxies TWO roles — senses → the AGX Orin and worker → the Jetson AGX Thor — so a caller addresses either on the Spark's own gateway and never dials the peer box. Both answer 200 with X-Lobes-Proxied-By, image input included. Note a proxied role reports feasible: false by design (it means "this box does not host it", not "you cannot use it here"); proxied: true + hosted_by + ready are the fields that say it is usable, and loaded is a wiring fact, not a running one. See docs/gateway-fleet.md#proxy-lobes-the-third-lobe-state-opt-in and docs/deployment-shapes.md#following-the-referral-proxy-lobes-opt-in.

Deployment model

lobes is scaffold-based, not checkout-based. The canonical docker-compose.yml + env.example are packaged under lobes/templates/ and shipped in the wheel. lobes init materialises them into a deployment dir — default ~/.lobes, or a TARGET path, or . for the local folder. Every model-ops verb resolves the deployment dir as: --compose-dir → $LOBES_DIR → ~/.lobes, falling back to the legacy $MODEL_GEAR_DIR → ~/.model-gear when those are set / already scaffolded (so a pre-rename deployment keeps working). There is no compose file at the repo root.

CLI surface

lobes/                 # Python package (pip install lobes-cli)
├── __init__.py             # __version__ via importlib.metadata("lobes-cli")
├── __main__.py             # python -m lobes
├── assess.py               # correctness probes + throughput/prefill (stdlib urllib)
├── catalog.py              # the supported-model catalog (the switchable "gears")
├── templates/              # packaged docker-compose*.yml + env.example + Dockerfiles (lobes init)
├── runtime/                # _env (.env r/w) · _compose (dir resolve + docker) · _health · _tunnel (cloudflared)
├── gateway/                # stdlib OpenAI-compatible reverse proxy (the fleet front)
├── realtime/               # /v1/audio/* facade: bridge · tts_client · chatterbox_server · _readiness
├── explain/                # markdown catalog for `lobes explain <path>`
└── cli/
    ├── __init__.py         # argparse main(); registers every verb
    ├── _errors.py          # ModelGearError + EXIT_USER_ERROR / EXIT_ENV_ERROR
    ├── _output.py          # strict stdout/stderr split; --json result emitter
    ├── _runtime_ops.py     # shared glue (deployment dir, port, compose_check)
    └── _commands/          # one module per verb: register(sub) + handler
        ├── switch.py serve.py stop.py status.py assess.py benchmark.py init.py fleet.py
        └── logs.py tunnel.py whoami.py learn.py explain.py overview.py doctor.py cli.py

Lifecycle (turn on / off): lobes serve (alias start) brings the default deployment up (docker compose up -d, then waits for /health) — since #69 lobes init/serve default to the main + multimodal duo (the legacy single-model scaffold is opt-in via lobes init --single/--legacy). lobes stop takes it down (docker compose down — it removes the containers, not a pause). The fleet lane mirrors this: lobes fleet up (up -d --build) / lobes fleet down. lobes switch <model> is a down+up with a model swap. lobes status / lobes fleet status observe without mutating.

Mutation safety: write verbs (switch, serve, stop, init, fleet up, fleet down, tunnel) default to dry-run; require --apply to commit. Agents call CLIs in loops, so safe-by-default is mandatory. The read-only verbs (status, assess, benchmark, logs, overview, whoami, explain, doctor) never change the world — with one opt-in exception: doctor --fix is doctor's write lane (#119), and it follows the same convention (--fix alone prints the missing-only heal plan; --fix --apply writes absent files/keys, never rewriting an existing .env line).

Build / test / publish

• Install for dev: uv sync
• Run CLI from source: uv run lobes --version / uv run python -m lobes whoami
• Tests (all): uv run pytest -n auto -v
• Single test: uv run pytest tests/test_cli_runtime.py::test_name -v
• Lint: uv run black --check lobes tests, uv run isort --check-only lobes tests, uv run flake8 lobes tests, uv run bandit -c pyproject.toml -r lobes
• Rubric gate: uv run afi cli doctor . --strict (CI blocks merge if it fails).
• Version bump (required every PR): python3 .claude/skills/version-bump/scripts/bump.py {patch|minor|major} — updates pyproject.toml and prepends a CHANGELOG entry. The version-check CI job fails the PR if the version equals main's (AgentCulture every-PR-bumps rule — no exceptions, even for docs/config-only changes). Version is the single source of truth in pyproject.toml; lobes.__version__ is read from package metadata at import.
• Publish: push to main → publish.yml builds with uv build and publishes lobes-cli to PyPI via Trusted Publishing (no API tokens); model-gear is published as a deprecated alias that redirects to lobes-cli. PRs publish a .dev<run_number> to TestPyPI. Fork PRs are skipped (no OIDC).

Skills convention

Six skills are vendored from steward (the canonical upstream) under .claude/skills/<name>/: cicd, communicate, version-bump, run-tests, sonarclaude, doc-test-alignment. This is cite-don't-import: copies are owned by this repo and may diverge from steward.

Three more are vendored from agentculture/devague (re-broadcast via guildmaster) — the idea→spec→plan→implementation operator chain for the deterministic devague CLI: think (idea→spec), spec-to-plan (spec→plan), and assign-to-workforce (plan→parallel implementation). These three carry type: command in their frontmatter — load-bearing on the culture/agex backend (a SKILL.md without type: is silently skipped when the repo declares an agent in culture.yaml). They depend on the devague CLI at runtime (uv tool install devague), resolved portably by the wrappers.

One skill is local to this repo (not vendored): model-runner — a thin pointer/shim to the lobes CLI for switching/serving/assessing the model. The real implementation is the lobes package; the shim execs lobes.

The provenance of every vendored skill (citation path + authoring origin) is recorded in docs/skill-sources.md.

Each skill ships:

1. SKILL.md — why and when to use it (frontmatter name must equal the directory name; short prose, no inline 10-step walk-throughs).
2. scripts/<entry-point> — the script that automates the workflow.
3. No external path dependencies. Scripts must not reach outside this repo.

Per-machine paths live in .claude/skills.local.yaml (git-ignored); a committed .claude/skills.local.yaml.example documents every key. Skills read the local file and fall back to the example. (The Culture posting nick is lobes — the deployed agent shares the repo/tool name.)

PR workflow

Every task gets its own branch and PR. Before merging:

1. Wait for all reviewer comments (Qodo, Copilot, humans).
2. Fix valid findings — commit to the same branch.
3. Push back on invalid findings — reply with reasoning.
4. Reply to every thread (fix confirmed or pushback explained).
5. Resolve all threads. Never merge with unaddressed review comments.

Bump the version (above) on every PR or CI's version-check job fails the run.

Working with the mesh from here

• Culture CLI: culture — server lifecycle, agent start/stop, mesh linking. Path references assume siblings are checked out alongside this repo (../culture, ../daria, ../steward).
• steward owns the six steward-sourced skills (the devague trio is owned upstream by agentculture/devague; see docs/skill-sources.md) and the sibling-pattern contract. steward files issues on siblings but never edits them — so scaffolding and alignment work for this repo happens in this repo. steward's docs/skill-sources.md "Downstream copies" column may still list this repo under a retired name (lepenseur or model-gear); fixing that is a PR on steward, not an edit from here.

Conventions and workflow

Memory discipline — recall before, remember after. This repo keeps its eidetic memory in-repo and public: records resolve to <repo-root>/.eidetic/memory — committed, and shared with the team and mesh peers (the claude and colleague backends both read the same lobes scope), so memory travels with the repo, not a private home-dir store. Make it a per-task habit:

• /recall before you start. Search the store for the area you're about to touch — prior decisions, gotchas, "have we done this before?" — so you build on what's already known instead of re-deriving it. Do this before non-trivial tasks, not just when asked.
• /remember when something worth keeping surfaces. A non-obvious decision and its rationale, a constraint, a fix and why it was needed, a gotcha that cost time, a fact the next session would otherwise re-learn. Capture it as it happens, not at the end when it's faded.

A plain /remember lands the note in ./.eidetic/memory in this repo — no flag needed (the wrappers here default to --visibility public; in-repo routing needs eidetic >= 0.10.0, older CLIs keep records in $HOME). Keep something out of the committed store only by passing --visibility private (routes to $HOME/.eidetic/memory, never committed); /recall reads both stores and merges. Don't store what the repo already records (code structure, git history, what's already in this file or CHANGELOG.md) — store what you'd have to re-derive. These are the recall/remember skills (.claude/skills/), backed by the eidetic store.