Skip to content

Commit 37740ec

Browse files
OriNachumclaude
andauthored
feat: add the minor lobe — warm co-resident Qwen3.5-4B (#64) (#65)
* spec: minor lobe — warm co-resident Qwen3.5-4B (#64) Converged devague /think spec for issue #64. Scope (user-confirmed) is the full minor-role runtime, not just a catalog row: - add Qwen3.5-4B as a role_hint=minor catalog gear, served bf16 WARM CO-RESIDENT behind the gateway - new read-only verbs: lobes run/route/eval, with route picking across lobes GEARS only (v1) plus an escalate flag + confidence - escalation + confidence governance (minor may prepare/classify/format/validate/ suggest; config-blocked from approve/finalize/delete/deploy/architect) - bf16 base chosen to ENABLE later unsloth-LoRA fine-tuning (training deferred) Surfaces two code constraints beyond the catalog edit: _parser.py must learn the qwen3.5 family (qwen3_coder, not hermes), and the catalog needs a convention for an unquantized bf16 generate gear. Verified model facts (hybrid linear-attn+ViT, multimodal, built-in MTP head, 256K, Apache-2.0) folded in. Includes the converged frame state. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * eidetic-memory: minor-lobe spec findings (#64) Records the verified Qwen3.5-4B HF facts, the minor-lobe spec decisions, and a devague tooling gotcha (rejecting a claim does not clear its blocking hard question; gate needs honesty on every spec-affecting claim). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * plan: minor lobe — Qwen3.5-4B build plan (#64) Forward leg (devague /spec-to-plan) from the converged minor-lobe spec. 11 TDD- gated, file-disjoint tasks in 5 dependency waves covering all 26 spec targets: wave 1 t1 catalog+bf16 convention · t2 parser rule · t3 minor urllib client wave 2 t4 governance/escalation · t5 run verb · t7 eval verb · t9 co-residency wave 3 t6 route verb · t10 docs wave 4 t8 wire run/route/eval into the CLI (sole owner of cli/__init__.py) wave 5 t11 end-to-end signals + read-only safety verification Risks: GB10 boot/NVFP4 quality untested (CI checks config only); confidence- metric source is a defined v1 heuristic; train/eval-contents/MTP-spec-decode deferred. Ready for /assign-to-workforce. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(parser): recognize Qwen3.5 family as qwen3_coder (t2, #64) * feat(minor): stdlib urllib client for the minor backend (t3, #64) Add lobes/minor/ package with chat_completion() and chat_text() helpers that POST OpenAI chat-completions requests using only urllib/json (no third-party deps). Tests use a real ephemeral HTTPServer on 127.0.0.1:0. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(minor): governance + escalation model for the minor role (t4, #64) * feat(cli): lobes run minor verb (t5, #64) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(cli): lobes eval minor verb (t7, #64) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(catalog): add Qwen3.5-4B minor gear + bf16 'none' convention (t1, #64) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * test(run): reconcile catalog-minor assumption after t1 merge (#64) * feat(cli): lobes route verb across gears (t6, #64) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(fleet): opt-in co-resident minor backend + gateway routing (t9, #64) Add a `vllm-minor` compose service (Qwen/Qwen3.5-4B, bf16) behind `profiles: [minor]` so a default `docker compose up` never starts it. Wire gateway routing to route `model: Qwen/Qwen3.5-4B` to the minor backend when MINOR_BASE_URL / MINOR_SERVED_NAME are set (env-gated via the same `_optional_backend` pattern as fallback/embed/rerank). Without those env vars the routing table is exactly as before. Details: - lobes/templates/fleet/docker-compose.yml: vllm-minor service with profiles: [minor], container_name model-gear-vllm-minor, healthcheck, --language-model-only (text-only, drops ViT), NO --quantization (bf16), ${VLLM_MINOR_MAX_MODEL_LEN:-32768} / ${VLLM_MINOR_GPU_MEM_UTIL:-0.10}. Gateway env gains MINOR_BASE_URL / MINOR_SERVED_NAME with empty defaults (opt-in gate: both empty → no backend added). - lobes/gateway/_config.py: _DEFAULT_MINOR constant + _optional_backend call (url_key=MINOR_BASE_URL, name_key=MINOR_SERVED_NAME, task=generate). - tests/test_fleet_minor.py: 17 new tests — gateway routing (TDD-first: opt-in gate, URL normalisation, resolve_model, order_backends failover to primary, embed-task isolation) and compose template assertions (vllm-minor present, profiles, --language-model-only, no --quantization, container_name, healthcheck, always-on services unchanged). 453 passed, 3 skipped (pre-existing numpy skips). No regressions. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(cli): wire run/route/eval verbs + align eval endpoint (t8, #64) Register run/route/eval in _build_parser(); change eval _DEFAULT_BASE_URL from port 8001 to 8000 (the gateway) to match run.py and route.py; add tests/test_cli_wiring.py asserting each verb is reachable and dispatches to the correct handler. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * docs: minor lobe governance, co-residency, fine-tune target (t10, #64) * docs(minor): align eval base-url to gateway :8000 after t8 (#64) * test: minor-lobe end-to-end integration + read-only safety (t11, #64) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * chore: bump version 0.29.0 -> 0.30.0 (minor lobe, #64) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(route): disable thinking + cap max_tokens so route doesn't time out (#64) Live-surfaced on the GB10: the 4B in thinking mode emitted a long <think> trace before the routing JSON, blowing past the client's 60s timeout. Add an extra_body passthrough to the minor client and have route disable thinking (chat_template_kwargs.enable_thinking=false) + cap max_tokens=512. Verified live: terse parseable decision; governance escalation intact. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(minor): note live GB10 verification + serving flags (#64) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * chore: sync uv.lock to 0.30.0 (#64) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * eidetic-memory: Qwen3.5 vLLM 0.19 serving finding (#64) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * style(run): merge implicit string concat in --model help (#64) Addresses a review finding (ISC001 same-line literal concatenation). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(minor): address Qodo review findings on #65 (#64) - route low-confidence escalation (qodo 3481974171): governance gains an UNCERTAINTY_THRESHOLD (0.25, per issue #64); decide() escalates when confidence is below it; route passes confidence through. - route escalation bypass (qodo 3481974173): normalize the model's conditions (str -> [str], strip/lower; no char-splitting), clamp chosen_gear to known roles, and fail closed on a malformed conditions field. - bf16 notice on flag (qodo 3481974176): the REMOVE --quantization compose-edit notice now fires on the effective quantization choice — explicit --quantization none on an uncatalogued model included, not just catalog metadata. - missing eval suite (qodo): ship examples/minor-evals/smoke.jsonl and point the doc's `lobes eval minor` example at it. Regression tests added for each. Full suite 504 passed; flake8/black/bandit clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
1 parent 03817a3 commit 37740ec

37 files changed

Lines changed: 4942 additions & 15 deletions

.devague/current

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1 +1 @@
1-
model-gear-now-serves-an-embedding-gear-qwen3-embe-2
1+
lobes-ships-a-minor-lobe-a-cheap-warm-co-resident

.devague/current_plan

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1 +1 @@
1-
model-gear-now-serves-an-embedding-gear-qwen3-embe-2
1+
lobes-ships-a-minor-lobe-a-cheap-warm-co-resident
Lines changed: 314 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,314 @@
1+
{
2+
"slug": "lobes-ships-a-minor-lobe-a-cheap-warm-co-resident",
3+
"title": "lobes ships a 'minor' lobe: a cheap, warm co-resident Qwen3.5-4B small-brain served bf16 alongside the 27B primary, driven by lobes run/route/eval with escalation + confidence governance, and chosen as the first unsloth-LoRA fine-tune target (training itself comes later)",
4+
"schema_version": 1,
5+
"status": "exported",
6+
"created": "2026-06-26T09:21:29Z",
7+
"updated": "2026-06-26T09:42:54Z",
8+
"claims": [
9+
{
10+
"id": "c1",
11+
"kind": "announcement",
12+
"text": "lobes ships a 'minor' lobe: a cheap, warm co-resident Qwen3.5-4B small-brain served bf16 alongside the 27B primary, driven by lobes run/route/eval with escalation + confidence governance, and chosen as the first unsloth-LoRA fine-tune target (training itself comes later)",
13+
"origin": "user",
14+
"status": "confirmed",
15+
"honesty_conditions": [
16+
{
17+
"id": "h1",
18+
"text": "every piece lands behind an EXISTING safety contract \u2014 read-only verbs (run/route/eval) and dry-run-by-default mutations (compose/switch) \u2014 not a new unguarded surface",
19+
"status": "confirmed"
20+
}
21+
],
22+
"hard_questions": [],
23+
"links": []
24+
},
25+
{
26+
"id": "c2",
27+
"kind": "audience",
28+
"text": "lobes maintainers and Culture-mesh agents; agent loops that need a cheap local model for high-frequency low-risk calls (classify, route, format, validate, summarize) instead of spending 27B-primary tokens",
29+
"origin": "llm",
30+
"status": "confirmed",
31+
"honesty_conditions": [
32+
{
33+
"id": "h6",
34+
"text": "the named reflex tasks (classify/route/format/validate/summarize) are within a 4B's reliable range; anything needing codebase context or judgment ESCALATES rather than silently degrading",
35+
"status": "confirmed"
36+
}
37+
],
38+
"hard_questions": [],
39+
"links": []
40+
},
41+
{
42+
"id": "c3",
43+
"kind": "before_state",
44+
"text": "lobes only switches/serves/assesses ONE served model; it has no notion of a cheap secondary worker, no co-resident small-brain, and no verb that invokes a model with a prompt (only assess.py's internal urllib correctness probes hit the model today)",
45+
"origin": "llm",
46+
"status": "confirmed",
47+
"honesty_conditions": [
48+
{
49+
"id": "h7",
50+
"text": "confirmed against the CLI surface: no existing verb invokes the model with a prompt (assess.py probes are internal), and the catalog has no role_hint=minor / co-resident generate gear today",
51+
"status": "confirmed"
52+
}
53+
],
54+
"hard_questions": [],
55+
"links": []
56+
},
57+
{
58+
"id": "c4",
59+
"kind": "why_it_matters",
60+
"text": "the architectural primitive becomes 'minor' (a role), not 'qwen4b' (a model): lobes stays model-agnostic while AgentCulture gets a practical cheap-local default and its first LoRA target, and reflex work stops burning the 27B",
61+
"origin": "llm",
62+
"status": "confirmed",
63+
"honesty_conditions": [
64+
{
65+
"id": "h8",
66+
"text": "the role abstraction holds \u2014 swapping the 4B for another small model later is a catalog edit only; the verbs target the 'minor' ROLE, not a model id",
67+
"status": "confirmed"
68+
}
69+
],
70+
"hard_questions": [],
71+
"links": []
72+
},
73+
{
74+
"id": "c5",
75+
"kind": "after_state",
76+
"text": "Qwen3.5-4B is a catalog gear with role_hint=minor, served WARM CO-RESIDENT behind the gateway; lobes overview --list and GET /v1/models/supported list it",
77+
"origin": "llm",
78+
"status": "confirmed",
79+
"honesty_conditions": [
80+
{
81+
"id": "h9",
82+
"text": "the new entry passes every tests/test_catalog.py invariant and appears in BOTH lobes overview --list and GET /v1/models/supported",
83+
"status": "confirmed"
84+
}
85+
],
86+
"hard_questions": [
87+
{
88+
"id": "q1",
89+
"text": "risk: lobes gaining run/route/eval shifts it from model-ops toward an agent-runtime \u2014 scope creep vs its stated identity ('runs, assesses, switches'); mitigate by keeping verbs THIN (prompt in / decision out), no agent loop, no persistent state",
90+
"resolved": false,
91+
"blocking": false
92+
}
93+
],
94+
"links": []
95+
},
96+
{
97+
"id": "c6",
98+
"kind": "after_state",
99+
"text": "new verbs: lobes run minor \"<prompt>\" invokes the minor model; lobes route \"<text>\" returns a routing decision; lobes eval minor --suite ... runs an eval; minor outputs carry confidence/uncertainty metadata",
100+
"origin": "llm",
101+
"status": "rejected",
102+
"honesty_conditions": [],
103+
"hard_questions": [
104+
{
105+
"id": "q2",
106+
"text": "what does 'route' decide BETWEEN \u2014 lobes gears, tools, or other agents/lobes? the routing-target taxonomy must be defined or the verb is underspecified",
107+
"resolved": true,
108+
"blocking": true
109+
}
110+
],
111+
"links": []
112+
},
113+
{
114+
"id": "c7",
115+
"kind": "after_state",
116+
"text": "governance: minor MAY prepare/classify/format/validate/suggest but is config-blocked from approve/finalize/delete/deploy/architectural decisions; escalation conditions (needs_codebase_context, security_sensitive, architectural_decision, write_or_delete_operation, final_review_required) route work upward",
117+
"origin": "llm",
118+
"status": "confirmed",
119+
"honesty_conditions": [
120+
{
121+
"id": "h10",
122+
"text": "the allowed/forbidden duty split and escalation conditions are ENFORCED in config (minor refuses/escalates a forbidden action), not merely documented prose",
123+
"status": "confirmed"
124+
}
125+
],
126+
"hard_questions": [],
127+
"links": []
128+
},
129+
{
130+
"id": "c8",
131+
"kind": "boundary",
132+
"text": "NOT actually fine-tuning yet \u2014 no training runs, datasets, or shipped adapters; bf16 base is chosen to ENABLE later unsloth LoRA. NOT making minor a reviewer/approver. NOT displacing the 27B primary. The model is an implementation detail of the 'minor' role",
133+
"origin": "llm",
134+
"status": "confirmed",
135+
"honesty_conditions": [
136+
{
137+
"id": "h11",
138+
"text": "the bf16-base decision is sufficient to LoRA-finetune later with unsloth without re-exporting the served checkpoint (bf16 base is the adapter target)",
139+
"status": "confirmed"
140+
}
141+
],
142+
"hard_questions": [],
143+
"links": []
144+
},
145+
{
146+
"id": "c9",
147+
"kind": "success_signal",
148+
"text": "lobes run minor returns a completion from the co-resident 4B; lobes route returns a structured decision with confidence; escalation thresholds are representable and tested; catalog + parser + afi cli doctor --strict tests pass; docs state allowed/forbidden minor duties and name Qwen3.5-4B as the first fine-tune target",
149+
"origin": "llm",
150+
"status": "confirmed",
151+
"honesty_conditions": [
152+
{
153+
"id": "h12",
154+
"text": "every listed signal is OBSERVABLE in CI/local (run returns a completion, route returns decision+confidence, catalog/parser/doctor tests pass), not subjective",
155+
"status": "confirmed"
156+
}
157+
],
158+
"hard_questions": [],
159+
"links": []
160+
},
161+
{
162+
"id": "c10",
163+
"kind": "decision",
164+
"text": "served checkpoint = Qwen/Qwen3.5-4B (bf16), chosen because unsloth LoRA needs the bf16 base; community cosmicproc/Qwen3.5-4B-NVFP4 is documented as an UNTESTED config alternative, not the default",
165+
"origin": "llm",
166+
"status": "confirmed",
167+
"honesty_conditions": [],
168+
"hard_questions": [],
169+
"links": []
170+
},
171+
{
172+
"id": "c11",
173+
"kind": "decision",
174+
"text": "role_hint=minor \u2014 a new behavior-free catalog label (only 'primary' is load-bearing in switch.py/MTP; overview prints role_hint, gateway routes by task not role)",
175+
"origin": "llm",
176+
"status": "confirmed",
177+
"honesty_conditions": [],
178+
"hard_questions": [],
179+
"links": []
180+
},
181+
{
182+
"id": "c12",
183+
"kind": "decision",
184+
"text": "warm co-resident (opt-in always-on second generate backend), not switch-only \u2014 served at a modest max_model_len and bounded gpu-mem so it does not crowd the 27B",
185+
"origin": "llm",
186+
"status": "confirmed",
187+
"honesty_conditions": [],
188+
"hard_questions": [],
189+
"links": []
190+
},
191+
{
192+
"id": "c13",
193+
"kind": "requirement",
194+
"text": "lobes/runtime/_parser.py learns the Qwen3.5 family so infer_parser(id) == the catalog tool_parser (the catalog test asserts equality), set to whatever tool format Qwen3.5-4B actually emits",
195+
"origin": "llm",
196+
"status": "confirmed",
197+
"honesty_conditions": [
198+
{
199+
"id": "h2",
200+
"text": "the parser is set from Qwen3.5-4B's REAL emitted tool format (verified, not assumed), and a _parser unit test pins infer_parser('Qwen3.5-4B') to that value",
201+
"status": "confirmed"
202+
}
203+
],
204+
"hard_questions": [],
205+
"links": []
206+
},
207+
{
208+
"id": "c14",
209+
"kind": "requirement",
210+
"text": "the catalog can represent an UNQUANTIZED bf16 generate gear: a quantization-field convention (a sentinel that lobes switch/compose translate to omitting --quantization) plus updated catalog tests, since today every generate entry requires a non-empty quantization",
211+
"origin": "llm",
212+
"status": "confirmed",
213+
"honesty_conditions": [
214+
{
215+
"id": "h3",
216+
"text": "a bf16 entry boots under the GB10 vLLM image with the chosen quantization convention (\u2014quantization omitted), and the updated catalog tests still pin every other invariant for the other gears",
217+
"status": "confirmed"
218+
}
219+
],
220+
"hard_questions": [],
221+
"links": []
222+
},
223+
{
224+
"id": "c15",
225+
"kind": "requirement",
226+
"text": "lobes run/route/eval reuse the existing stdlib-only urllib client pattern (assess.py) to call the OpenAI-compatible endpoint \u2014 no new heavy deps \u2014 and stay read-only per the mutation-safety contract",
227+
"origin": "llm",
228+
"status": "confirmed",
229+
"honesty_conditions": [
230+
{
231+
"id": "h4",
232+
"text": "run/route/eval add zero runtime deps beyond stdlib (matching assess.py) and pass the read-only / mutation-safety checks in doctor",
233+
"status": "confirmed"
234+
}
235+
],
236+
"hard_questions": [],
237+
"links": []
238+
},
239+
{
240+
"id": "c16",
241+
"kind": "requirement",
242+
"text": "warm co-residency: the fleet compose gains an opt-in 'minor' generate backend and the gateway can target minor vs primary for generate traffic",
243+
"origin": "llm",
244+
"status": "confirmed",
245+
"honesty_conditions": [
246+
{
247+
"id": "h5",
248+
"text": "the co-resident minor backend boots alongside the 27B on the GB10 within memory budget (modest max_model_len + bounded gpu-mem) without OOMing the primary",
249+
"status": "confirmed"
250+
}
251+
],
252+
"hard_questions": [],
253+
"links": []
254+
},
255+
{
256+
"id": "c17",
257+
"kind": "after_state",
258+
"text": "new verbs (read-only): lobes run minor \"<prompt>\" calls the co-resident 4B; lobes route \"<text>\" picks which GEAR handles a task (minor vs 27B primary vs candidate) AND whether to escalate, with a confidence score; lobes eval minor --suite ... runs an eval; v1 routes ONLY across lobes gears (not tools, not mesh agents)",
259+
"origin": "llm",
260+
"status": "confirmed",
261+
"honesty_conditions": [
262+
{
263+
"id": "h13",
264+
"text": "lobes route emits a structured, machine-parseable decision (chosen gear + escalate flag + confidence) validated by a test, so agent loops can consume it",
265+
"status": "confirmed"
266+
}
267+
],
268+
"hard_questions": [],
269+
"links": []
270+
},
271+
{
272+
"id": "c18",
273+
"kind": "decision",
274+
"text": "VERIFIED model facts (HF config.json): Qwen3.5-4B is hybrid linear-attn (Gated Delta/SSM) + full-attention (NOT MoE, so no moe_backend), multimodal (ViT image+video), built-in MTP draft head, 256K native (max_position_embeddings 262144), bf16, Apache-2.0, public; tool format = qwen3_coder (XML); served as a text-only reflex brain via --language-model-only to drop the vision tower",
275+
"origin": "llm",
276+
"status": "confirmed",
277+
"honesty_conditions": [],
278+
"hard_questions": [],
279+
"links": []
280+
}
281+
],
282+
"open_vagueness": [
283+
{
284+
"id": "v1",
285+
"text": "lobes train minor (unsloth LoRA/QLoRA): real training runs, datasets, adapter artifacts, adapter hot-swap/serving \u2014 deferred ('not yet'); the bf16-base decision exists to enable it",
286+
"kind": "follow_up",
287+
"claim_id": null
288+
},
289+
{
290+
"id": "v2",
291+
"text": "eval-suite CONTENTS (evals/tool-routing, formatting, contract-following datasets) \u2014 the 'lobes eval' verb lands now; the curated suites come later",
292+
"kind": "follow_up",
293+
"claim_id": null
294+
},
295+
{
296+
"id": "v3",
297+
"text": "the 4B's real architecture (dense vs hybrid/MoE, vision tower?) \u2014 sets the catalog shape field and whether moe_backend is needed (Sonnet agent verifying)",
298+
"kind": "unknown_nonblocking",
299+
"claim_id": null
300+
},
301+
{
302+
"id": "v4",
303+
"text": "community cosmicproc/Qwen3.5-4B-NVFP4 quality/throughput on the GB10 \u2014 untested; stays docs-only until load-tested",
304+
"kind": "unknown_nonblocking",
305+
"claim_id": null
306+
},
307+
{
308+
"id": "v5",
309+
"text": "Qwen3.5-4B ships a built-in MTP draft head (mtp_num_hidden_layers=1) \u2014 native speculative decoding for the minor with no separate draft checkpoint; deferred because the catalog test gates speculative_config on an 'MTP' id substring",
310+
"kind": "follow_up",
311+
"claim_id": null
312+
}
313+
]
314+
}

0 commit comments

Comments
 (0)