Skip to content

Commit e51ffc8

Browse files
OriNachumclaude
andauthored
feat: Gemma 4 12B multimodal duo — default main+multimodal serve (#69) (v0.32.1) (#72)
* spec: Gemma 4 multimodal duo for default serve (issue #69) (v0.32.1) /think frame for making `lobes serve` default to the Qwen3.6-27B-MTP + Gemma 4 12B NVFP4 duo. Gemma takes a new 'multimodal' role (vision + audio); tiers reframed to main/minor/multimodal; native MTP on by default with the DeepSeek DSpark draft as a disabled experiment; the Qwen3-14B is demoted to a legacy candidate. Five accepted plan risks parked for /spec-to-plan. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01K9gvEGUZaAUHey7mVHEHLR * plan: Gemma multimodal duo build plan (issue #69) /spec-to-plan forward leg: 9 tasks across 5 dependency waves covering all 26 spec targets, 6 accepted-risk objects. Pressure-ladder seam (main/minor/multimodal) resolved as task t6 (degrade to minor under pressure). Ready for /assign-to-workforce. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01K9gvEGUZaAUHey7mVHEHLR * t1: infer_parser rule for the Gemma 4 12B id Add a _RULES entry matching "gemma-4" / "gemma4" → "pythonic" so that `infer_parser("sakamakismile/gemma-4-12B-coder-fable5-composer2.5-MTP-NVFP4")` returns "pythonic". Gemma 4 (Google DeepMind) uses a Python-style function-call syntax; vLLM's Gemma 4 recipe prescribes --tool-call-parser pythonic. Markers are scoped to "gemma-4" / "gemma4" to avoid matching older Gemma 1/2/3 checkpoints whose tool-call story is less established. Unit test added to tests/test_parser.py asserting the exact model id. TODO(risk r2): confirm parser value against the served checkpoint during t7 live validation on the Spark. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01K9gvEGUZaAUHey7mVHEHLR * t2: Gemma multimodal catalog gear + main/minor/multimodal tiers + 14B demotion - Add SupportedModel for sakamakismile/gemma-4-12B-coder-fable5-composer2.5-MTP-NVFP4 (role_hint=multimodal, tool_parser=pythonic, modelopt_fp4, native MTP speculative_config, status=configured). TODO(risk r4): confirm the Gemma4 native-MTP method string in t7. TODO(risk): confirm native context (using 128K). - Demote nvidia/Qwen3-14B-NVFP4 from role_hint=middle to candidate (KEPT, not deleted). - Reframe TIER_ROLE to main/minor/multimodal with cheap/normal/hard back-compat aliases; normal+multimodal -> Gemma, main+hard -> primary, cheap+minor -> 4B. - Add docs/gemma-4-12b-nvfp4.md stub (full content in t9). - Extend tests/test_catalog.py with Gemma-gear, tier-resolution and 14B-demotion tests. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01K9gvEGUZaAUHey7mVHEHLR * test: reconcile test_catalog_tiers to main/minor/multimodal vocab (14B->candidate) Integration fix after t2 keystone: the 14B is now a legacy candidate and normal->multimodal. Catalog-domain, owned by no single task. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01K9gvEGUZaAUHey7mVHEHLR * t3: default-on vllm-multimodal (vision+audio+native-MTP), 14B legacy profile, DSpark off - New vllm-multimodal service (NO profiles: key, always starts with the fleet) serving sakamakismile/gemma-4-12B-coder-fable5-composer2.5-MTP-NVFP4 at util=0.12, 128K context, tool-call-parser=pythonic, --speculative-config gemma4_mtp, --trust-remote-code; NO --language-model-only so vision+audio stay active (the headline capability). - Gateway wired: MULTIMODAL_BASE_URL / MULTIMODAL_SERVED_NAME added to gateway environment; vllm-multimodal added to depends_on. - vllm-middle comment updated to "LEGACY CANDIDATE – no longer the normal tier". - env.example: MULTIMODAL_* section added; DSpark draft-model override documented as a commented, disabled-by-default experiment; GPU budget comment updated to the new default (primary 0.45 + multimodal 0.12 + embed 0.06 + rerank 0.06 = 0.69). - Three new template assertions in test_cli_fleet.py: multimodal is default-on (no profiles:), command lacks --language-model-only and has gemma4_mtp speculative-config, and vllm-middle is behind a profiles: key. TODOs left: TODO(risk r4) — confirm Gemma4 native-MTP method in t7; TODO(risk r5) — measure real multimodal util (vision+audio KV) in t7. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01K9gvEGUZaAUHey7mVHEHLR * t4: gateway routes main/minor/multimodal (+back-compat), middle->multimodal wiring - _config.py: replace _DEFAULT_MIDDLE/MIDDLE_BASE_URL/MIDDLE_SERVED_NAME with _DEFAULT_MULTIMODAL (pinned Gemma 4 12B id) / MULTIMODAL_BASE_URL / MULTIMODAL_SERVED_NAME; backend name changes from "middle" to "multimodal" (default url http://vllm-multimodal:8000). The 14B Qwen3 is now legacy — no tier resolves to it; address it explicitly by model id if needed. - _routing.py: fix tier_aliases() to handle the new mixed-vocabulary TIER_ROLE (primary: main/minor/multimodal + back-compat: cheap/normal/hard). The old algorithm relied on ascending-capability key order which the new TIER_ROLE breaks. New algorithm: sort unique roles by last occurrence in tier_role.values() (the back-compat aliases appear last in ascending order, anchoring the correct [minor, multimodal, primary] sequence), then apply upward fallback on unique roles only, and replicate results to all tier aliases. All six tier names (main/minor/multimodal + cheap/normal/hard) now resolve correctly with proper upward fallback when a gear is absent. - tests/test_gateway_tiers.py: full rewrite — 6 old "middle" tests renamed/updated to "multimodal"; 7 new tests added (main->primary, multimodal->Gemma backend, normal->multimodal back-compat, multimodal absent falls back to primary, primary vocab assertions in full-fleet tests, new tier_aliases helper test for minor- absent/multimodal-present escalation). 23 tests total, all passing. - tests/test_gateway_server.py, tests/test_tier_request.py: update _fleet_cfg / _full_fleet helpers to use MULTIMODAL_BASE_URL/MULTIMODAL_SERVED_NAME. - tests/test_status_pressure.py: fix pre-existing break from TIER_ROLE reframing (_MIDDLE_ID → _MULTIMODAL_ID for the normal tier; was failing before this PR). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01K9gvEGUZaAUHey7mVHEHLR * t5: lobes serve/init default to the main+multimodal duo; --single restores legacy * t6: pressure seam — degraded mode downgrades main/multimodal to minor (main/minor/multimodal vocab) Resolve the pressure-ladder seam under the #69 main/minor/multimodal vocabulary. multimodal is a different capability (vision+audio), not a cheaper rung below main, so "downgrade under pressure" cannot walk main->multimodal->minor. The only cheaper target is minor: under degraded pressure (swap>75% OR iowait>50%) both a main and a multimodal request collapse to minor (reason=pressure); below the degraded floor nothing is downgraded (the old linear intermediate band is gone). Back-compat cheap/normal/hard inputs still normalize to the new vocab. - _pressure_policy.decide: binary degraded model in new vocab; max_allowed_tier is minor when degraded else main; allowed_tier granted as requested when warm, minor when degraded. Adds normalize_tier(); keeps the degraded thresholds + env overrides and retains the no-hard/prefer-cheap thresholds as advisory (no longer cap the tier — no intermediate rung). Docstring documents the seam. - _tier_request: served_tier reported in main/minor/multimodal; override path normalizes; is_tier_alias recognizes all six tier keys (catalog.TIER_ROLE). - status.py: lobes status --pressure queries with requested_tier=main and reports tier/model in the new vocabulary. - Migrate pressure/tier-request tests (test_pressure_policy, test_tier_request, test_status_pressure, test_gateway_server) off the old cheap/normal/hard->cheap degraded behavior. Full suite green (774 passed). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01K9gvEGUZaAUHey7mVHEHLR * docs(gateway): update pressure docstring to main/minor/multimodal vocab (#69) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01K9gvEGUZaAUHey7mVHEHLR * colleague: Expand the stub file docs/gemma-4-12b-nvfp4.md into a COMPLET... Expand the stub file docs/gemma-4-12b-nvfp4.md into a COMPLETE per-model doc for the Gemma 4 12B 'multimodal' fleet gear (issue #69), closely matching the structure and tone of the sibling doc docs/qwen3-14b-nvfp4.md. Read these for accuracy BEFORE writing: lobes/catalog.py (the SupportedModel entry whose id is sakamakismile/gemma-4-12B-coder-fable5-composer2.5-MTP-NVFP4 — role_hint=multimodal, tool_parser=pythonic, quantization=modelopt_fp4, speculative_config gemma4_mtp, status=configured), lobes/templates/fleet/docker-compose.yml (the default-on vllm-multimodal service and its flags), and docs/qwen3-14b-nvfp4.md (template). The doc MUST cover: (1) what it is — Gemma 4 12B unified multimodal (text+IMAGE+AUDIO via Gemma4UnifiedForConditionalGeneration), native MTP draft head, NVFP4; (2) the pinned checkpoint id; (3) tier alias usage: model=multimodal (and model=normal back-compat) resolves here at the gateway, role_hint=multimodal; (4) serving in the fleet: it is DEFAULT-ON (not a profile), served WITHOUT --language-model-only so vision AND audio are active, --tool-call-parser=pythonic, native-MTP --speculative-config, --gpu-memory-utilization 0.12; (5) the DSpark experiment — DISABLED by default, an env override swaps in deepseek-ai/dspark_gemma4_12b_block7; (6) a GPU budget table for the default fleet: primary 0.45 + multimodal 0.12 + embed 0.06 + rerank 0.06 = 0.69 on the 128GB GB10; (7) accepted plan risks pending t7 live validation on the Spark: exact checkpoint pick, the gemma4_mtp method string, the 131072 native context, and the measured GPU util; (8) a Related docs section linking gateway-fleet.md, qwen3.6-27b-text-nvfp4-mtp.md, and qwen3-14b-nvfp4.md. Write ONLY docs/gemma-4-12b-nvfp4.md. Do not touch any other file. Implement the task above in this repository. Rules: - Make the SMALLEST change that correctly satisfies the task. - Follow the repository's existing patterns, style, and conventions — read the neighbouring files first so your change reads like the surrounding code. - Keep edits lint-clean: respect the project's maximum line length and end every text file with exactly one trailing newline. - You may read, create, modify files, and run commands as needed. - Don't widen the scope: do exactly what was asked, nothing more. When you are done, call finish with a short summary of exactly what you changed and why. * docs(gemma): fix DSpark toggle (speculative-config override, not model swap) + topology wording Verification correction on the colleague-authored doc: DSpark is a speculative-decoding draft head, enabled via MULTIMODAL_SPECULATIVE_CONFIG, not by replacing MULTIMODAL_MODEL. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01K9gvEGUZaAUHey7mVHEHLR * docs(gemma): markdownlint — avoid '+ AUDIO' wrapped-line list-style false positive * t8: smoke test — duo reachable (live-gated) + legacy profile selectable Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01K9gvEGUZaAUHey7mVHEHLR * t9b: update supporting docs (fleet topology, 14B demotion, pressure seam) to the Gemma duo Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01K9gvEGUZaAUHey7mVHEHLR * docs(CLAUDE): serve/init default to the duo (not single-model) since #69 * test(pressure): guard _TIER_ROLE mirror against catalog.TIER_ROLE drift Colleague full-branch review (#69) flagged the duplicated tier->role map in _pressure_policy.py as a silent-drift risk. Add a sync assertion. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01K9gvEGUZaAUHey7mVHEHLR * docs/catalog: record t7 finding — Gemma gemma4_unified unsupported on nv26.04/26.05 (track in #71) Live validation on the DGX Spark: the Gemma 4 12B multimodal gear does not load on any released NGC vLLM image (gemma4_unified arch absent from vLLM 0.19.0/0.21.0 + Transformers 4.57.6/5.6.0). Gear stays status=configured; custom-image unblock tracked in #71. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01K9gvEGUZaAUHey7mVHEHLR * eidetic: remember t7 finding — gemma4_unified unsupported on released vLLM images (#71) * chore(sonar): retire 3 Gemma-risk TODO markers → cite tracking issue #71 SonarCloud python:S1135 flagged three bare TODO(risk …) comments on PR #72, all marking deferred Gemma 4 12B live-validation work (native context, the native-MTP method string, the pythonic tool parser). That work is now blocked on a gemma4_unified-capable vLLM image and formally tracked in issue #71, so the comments are reworded from "TODO(risk …)" to "Risk … (pending #71)" — they cite the tracking issue instead of an untracked TODO. The premise of S1135 (untracked work rots) no longer holds, and the fix improves traceability. - lobes/catalog.py: 2 markers (native context default; native-MTP method) - lobes/runtime/_parser.py: 1 marker (pythonic parser confirmation) Comment-only; 158 catalog/parser/tier tests pass, black/isort/flake8 clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01K9gvEGUZaAUHey7mVHEHLR * fix(gateway): restore 14B middle backend + honor legacy-keyed alias overrides Two regressions surfaced by Qodo review on PR #72 (#69 Gemma duo): 1. Middle backend unreachable. The #69 14B demotion removed the `middle` optional-backend wiring from build_config(), but the fleet compose still ships the `vllm-middle` service (profile `middle`), still exposes MIDDLE_BASE_URL / MIDDLE_SERVED_NAME on the gateway, and still documents "set MIDDLE_BASE_URL … so it routes 14B requests." Result: enabling the profile silently fell back to the primary. Re-wire the optional `middle` generate backend — reachable by explicit served name, and (because "middle" is not a TIER_ROLE role) deliberately NOT a tier alias. 2. Legacy-keyed operator override ignored. resolve_tier_request() normalizes a legacy tier (hard→main) before the alias lookup, so a GATEWAY_ALIASES override keyed only by a legacy name (hard/cheap/normal) was bypassed on the pressure-aware path (the default path, since PressureCache is always wired). build_config() now mirrors a tier-keyed override onto its vocabulary synonyms via _expand_tier_alias_synonyms(); explicit keys still win and non-tier custom aliases pass through untouched. Tests: +4 middle-wiring tests + 3 override-expansion tests in test_gateway_tiers.py, +1 end-to-end legacy-override test in test_tier_request.py. Full suite 802 passed, 6 skipped; black/isort/flake8/bandit clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01K9gvEGUZaAUHey7mVHEHLR --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
1 parent 7dc1944 commit e51ffc8

40 files changed

Lines changed: 3028 additions & 780 deletions

.devague/current

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1 +1 @@
1-
lobes-adds-a-third-middle-model-tier-and-pressure
1+
lobes-now-defaults-to-a-spark-duo-the-qwen3-6-27b

.devague/current_plan

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1 +1 @@
1-
lobes-adds-a-third-middle-model-tier-and-pressure
1+
lobes-now-defaults-to-a-spark-duo-the-qwen3-6-27b
Lines changed: 314 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,314 @@
1+
{
2+
"slug": "lobes-now-defaults-to-a-spark-duo-the-qwen3-6-27b",
3+
"title": "lobes now defaults to a Spark duo: the Qwen3.6-27B-MTP primary paired with a Gemma 4 12B NVFP4 multimodal worker that takes over the normal/middle tier, replacing the text-only Qwen3-14B (demoted to a legacy candidate profile); the duo gives Spark multimodal senses, a diverse non-Qwen mind, and a DSpark speculative-decoding test target, with old models kept behind explicit legacy profiles.",
4+
"schema_version": 1,
5+
"status": "exported",
6+
"created": "2026-06-30T12:07:56Z",
7+
"updated": "2026-06-30T12:35:33Z",
8+
"claims": [
9+
{
10+
"id": "c1",
11+
"kind": "announcement",
12+
"text": "lobes now defaults to a Spark duo: the Qwen3.6-27B-MTP primary paired with a Gemma 4 12B NVFP4 multimodal worker that takes over the normal/middle tier, replacing the text-only Qwen3-14B (demoted to a legacy candidate profile); the duo gives Spark multimodal senses, a diverse non-Qwen mind, and a DSpark speculative-decoding test target, with old models kept behind explicit legacy profiles.",
13+
"origin": "user",
14+
"status": "confirmed",
15+
"honesty_conditions": [
16+
{
17+
"id": "h9",
18+
"text": "a fresh lobes install with no overrides serves exactly the Qwen 'main' + Gemma 'multimodal' duo, and every legacy model (14B, minor, etc.) is reachable only behind an explicit profile/override",
19+
"status": "confirmed"
20+
}
21+
],
22+
"hard_questions": [],
23+
"links": []
24+
},
25+
{
26+
"id": "c2",
27+
"kind": "audience",
28+
"text": "Spark's agent stack and the Culture mesh peers that consume the lobes OpenAI endpoint \u2014 they gain a multimodal worker and a diverse second mind alongside the Qwen primary",
29+
"origin": "llm",
30+
"status": "confirmed",
31+
"honesty_conditions": [
32+
{
33+
"id": "h10",
34+
"text": "a Spark agent / Culture mesh peer reaches the multimodal gear through the existing lobes OpenAI endpoint with no client change beyond sending model=multimodal",
35+
"status": "confirmed"
36+
}
37+
],
38+
"hard_questions": [],
39+
"links": []
40+
},
41+
{
42+
"id": "c3",
43+
"kind": "after_state",
44+
"text": "lobes serve / fleet up brings up the Qwen3.6-27B-MTP primary + a Gemma 4 12B NVFP4 multimodal worker by default; callers send model=normal and reach Gemma (vision-capable), model=hard reaches the Qwen primary, with the 14B kept only as an opt-in legacy candidate",
45+
"origin": "llm",
46+
"status": "rejected",
47+
"honesty_conditions": [],
48+
"hard_questions": [],
49+
"links": []
50+
},
51+
{
52+
"id": "c4",
53+
"kind": "before_state",
54+
"text": "today the normal/middle tier is a text-only nvidia/Qwen3-14B-NVFP4 that is opt-in (COMPOSE_PROFILES=middle), never load-tested (status=configured), and same-family as the primary \u2014 so the fleet has no multimodal generate gear and no non-Qwen mind",
55+
"origin": "llm",
56+
"status": "confirmed",
57+
"honesty_conditions": [
58+
{
59+
"id": "h11",
60+
"text": "this matches today's shipped state, verifiable in the repo: the 14B middle is opt-in (COMPOSE_PROFILES=middle), status=configured in catalog.py, hermes parser, text-only \u2014 and there is no multimodal or non-Qwen generate gear in the fleet",
61+
"status": "confirmed"
62+
}
63+
],
64+
"hard_questions": [],
65+
"links": []
66+
},
67+
{
68+
"id": "c5",
69+
"kind": "why_it_matters",
70+
"text": "a multimodal worker lets Spark see screenshots/UI and images (the MTP primary is text-only); a non-Qwen family gives genuinely diverse reasoning for review/triage; and Gemma 4 gives a concrete DSpark/MTP speculative-decoding acceleration target on Blackwell",
71+
"origin": "llm",
72+
"status": "confirmed",
73+
"honesty_conditions": [
74+
{
75+
"id": "h12",
76+
"text": "an image-bearing request the text-only primary cannot serve is answered by the multimodal gear, and the served family is observably Gemma (a different lineage from the Qwen primary/minor)",
77+
"status": "confirmed"
78+
}
79+
],
80+
"hard_questions": [],
81+
"links": []
82+
},
83+
{
84+
"id": "c6",
85+
"kind": "boundary",
86+
"text": "not removing the Qwen3-14B or any catalog gear (kept as legacy candidates); not changing the embed/rerank pooling gears, the audio overlay, or the minor 4B LoRA base; not adding a lobes train verb; DSpark stays a disabled-by-default experiment",
87+
"origin": "llm",
88+
"status": "confirmed",
89+
"honesty_conditions": [
90+
{
91+
"id": "h13",
92+
"text": "after the change the embed/rerank/audio/minor gears and all other catalog entries are unchanged except the 14B role demotion and the new Gemma entry; no lobes train verb is added; DSpark ships off",
93+
"status": "confirmed"
94+
}
95+
],
96+
"hard_questions": [],
97+
"links": []
98+
},
99+
{
100+
"id": "c7",
101+
"kind": "success_signal",
102+
"text": "a smoke test confirms both default models are reachable (model=hard text + model=normal multimodal image round-trip) and that a legacy profile (e.g. the 14B) can still be selected explicitly",
103+
"origin": "llm",
104+
"status": "confirmed",
105+
"honesty_conditions": [
106+
{
107+
"id": "h14",
108+
"text": "a committed smoke test asserts model=main (text) and model=multimodal (image round-trip) both return valid output, and that an explicit legacy profile (e.g. the 14B) still boots and serves",
109+
"status": "confirmed"
110+
}
111+
],
112+
"hard_questions": [],
113+
"links": []
114+
},
115+
{
116+
"id": "c8",
117+
"kind": "requirement",
118+
"text": "Gemma serves WITH its vision tower enabled (NOT --language-model-only) so model=normal accepts image inputs \u2014 multimodal coverage is the headline capability",
119+
"origin": "llm",
120+
"status": "confirmed",
121+
"honesty_conditions": [
122+
{
123+
"id": "h1",
124+
"text": "vLLM on the production image serves Gemma 4 12B with the vision tower and returns a correct answer to an image+text request via the gateway model=normal route",
125+
"status": "confirmed"
126+
}
127+
],
128+
"hard_questions": [],
129+
"links": []
130+
},
131+
{
132+
"id": "c9",
133+
"kind": "requirement",
134+
"text": "Gemma takes role_hint=middle and the normal tier alias in catalog.py + TIER_ROLE, replacing nvidia/Qwen3-14B-NVFP4 as what model=normal resolves to",
135+
"origin": "llm",
136+
"status": "rejected",
137+
"honesty_conditions": [
138+
{
139+
"id": "h2",
140+
"text": "catalog test (tool_parser==infer_parser, doc exists) stays green after Gemma replaces the 14B as middle, and model=normal resolves to Gemma at the gateway",
141+
"status": "proposed"
142+
}
143+
],
144+
"hard_questions": [],
145+
"links": []
146+
},
147+
{
148+
"id": "c10",
149+
"kind": "requirement",
150+
"text": "the Qwen-primary + Gemma duo is the DEFAULT served fleet (Gemma moves from opt-in COMPOSE_PROFILES=middle to default-on); old gears require an explicit legacy profile/override to serve",
151+
"origin": "llm",
152+
"status": "rejected",
153+
"honesty_conditions": [
154+
{
155+
"id": "h3",
156+
"text": "after the change, a fresh lobes serve/fleet up with no extra flags brings up BOTH the primary and Gemma, and brings up NEITHER the 14B nor any other legacy gear",
157+
"status": "proposed"
158+
}
159+
],
160+
"hard_questions": [],
161+
"links": []
162+
},
163+
{
164+
"id": "c11",
165+
"kind": "requirement",
166+
"text": "a disabled-by-default experiment wires the DeepSeek DSpark draft (deepseek-ai/dspark_gemma4_12b_block7) as a Gemma4 speculative-decoding target, off unless explicitly enabled",
167+
"origin": "llm",
168+
"status": "rejected",
169+
"honesty_conditions": [
170+
{
171+
"id": "h4",
172+
"text": "the DSpark experiment is provably off by default (no speculative-config on the default Gemma gear) and can be turned on by one documented toggle",
173+
"status": "proposed"
174+
}
175+
],
176+
"hard_questions": [],
177+
"links": []
178+
},
179+
{
180+
"id": "c12",
181+
"kind": "requirement",
182+
"text": "the duo + pooling gears fit the 128GB GB10 unified memory budget with measured gpu-memory-utilization values (Gemma multimodal weights + ViT + image KV vs the 14B's 0.12)",
183+
"origin": "llm",
184+
"status": "confirmed",
185+
"honesty_conditions": [
186+
{
187+
"id": "h5",
188+
"text": "a measured run shows primary + Gemma + embed + rerank (+ minor if kept) all healthy and co-resident under 1.0 total util on the 128GB GB10",
189+
"status": "confirmed"
190+
}
191+
],
192+
"hard_questions": [],
193+
"links": []
194+
},
195+
{
196+
"id": "c13",
197+
"kind": "after_state",
198+
"text": "lobes serve (and fleet up) bring up the Qwen3.6-27B-MTP 'main' gear + the Gemma 4 12B NVFP4 'multimodal' gear by default; the three live generate gears are addressed as main / minor / multimodal, and model=multimodal reaches Gemma for image+text",
199+
"origin": "user",
200+
"status": "confirmed",
201+
"honesty_conditions": [
202+
{
203+
"id": "h6",
204+
"text": "model=multimodal routes to Gemma at the gateway, model=main routes to the Qwen primary, and the catalog test stays green with role_hint='multimodal' present and the 14B as a candidate",
205+
"status": "confirmed"
206+
}
207+
],
208+
"hard_questions": [],
209+
"links": []
210+
},
211+
{
212+
"id": "c14",
213+
"kind": "requirement",
214+
"text": "Gemma gets a new role_hint='multimodal' in catalog.py; the old 'middle' role and 'normal' alias are deprecated; the normal tier slot resolves to Gemma for back-compat, and nvidia/Qwen3-14B-NVFP4 is demoted to role_hint=candidate (legacy, kept)",
215+
"origin": "user",
216+
"status": "confirmed",
217+
"honesty_conditions": [
218+
{
219+
"id": "h7",
220+
"text": "a clean lobes serve with no flags yields exactly two generate gears up (main + multimodal) and zero legacy gears, verifiable via /v1/models and the pressure/status output",
221+
"status": "confirmed"
222+
}
223+
],
224+
"hard_questions": [],
225+
"links": []
226+
},
227+
{
228+
"id": "c15",
229+
"kind": "requirement",
230+
"text": "lobes serve no longer means single-model: with no extra flags it brings up BOTH main (Qwen primary) and multimodal (Gemma); minor, the 14B, and every other legacy gear require an explicit profile/override to serve",
231+
"origin": "user",
232+
"status": "confirmed",
233+
"honesty_conditions": [
234+
{
235+
"id": "h8",
236+
"text": "the default Gemma gear boots with native-MTP speculative-config active (measurable draft acceptance > 0), and flipping the DSpark toggle off is the default state with no DSpark weights loaded",
237+
"status": "confirmed"
238+
}
239+
],
240+
"hard_questions": [],
241+
"links": []
242+
},
243+
{
244+
"id": "c16",
245+
"kind": "requirement",
246+
"text": "the default Gemma 'multimodal' gear serves NVFP4 + its native MTP assistant (speculative decoding ON); the DeepSeek DSpark draft (dspark_gemma4_12b_block7) is a separate, disabled-by-default experiment toggle",
247+
"origin": "user",
248+
"status": "confirmed",
249+
"honesty_conditions": [
250+
{
251+
"id": "h15",
252+
"text": "the default Gemma gear's compose command carries a native-MTP --speculative-config and NO DSpark draft, and a single documented env toggle (off by default) swaps in deepseek-ai/dspark_gemma4_12b_block7",
253+
"status": "confirmed"
254+
}
255+
],
256+
"hard_questions": [],
257+
"links": []
258+
},
259+
{
260+
"id": "c17",
261+
"kind": "requirement",
262+
"text": "the default 'multimodal' gear ALSO serves Gemma 4's native AUDIO modality (audio-in understanding / ASR via the Gemma4Unified multimodal embedder, which vLLM supports), so model=multimodal accepts audio content in a chat request alongside text+image \u2014 distinct from and NOT replacing the existing /v1/audio/* overlay (Parakeet STT / Chatterbox TTS) per c6",
263+
"origin": "user",
264+
"status": "confirmed",
265+
"honesty_conditions": [
266+
{
267+
"id": "h16",
268+
"text": "vLLM on our image serves the chosen Gemma 4 12B build with the audio modality active, and model=multimodal returns correct text for an audio-bearing chat request (e.g. a short transcription) without disabling vision or the existing /v1/audio/* overlay",
269+
"status": "confirmed"
270+
}
271+
],
272+
"hard_questions": [],
273+
"links": []
274+
}
275+
],
276+
"open_vagueness": [
277+
{
278+
"id": "v1",
279+
"text": "exact HF checkpoint id for the default Gemma 4 12B NVFP4 gear (accepted plan risk \u2014 verify+pick during implementation, like the 14B t1). NVFP4 12B builds PROVABLY EXIST: sakamakismile/gemma-4-12B-coder-fable5-composer2.5-MTP-NVFP4 (SAME publisher as the lobes primary, NVFP4+native-MTP \u2014 leading candidate, matches the MTP-on-by-default decision), AxionML/Gemma-4-12B-NVFP4, coolthor/gemma-4-12B-it-NVFP4A16. NVIDIA ships only 31B + 26B-A4B NVFP4, so the 12B is community.",
280+
"kind": "unknown_nonblocking",
281+
"claim_id": "c16"
282+
},
283+
{
284+
"id": "v2",
285+
"text": "vLLM --tool-call-parser to use for Gemma 4 (Gemma has no strong native tool format; must add a runtime/_parser.py infer_parser rule so the catalog test passes)",
286+
"kind": "unknown_nonblocking",
287+
"claim_id": null
288+
},
289+
{
290+
"id": "v3",
291+
"text": "whether the chosen Gemma 4 12B NVFP4 build loads non-gibberish on the current Blackwell vLLM image (nvcr.io/nvidia/vllm:26.04) or needs a newer engine \u2014 a load-validation task (like the 14B t9), now LOWER RISK: vLLM registers Gemma4UnifiedForConditionalGeneration and auto-detects NVFP4 (may not even need --quantization), and Gemma4 is a mainstream multimodal arch, NOT the hybrid Gated-DeltaNet/FLA arch that hit the Qwen3.5 sm_120/121 gibberish bug. Verify the image engine version is new enough before promoting status to load-tested.",
292+
"kind": "unknown_nonblocking",
293+
"claim_id": "c12"
294+
},
295+
{
296+
"id": "v4",
297+
"text": "the exact vLLM --speculative-config method+JSON for (a) Gemma4 NATIVE MTP (the default gear's spec-decode, analogous to the primary's {method: qwen3_5_mtp}) and (b) the DSpark experiment draft. DSpark shipped 2026-06-27 via DeepSeek's DeepSpec codebase (deepseek-ai/dspark_gemma4_12b_block7, validated on Gemma4-12B, vLLM-servable). Gemma4 assistant checkpoints use the Gemma4 MTP path, not a generic draft \u2014 confirm the method string against the served checkpoint's config.",
298+
"kind": "unknown_nonblocking",
299+
"claim_id": "c16"
300+
},
301+
{
302+
"id": "v5",
303+
"text": "measured GPU util for multimodal Gemma (vision tower + image KV) \u2014 does 0.12 hold or does the duo need primary/util retuning to fit 128GB",
304+
"kind": "unknown_nonblocking",
305+
"claim_id": null
306+
},
307+
{
308+
"id": "v6",
309+
"text": "Gemma 4's native ASR (model=multimodal audio-in) OVERLAPS the Parakeet STT sidecar behind /v1/audio/transcriptions \u2014 decide in planning whether Gemma subsumes, augments, or co-exists with Parakeet (the OpenAI /v1/audio/* endpoint shape still needs Parakeet/Chatterbox; Gemma audio-in is chat-only). Bonus: Gemma 4 also ingests VIDEO \u2014 out of scope for v1 but available.",
310+
"kind": "follow_up",
311+
"claim_id": null
312+
}
313+
]
314+
}

0 commit comments

Comments
 (0)