Commit e51ffc8
* spec: Gemma 4 multimodal duo for default serve (issue #69) (v0.32.1)
/think frame for making `lobes serve` default to the Qwen3.6-27B-MTP + Gemma 4 12B NVFP4 duo. Gemma takes a new 'multimodal' role (vision + audio); tiers reframed to main/minor/multimodal; native MTP on by default with the DeepSeek DSpark draft as a disabled experiment; the Qwen3-14B is demoted to a legacy candidate. Five accepted plan risks parked for /spec-to-plan.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K9gvEGUZaAUHey7mVHEHLR
* plan: Gemma multimodal duo build plan (issue #69)
/spec-to-plan forward leg: 9 tasks across 5 dependency waves covering all 26 spec targets, 6 accepted-risk objects. Pressure-ladder seam (main/minor/multimodal) resolved as task t6 (degrade to minor under pressure). Ready for /assign-to-workforce.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K9gvEGUZaAUHey7mVHEHLR
* t1: infer_parser rule for the Gemma 4 12B id
Add a _RULES entry matching "gemma-4" / "gemma4" → "pythonic" so that
`infer_parser("sakamakismile/gemma-4-12B-coder-fable5-composer2.5-MTP-NVFP4")`
returns "pythonic". Gemma 4 (Google DeepMind) uses a Python-style
function-call syntax; vLLM's Gemma 4 recipe prescribes
--tool-call-parser pythonic.
Markers are scoped to "gemma-4" / "gemma4" to avoid matching older
Gemma 1/2/3 checkpoints whose tool-call story is less established.
Unit test added to tests/test_parser.py asserting the exact model id.
TODO(risk r2): confirm parser value against the served checkpoint
during t7 live validation on the Spark.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K9gvEGUZaAUHey7mVHEHLR
* t2: Gemma multimodal catalog gear + main/minor/multimodal tiers + 14B demotion
- Add SupportedModel for sakamakismile/gemma-4-12B-coder-fable5-composer2.5-MTP-NVFP4
(role_hint=multimodal, tool_parser=pythonic, modelopt_fp4, native MTP
speculative_config, status=configured). TODO(risk r4): confirm the Gemma4
native-MTP method string in t7. TODO(risk): confirm native context (using 128K).
- Demote nvidia/Qwen3-14B-NVFP4 from role_hint=middle to candidate (KEPT, not deleted).
- Reframe TIER_ROLE to main/minor/multimodal with cheap/normal/hard back-compat
aliases; normal+multimodal -> Gemma, main+hard -> primary, cheap+minor -> 4B.
- Add docs/gemma-4-12b-nvfp4.md stub (full content in t9).
- Extend tests/test_catalog.py with Gemma-gear, tier-resolution and 14B-demotion tests.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K9gvEGUZaAUHey7mVHEHLR
* test: reconcile test_catalog_tiers to main/minor/multimodal vocab (14B->candidate)
Integration fix after t2 keystone: the 14B is now a legacy candidate and normal->multimodal. Catalog-domain, owned by no single task.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K9gvEGUZaAUHey7mVHEHLR
* t3: default-on vllm-multimodal (vision+audio+native-MTP), 14B legacy profile, DSpark off
- New vllm-multimodal service (NO profiles: key, always starts with the fleet)
serving sakamakismile/gemma-4-12B-coder-fable5-composer2.5-MTP-NVFP4 at
util=0.12, 128K context, tool-call-parser=pythonic, --speculative-config
gemma4_mtp, --trust-remote-code; NO --language-model-only so vision+audio stay
active (the headline capability).
- Gateway wired: MULTIMODAL_BASE_URL / MULTIMODAL_SERVED_NAME added to gateway
environment; vllm-multimodal added to depends_on.
- vllm-middle comment updated to "LEGACY CANDIDATE – no longer the normal tier".
- env.example: MULTIMODAL_* section added; DSpark draft-model override documented
as a commented, disabled-by-default experiment; GPU budget comment updated to
the new default (primary 0.45 + multimodal 0.12 + embed 0.06 + rerank 0.06 = 0.69).
- Three new template assertions in test_cli_fleet.py: multimodal is default-on
(no profiles:), command lacks --language-model-only and has gemma4_mtp
speculative-config, and vllm-middle is behind a profiles: key.
TODOs left: TODO(risk r4) — confirm Gemma4 native-MTP method in t7;
TODO(risk r5) — measure real multimodal util (vision+audio KV) in t7.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K9gvEGUZaAUHey7mVHEHLR
* t4: gateway routes main/minor/multimodal (+back-compat), middle->multimodal wiring
- _config.py: replace _DEFAULT_MIDDLE/MIDDLE_BASE_URL/MIDDLE_SERVED_NAME with
_DEFAULT_MULTIMODAL (pinned Gemma 4 12B id) / MULTIMODAL_BASE_URL /
MULTIMODAL_SERVED_NAME; backend name changes from "middle" to "multimodal"
(default url http://vllm-multimodal:8000). The 14B Qwen3 is now legacy —
no tier resolves to it; address it explicitly by model id if needed.
- _routing.py: fix tier_aliases() to handle the new mixed-vocabulary TIER_ROLE
(primary: main/minor/multimodal + back-compat: cheap/normal/hard). The old
algorithm relied on ascending-capability key order which the new TIER_ROLE
breaks. New algorithm: sort unique roles by last occurrence in tier_role.values()
(the back-compat aliases appear last in ascending order, anchoring the correct
[minor, multimodal, primary] sequence), then apply upward fallback on unique
roles only, and replicate results to all tier aliases. All six tier names
(main/minor/multimodal + cheap/normal/hard) now resolve correctly with proper
upward fallback when a gear is absent.
- tests/test_gateway_tiers.py: full rewrite — 6 old "middle" tests renamed/updated
to "multimodal"; 7 new tests added (main->primary, multimodal->Gemma backend,
normal->multimodal back-compat, multimodal absent falls back to primary, primary
vocab assertions in full-fleet tests, new tier_aliases helper test for minor-
absent/multimodal-present escalation). 23 tests total, all passing.
- tests/test_gateway_server.py, tests/test_tier_request.py: update _fleet_cfg /
_full_fleet helpers to use MULTIMODAL_BASE_URL/MULTIMODAL_SERVED_NAME.
- tests/test_status_pressure.py: fix pre-existing break from TIER_ROLE reframing
(_MIDDLE_ID → _MULTIMODAL_ID for the normal tier; was failing before this PR).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K9gvEGUZaAUHey7mVHEHLR
* t5: lobes serve/init default to the main+multimodal duo; --single restores legacy
* t6: pressure seam — degraded mode downgrades main/multimodal to minor (main/minor/multimodal vocab)
Resolve the pressure-ladder seam under the #69 main/minor/multimodal vocabulary.
multimodal is a different capability (vision+audio), not a cheaper rung below
main, so "downgrade under pressure" cannot walk main->multimodal->minor. The
only cheaper target is minor: under degraded pressure (swap>75% OR iowait>50%)
both a main and a multimodal request collapse to minor (reason=pressure); below
the degraded floor nothing is downgraded (the old linear intermediate band is
gone). Back-compat cheap/normal/hard inputs still normalize to the new vocab.
- _pressure_policy.decide: binary degraded model in new vocab; max_allowed_tier
is minor when degraded else main; allowed_tier granted as requested when warm,
minor when degraded. Adds normalize_tier(); keeps the degraded thresholds +
env overrides and retains the no-hard/prefer-cheap thresholds as advisory
(no longer cap the tier — no intermediate rung). Docstring documents the seam.
- _tier_request: served_tier reported in main/minor/multimodal; override path
normalizes; is_tier_alias recognizes all six tier keys (catalog.TIER_ROLE).
- status.py: lobes status --pressure queries with requested_tier=main and
reports tier/model in the new vocabulary.
- Migrate pressure/tier-request tests (test_pressure_policy, test_tier_request,
test_status_pressure, test_gateway_server) off the old cheap/normal/hard->cheap
degraded behavior. Full suite green (774 passed).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K9gvEGUZaAUHey7mVHEHLR
* docs(gateway): update pressure docstring to main/minor/multimodal vocab (#69)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K9gvEGUZaAUHey7mVHEHLR
* colleague: Expand the stub file docs/gemma-4-12b-nvfp4.md into a COMPLET...
Expand the stub file docs/gemma-4-12b-nvfp4.md into a COMPLETE per-model doc for the Gemma 4 12B 'multimodal' fleet gear (issue #69), closely matching the structure and tone of the sibling doc docs/qwen3-14b-nvfp4.md. Read these for accuracy BEFORE writing: lobes/catalog.py (the SupportedModel entry whose id is sakamakismile/gemma-4-12B-coder-fable5-composer2.5-MTP-NVFP4 — role_hint=multimodal, tool_parser=pythonic, quantization=modelopt_fp4, speculative_config gemma4_mtp, status=configured), lobes/templates/fleet/docker-compose.yml (the default-on vllm-multimodal service and its flags), and docs/qwen3-14b-nvfp4.md (template). The doc MUST cover: (1) what it is — Gemma 4 12B unified multimodal (text+IMAGE+AUDIO via Gemma4UnifiedForConditionalGeneration), native MTP draft head, NVFP4; (2) the pinned checkpoint id; (3) tier alias usage: model=multimodal (and model=normal back-compat) resolves here at the gateway, role_hint=multimodal; (4) serving in the fleet: it is DEFAULT-ON (not a profile), served WITHOUT --language-model-only so vision AND audio are active, --tool-call-parser=pythonic, native-MTP --speculative-config, --gpu-memory-utilization 0.12; (5) the DSpark experiment — DISABLED by default, an env override swaps in deepseek-ai/dspark_gemma4_12b_block7; (6) a GPU budget table for the default fleet: primary 0.45 + multimodal 0.12 + embed 0.06 + rerank 0.06 = 0.69 on the 128GB GB10; (7) accepted plan risks pending t7 live validation on the Spark: exact checkpoint pick, the gemma4_mtp method string, the 131072 native context, and the measured GPU util; (8) a Related docs section linking gateway-fleet.md, qwen3.6-27b-text-nvfp4-mtp.md, and qwen3-14b-nvfp4.md. Write ONLY docs/gemma-4-12b-nvfp4.md. Do not touch any other file.
Implement the task above in this repository.
Rules:
- Make the SMALLEST change that correctly satisfies the task.
- Follow the repository's existing patterns, style, and conventions — read the
neighbouring files first so your change reads like the surrounding code.
- Keep edits lint-clean: respect the project's maximum line length and end every
text file with exactly one trailing newline.
- You may read, create, modify files, and run commands as needed.
- Don't widen the scope: do exactly what was asked, nothing more.
When you are done, call finish with a short summary of exactly what you changed
and why.
* docs(gemma): fix DSpark toggle (speculative-config override, not model swap) + topology wording
Verification correction on the colleague-authored doc: DSpark is a speculative-decoding draft head, enabled via MULTIMODAL_SPECULATIVE_CONFIG, not by replacing MULTIMODAL_MODEL.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K9gvEGUZaAUHey7mVHEHLR
* docs(gemma): markdownlint — avoid '+ AUDIO' wrapped-line list-style false positive
* t8: smoke test — duo reachable (live-gated) + legacy profile selectable
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K9gvEGUZaAUHey7mVHEHLR
* t9b: update supporting docs (fleet topology, 14B demotion, pressure seam) to the Gemma duo
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K9gvEGUZaAUHey7mVHEHLR
* docs(CLAUDE): serve/init default to the duo (not single-model) since #69
* test(pressure): guard _TIER_ROLE mirror against catalog.TIER_ROLE drift
Colleague full-branch review (#69) flagged the duplicated tier->role map in _pressure_policy.py as a silent-drift risk. Add a sync assertion.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K9gvEGUZaAUHey7mVHEHLR
* docs/catalog: record t7 finding — Gemma gemma4_unified unsupported on nv26.04/26.05 (track in #71)
Live validation on the DGX Spark: the Gemma 4 12B multimodal gear does not load on any released NGC vLLM image (gemma4_unified arch absent from vLLM 0.19.0/0.21.0 + Transformers 4.57.6/5.6.0). Gear stays status=configured; custom-image unblock tracked in #71.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K9gvEGUZaAUHey7mVHEHLR
* eidetic: remember t7 finding — gemma4_unified unsupported on released vLLM images (#71)
* chore(sonar): retire 3 Gemma-risk TODO markers → cite tracking issue #71
SonarCloud python:S1135 flagged three bare TODO(risk …) comments on PR #72,
all marking deferred Gemma 4 12B live-validation work (native context, the
native-MTP method string, the pythonic tool parser). That work is now blocked
on a gemma4_unified-capable vLLM image and formally tracked in issue #71, so
the comments are reworded from "TODO(risk …)" to "Risk … (pending #71)" — they
cite the tracking issue instead of an untracked TODO. The premise of S1135
(untracked work rots) no longer holds, and the fix improves traceability.
- lobes/catalog.py: 2 markers (native context default; native-MTP method)
- lobes/runtime/_parser.py: 1 marker (pythonic parser confirmation)
Comment-only; 158 catalog/parser/tier tests pass, black/isort/flake8 clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K9gvEGUZaAUHey7mVHEHLR
* fix(gateway): restore 14B middle backend + honor legacy-keyed alias overrides
Two regressions surfaced by Qodo review on PR #72 (#69 Gemma duo):
1. Middle backend unreachable. The #69 14B demotion removed the `middle`
optional-backend wiring from build_config(), but the fleet compose still
ships the `vllm-middle` service (profile `middle`), still exposes
MIDDLE_BASE_URL / MIDDLE_SERVED_NAME on the gateway, and still documents
"set MIDDLE_BASE_URL … so it routes 14B requests." Result: enabling the
profile silently fell back to the primary. Re-wire the optional `middle`
generate backend — reachable by explicit served name, and (because "middle"
is not a TIER_ROLE role) deliberately NOT a tier alias.
2. Legacy-keyed operator override ignored. resolve_tier_request() normalizes a
legacy tier (hard→main) before the alias lookup, so a GATEWAY_ALIASES
override keyed only by a legacy name (hard/cheap/normal) was bypassed on the
pressure-aware path (the default path, since PressureCache is always wired).
build_config() now mirrors a tier-keyed override onto its vocabulary
synonyms via _expand_tier_alias_synonyms(); explicit keys still win and
non-tier custom aliases pass through untouched.
Tests: +4 middle-wiring tests + 3 override-expansion tests in
test_gateway_tiers.py, +1 end-to-end legacy-override test in test_tier_request.py.
Full suite 802 passed, 6 skipped; black/isort/flake8/bandit clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K9gvEGUZaAUHey7mVHEHLR
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
1 parent 7dc1944 commit e51ffc8
40 files changed
Lines changed: 3028 additions & 780 deletions
File tree
- .devague
- frames
- plans
- .eidetic/memory
- docs
- plans
- specs
- lobes
- cli/_commands
- gateway
- runtime
- templates/fleet
- tests
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1 | | - | |
| 1 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1 | | - | |
| 1 | + | |
Lines changed: 314 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
| 99 | + | |
| 100 | + | |
| 101 | + | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
| 107 | + | |
| 108 | + | |
| 109 | + | |
| 110 | + | |
| 111 | + | |
| 112 | + | |
| 113 | + | |
| 114 | + | |
| 115 | + | |
| 116 | + | |
| 117 | + | |
| 118 | + | |
| 119 | + | |
| 120 | + | |
| 121 | + | |
| 122 | + | |
| 123 | + | |
| 124 | + | |
| 125 | + | |
| 126 | + | |
| 127 | + | |
| 128 | + | |
| 129 | + | |
| 130 | + | |
| 131 | + | |
| 132 | + | |
| 133 | + | |
| 134 | + | |
| 135 | + | |
| 136 | + | |
| 137 | + | |
| 138 | + | |
| 139 | + | |
| 140 | + | |
| 141 | + | |
| 142 | + | |
| 143 | + | |
| 144 | + | |
| 145 | + | |
| 146 | + | |
| 147 | + | |
| 148 | + | |
| 149 | + | |
| 150 | + | |
| 151 | + | |
| 152 | + | |
| 153 | + | |
| 154 | + | |
| 155 | + | |
| 156 | + | |
| 157 | + | |
| 158 | + | |
| 159 | + | |
| 160 | + | |
| 161 | + | |
| 162 | + | |
| 163 | + | |
| 164 | + | |
| 165 | + | |
| 166 | + | |
| 167 | + | |
| 168 | + | |
| 169 | + | |
| 170 | + | |
| 171 | + | |
| 172 | + | |
| 173 | + | |
| 174 | + | |
| 175 | + | |
| 176 | + | |
| 177 | + | |
| 178 | + | |
| 179 | + | |
| 180 | + | |
| 181 | + | |
| 182 | + | |
| 183 | + | |
| 184 | + | |
| 185 | + | |
| 186 | + | |
| 187 | + | |
| 188 | + | |
| 189 | + | |
| 190 | + | |
| 191 | + | |
| 192 | + | |
| 193 | + | |
| 194 | + | |
| 195 | + | |
| 196 | + | |
| 197 | + | |
| 198 | + | |
| 199 | + | |
| 200 | + | |
| 201 | + | |
| 202 | + | |
| 203 | + | |
| 204 | + | |
| 205 | + | |
| 206 | + | |
| 207 | + | |
| 208 | + | |
| 209 | + | |
| 210 | + | |
| 211 | + | |
| 212 | + | |
| 213 | + | |
| 214 | + | |
| 215 | + | |
| 216 | + | |
| 217 | + | |
| 218 | + | |
| 219 | + | |
| 220 | + | |
| 221 | + | |
| 222 | + | |
| 223 | + | |
| 224 | + | |
| 225 | + | |
| 226 | + | |
| 227 | + | |
| 228 | + | |
| 229 | + | |
| 230 | + | |
| 231 | + | |
| 232 | + | |
| 233 | + | |
| 234 | + | |
| 235 | + | |
| 236 | + | |
| 237 | + | |
| 238 | + | |
| 239 | + | |
| 240 | + | |
| 241 | + | |
| 242 | + | |
| 243 | + | |
| 244 | + | |
| 245 | + | |
| 246 | + | |
| 247 | + | |
| 248 | + | |
| 249 | + | |
| 250 | + | |
| 251 | + | |
| 252 | + | |
| 253 | + | |
| 254 | + | |
| 255 | + | |
| 256 | + | |
| 257 | + | |
| 258 | + | |
| 259 | + | |
| 260 | + | |
| 261 | + | |
| 262 | + | |
| 263 | + | |
| 264 | + | |
| 265 | + | |
| 266 | + | |
| 267 | + | |
| 268 | + | |
| 269 | + | |
| 270 | + | |
| 271 | + | |
| 272 | + | |
| 273 | + | |
| 274 | + | |
| 275 | + | |
| 276 | + | |
| 277 | + | |
| 278 | + | |
| 279 | + | |
| 280 | + | |
| 281 | + | |
| 282 | + | |
| 283 | + | |
| 284 | + | |
| 285 | + | |
| 286 | + | |
| 287 | + | |
| 288 | + | |
| 289 | + | |
| 290 | + | |
| 291 | + | |
| 292 | + | |
| 293 | + | |
| 294 | + | |
| 295 | + | |
| 296 | + | |
| 297 | + | |
| 298 | + | |
| 299 | + | |
| 300 | + | |
| 301 | + | |
| 302 | + | |
| 303 | + | |
| 304 | + | |
| 305 | + | |
| 306 | + | |
| 307 | + | |
| 308 | + | |
| 309 | + | |
| 310 | + | |
| 311 | + | |
| 312 | + | |
| 313 | + | |
| 314 | + | |
0 commit comments