Skip to content

Commit 8e65111

Browse files
OriNachumclaude
andauthored
Fallback model + single front OpenAI gateway (3-container fleet) (#16)
* Add fallback model + single front OpenAI gateway (3-container fleet) `model init --fleet` scaffolds a three-container deployment — two always-warm vLLM backends (a dense primary + an MoE fallback) behind one stdlib gateway — managed by model-gear as `model-gear-gateway` / `model-gear-vllm-primary` / `model-gear-vllm-fallback`. The gateway fronts both models on the host port acp already expects (8000): it routes each request by its `model` field, defaults an unknown/missing name to the primary, and fails over to the other backend when the chosen one refuses the connection or returns a 5xx before the response body (4xx verbatim; no mid-stream retry). SSE streams are relayed chunk-by-chunk. - New `model_gear/gateway/` package (pure stdlib http.server + http.client, no runtime deps): `_routing.py` (routing/failover order), `_config.py` (env → config), `server.py` (`handle_post` failover seam + upstream client + handler). - New verbs: `model init --fleet` (pins MODEL_GEAR_VERSION to the running release) and `model fleet up | down | status` (up/down dry-run by default). - `_compose.py`: template registry (SINGLE/FLEET) + `templates=` arg (single-model stays default), `compose_up_build`, `FLEET_CONTAINERS`. - Fleet `.env` mirrors VLLM_* (= primary) so status/whoami/doctor stay coherent; `model switch` remains single-model only. - Docs: docs/gateway-fleet.md, docs/qwen3.6-35b-a3b-nvfp4.md, README fleet section, `model explain fleet` / `gateway` entries. - Tests: 36 new (routing/config/body helpers, handle_post failover matrix, loopback relay + chunked streaming, open_upstream, fleet verbs, init --fleet). Full suite 125 passed; coverage 90%. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * Address SonarCloud findings on the fleet - Dockerfile.gateway: pin the model-gear install unconditionally (`==${MODEL_GEAR_VERSION}`, no unpinned "latest" fallback) so the image is reproducible (docker:S8544), and run as a non-root `gateway` user (clears the root-user security hotspot). MODEL_GEAR_VERSION is now required; init --fleet fills it and dev boxes set a TestPyPI .devN. - fleet.py: restructure cmd_fleet_up / cmd_fleet_down to a single return path (python:S3516) and hoist the repeated --json / --port help strings into _JSON_HELP / _PORT_HELP constants (python:S1192). - Docs/env.example updated for the required pinned version. The remaining new-code hotspots are the internal-network `http://vllm-*` URLs (no TLS between sibling compose containers, by design) — reviewed as safe. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * gateway: decode chunked request bodies (Qodo review) `_read_body` honored only Content-Length, so a valid HTTP/1.1 `Transfer-Encoding: chunked` request body was forwarded empty (misrouted to the default + invalid JSON upstream). Add a stdlib `read_chunked_body` decoder (ignores chunk extensions, caps total size) and use it when Content-Length is absent but the request is chunked. Adds a unit test for the decoder and a loopback integration test posting a chunked body through the gateway. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * gateway: fail over on a malformed backend URL (Qodo review) `open_upstream` caught only OSError, so a malformed `base_url` — e.g. a non-numeric port (`parts.port` raises ValueError) or a bad host/path (`http.client.InvalidURL`) — propagated as an uncaught 500 instead of failing over. Wrap the parse + connect + request in one guard that maps (OSError, http.client.HTTPException, ValueError) to UpstreamError, so a misconfigured backend fails over (or 502s) cleanly. Adds a test. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * gateway Dockerfile: merge the two RUN layers into one (Sonar) The pip-install and useradd both run as root before the USER switch, so there's no reason for two image layers. Combine them with `&&`. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
1 parent d1234ea commit 8e65111

27 files changed

Lines changed: 2044 additions & 27 deletions

CHANGELOG.md

Lines changed: 42 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,48 @@ All notable changes to this project are documented here. The format is based on
44
[Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and this project
55
adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
66

7+
## [0.9.0] - 2026-05-28
8+
9+
### Added
10+
11+
- **Fallback model + single front OpenAI gateway ("fleet").** A new
12+
scaffold-based deployment runs **two always-warm vLLM backends behind one
13+
stdlib gateway** that model-gear manages as three containers
14+
(`model-gear-gateway`, `model-gear-vllm-primary`, `model-gear-vllm-fallback`).
15+
The gateway routes each request by its `model` field, defaults an
16+
unknown/missing name to the primary, and fails over to the other backend when
17+
the chosen one refuses the connection or returns a 5xx **before** the response
18+
body (4xx is returned verbatim; no mid-stream retry). SSE streams are relayed
19+
chunk-by-chunk. Default fallback: the MoE `mmangkad/Qwen3.6-35B-A3B-NVFP4`.
20+
- **New gateway package `model_gear/gateway/`** — a pure-stdlib
21+
(`http.server` + `http.client`, no runtime deps) reverse proxy: `_routing.py`
22+
(pure name/alias/default routing + failover ordering), `_config.py` (env →
23+
routing table + server config), `server.py` (the `handle_post` failover seam,
24+
upstream client, and `ThreadingHTTPServer` handler), run as
25+
`python -m model_gear.gateway`.
26+
- **`model init --fleet`** scaffolds the fleet templates
27+
(`docker-compose.yml` + `.env` + `Dockerfile.gateway`) and pins
28+
`MODEL_GEAR_VERSION` to the running release; **`model fleet up | down |
29+
status`** drives the deployment (`up`/`down` dry-run by default, `--apply` to
30+
commit; `status` is read-only and reports all three containers + the gateway
31+
`/health` + `/v1/models`).
32+
- **Docs:** `docs/gateway-fleet.md` (topology, routing/failover, memory,
33+
verbs), `docs/qwen3.6-35b-a3b-nvfp4.md` (the MoE fallback), a README "fleet"
34+
section, and `model explain fleet` / `model explain gateway` entries.
35+
36+
### Changed
37+
38+
- `model_gear/runtime/_compose.py` gained a template registry
39+
(`SINGLE_TEMPLATES` / `FLEET_TEMPLATES`), a `templates=` argument on
40+
`scaffold_plan` / `write_scaffold` (single-model stays the default — existing
41+
callers unchanged), a `compose_up_build` helper, and `FLEET_CONTAINERS`.
42+
- The fleet `.env` mirrors `VLLM_MODEL` / `VLLM_SERVED_NAME` /
43+
`VLLM_TOOL_CALL_PARSER` (= the primary) so the read-only single-model verbs
44+
(`status` / `whoami` / `doctor`) stay coherent on a fleet deployment.
45+
`model switch` remains single-model only.
46+
47+
### Fixed
48+
749
## [0.8.1] - 2026-05-27
850

951
### Changed

README.md

Lines changed: 33 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -77,6 +77,36 @@ model whose repo ships custom modeling code. If vLLM rejects the `nvidia/`
7777
ModelOpt checkpoint, set `VLLM_MODEL` to the vLLM-native `RedHatAI/Qwen3-32B-NVFP4`
7878
and drop `--quantization` from the compose `command`.
7979

80+
## Running two models behind one gateway (fleet)
81+
82+
`model init --fleet` scaffolds a **three-container** deployment instead of one:
83+
two always-warm vLLM backends (a primary + an MoE fallback) and a single stdlib
84+
**gateway** that fronts them on the host port the acp `vllm-local` provider
85+
already expects. The gateway routes each request by its `model` field, defaults an
86+
unknown/missing name to the primary, and fails over to the other backend if the
87+
chosen one is down — so existing single-model clients keep working unchanged while
88+
a second model becomes addressable by name.
89+
90+
```bash
91+
model init --fleet --apply # ~/.model-gear/{docker-compose.yml,.env,Dockerfile.gateway}
92+
docker login nvcr.io # NGC API key for the vLLM image
93+
model fleet up --apply # builds the gateway image + starts all three
94+
model fleet status # container states + gateway /health + /v1/models
95+
```
96+
97+
```bash
98+
curl -s http://localhost:8000/v1/models # lists BOTH served models
99+
# route explicitly by name; an unknown/missing model falls back to the primary
100+
curl -s http://localhost:8000/v1/chat/completions -d '{"model":"mmangkad/Qwen3.6-35B-A3B-NVFP4","messages":[...]}'
101+
```
102+
103+
Both models stay loaded, so set `PRIMARY_GPU_MEM_UTIL` + `FALLBACK_GPU_MEM_UTIL`
104+
in the fleet `.env` to sum well under 1.0 (they share the 128 GB unified memory).
105+
`model switch` is single-model only — change fleet models by editing the fleet
106+
`.env` and re-running `model fleet up --apply`. See `model explain fleet` /
107+
`model explain gateway` for the routing and failover semantics, and
108+
[`docs/gateway-fleet.md`](docs/gateway-fleet.md) for the full topology.
109+
80110
### Per-model notes
81111

82112
Each runtime model has a doc under `docs/` recording how to run it, live test
@@ -87,6 +117,9 @@ results, and caveats:
87117
- [`docs/qwen3.6-27b-nvfp4.md`](docs/qwen3.6-27b-nvfp4.md) — a **candidate**
88118
(`mmangkad/Qwen3.6-27B-NVFP4`), load-tested on DGX Spark; loads under the
89119
current vLLM image but is slower on decode, so the 32B stays.
120+
- [`docs/qwen3.6-35b-a3b-nvfp4.md`](docs/qwen3.6-35b-a3b-nvfp4.md) — the **MoE
121+
fallback** (`mmangkad/Qwen3.6-35B-A3B-NVFP4`) the gateway fleet pairs with the
122+
32B; ~3B active params decode much faster on this box.
90123

91124
The numbers in each doc come from `model switch <model> --apply` then `model
92125
assess` (correctness) and `model benchmark` (throughput). `model overview --list`

docs/gateway-fleet.md

Lines changed: 108 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,108 @@
1+
# Fleet: two models behind one OpenAI gateway
2+
3+
The **fleet** runs two always-warm vLLM models behind a single stdlib
4+
OpenAI-compatible gateway, managed by model-gear as three Docker containers. It is
5+
an alternative to the single-model deployment — scaffold it with
6+
`model init --fleet` (the single-model `model init` is unchanged and remains the
7+
default).
8+
9+
## Why
10+
11+
The single-model deployment serves one model on `:8000` and `model switch` swaps
12+
it (freeing the prior model). The fleet instead keeps **both** models loaded and
13+
puts one OpenAI endpoint in front of them, so:
14+
15+
- existing clients (the acp `vllm-local` provider, `curl`, …) keep pointing at
16+
`:8000` and keep working — an unknown/missing `model` defaults to the primary;
17+
- a second model is addressable by name in the same `/v1/...` calls;
18+
- if the chosen backend is down, the gateway fails over to the other one.
19+
20+
On the DGX Spark (GB10, 128 GB unified memory) both ~30B-class NVFP4 models fit at
21+
once; the fleet pairs the dense primary with an **MoE** fallback (`A3B` ≈ 3B active
22+
params) that decodes much faster, so the fast model stays fast.
23+
24+
## Topology
25+
26+
```text
27+
client / acp ──:8000──▶ model-gear-gateway (python -m model_gear.gateway)
28+
│ route by `model` → default → failover
29+
├──▶ model-gear-vllm-primary :8000 (internal)
30+
└──▶ model-gear-vllm-fallback :8000 (internal)
31+
```
32+
33+
Three containers, all `restart: unless-stopped`:
34+
35+
| Container | Role | Host port |
36+
|---|---|---|
37+
| `model-gear-gateway` | stdlib reverse proxy (the single OpenAI front) | `${VLLM_PORT:-8000}` |
38+
| `model-gear-vllm-primary` | primary model (default: `nvidia/Qwen3-32B-NVFP4`) | internal only |
39+
| `model-gear-vllm-fallback` | MoE fallback (default: `mmangkad/Qwen3.6-35B-A3B-NVFP4`) | internal only |
40+
41+
The backends are reachable only on the compose network
42+
(`http://vllm-primary:8000`, `http://vllm-fallback:8000`); only the gateway is
43+
published to the host. The gateway needs no Docker socket access — compose owns
44+
the lifecycle; the gateway only routes.
45+
46+
## The gateway
47+
48+
A pure-stdlib (`http.server` + `http.client`, no third-party deps) reverse proxy:
49+
50+
- **Name routing** — a request's `model` routes to the backend that serves it,
51+
plus any `GATEWAY_ALIASES`. The forwarded body's `model` is rewritten to the
52+
backend's `--served-model-name` so the backend accepts aliased/default routes.
53+
- **Default model** — a missing or unknown `model` routes to
54+
`GATEWAY_DEFAULT_MODEL` (the primary).
55+
- **Failover** — if the chosen backend refuses the connection or returns a 5xx
56+
**before any response body**, the request is retried against the other backend.
57+
A 4xx is a client error (returned verbatim, no failover). Once a 2xx body starts
58+
streaming there is no retry — the client already has bytes.
59+
- **Streaming**`"stream": true` (SSE) is relayed chunk-by-chunk with per-chunk
60+
flushing; normal JSON is buffered with `Content-Length`.
61+
- **Endpoints**`/v1/chat/completions`, `/v1/completions`, `/v1/embeddings`
62+
(proxied), `/v1/models` (lists both backends), `/health` (gateway liveness).
63+
64+
The gateway image is built from the scaffolded `Dockerfile.gateway`
65+
(`pip install model-gear==${MODEL_GEAR_VERSION}`, as a non-root user); `model init
66+
--fleet` pins `MODEL_GEAR_VERSION` to the running model-gear release. The version
67+
is required (pinning keeps the image reproducible); from-source/dev boxes that run
68+
ahead of a PyPI release point `MODEL_GEAR_VERSION` at a published TestPyPI `.devN`
69+
build.
70+
71+
## Verbs
72+
73+
```bash
74+
model init --fleet --apply # scaffold compose + .env + Dockerfile.gateway
75+
model fleet up --apply # docker compose up -d --build, wait for gateway /health
76+
model fleet status # each container's state + gateway /health + /v1/models
77+
model fleet down --apply # docker compose down
78+
```
79+
80+
`model fleet up` / `down` are **dry-run by default**; pass `--apply` to commit.
81+
`--compose-dir` overrides the deployment dir (default `$MODEL_GEAR_DIR` or
82+
`~/.model-gear`). `model fleet status` is read-only.
83+
84+
**`model switch` does not drive the fleet** — it rewrites the single-model
85+
`VLLM_*` keys. Change fleet models by editing the fleet `.env`
86+
(`PRIMARY_MODEL` / `FALLBACK_MODEL` and their `*_SERVED_NAME` / `*_GPU_MEM_UTIL`
87+
/ `*_TOOL_CALL_PARSER` / `*_QUANTIZATION`) and re-running `model fleet up --apply`.
88+
89+
## Memory (both warm)
90+
91+
Both models stay resident, so `PRIMARY_GPU_MEM_UTIL` + `FALLBACK_GPU_MEM_UTIL`
92+
must sum well under 1.0 of the 128 GB. The scaffolded defaults are **0.40** +
93+
**0.35** (≈ 96 GB reserved, leaving headroom for the OS and KV growth). These are
94+
estimates for a dense 32B + a 35B-A3B MoE — **validate live** (watch `nvidia-smi`
95+
at `model fleet up`; OOM is the top operational risk) and tune the two values.
96+
97+
Note the throughput trade-off: decode is memory-bandwidth bound and the bandwidth
98+
(~273 GB/s) is **shared**. The MoE reads only its active experts per token, so it
99+
stays fast; two backends decoding *simultaneously* split the bandwidth. The
100+
gateway routes one request to one backend, so a single client sees full speed.
101+
102+
## Coherence with the single-model verbs
103+
104+
The fleet `.env` mirrors `VLLM_MODEL` / `VLLM_SERVED_NAME` / `VLLM_TOOL_CALL_PARSER`
105+
(= the primary's) so the read-only single-model verbs (`model status`,
106+
`model whoami`, `model doctor`'s `env_coherence` check) stay sensible on a fleet
107+
deployment. `culture.yaml` needs no change: its `model: vllm-local/nvidia/Qwen3-32B-NVFP4`
108+
resolves through the gateway on `:8000` as the default.

docs/qwen3.6-35b-a3b-nvfp4.md

Lines changed: 82 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,82 @@
1+
# Fallback model: `mmangkad/Qwen3.6-35B-A3B-NVFP4`
2+
3+
The **MoE fallback** the gateway fleet pairs with the dense primary
4+
(`nvidia/Qwen3-32B-NVFP4`). See [`docs/gateway-fleet.md`](gateway-fleet.md) for the
5+
fleet topology; this doc records what the model is and how it is configured in the
6+
fleet.
7+
8+
Source: <https://huggingface.co/mmangkad/Qwen3.6-35B-A3B-NVFP4>.
9+
10+
> **Status: configured, not yet load-tested on this hardware.** The numbers below
11+
> are *expectations* from the architecture, not measured values. Fill in the
12+
> Benchmark table from a live `model fleet up``model assess` / `model benchmark`
13+
> run (and confirm the quantization/parser caveats) before relying on them.
14+
15+
## What it is
16+
17+
- An **NVFP4 (Mixture-of-Experts)** checkpoint: ~35B total parameters, **~3B
18+
active per token** (`A3B`). vLLM loads *all* experts into memory; the small
19+
active set only reduces per-token compute.
20+
- Decode is memory-bandwidth bound on the GB10 (~273 GB/s shared). Reading only
21+
~3B active params per token (≈1.5 GB at 4-bit) gives an **expected decode
22+
ceiling far above the dense 32B** (which reads ~18 GB/token) — the reason it is
23+
the fast fallback. *Confirm live.*
24+
25+
## How it runs in the fleet
26+
27+
Configured via the `FALLBACK_*` keys in the fleet `.env` (scaffolded by
28+
`model init --fleet`); served by the `model-gear-vllm-fallback` container:
29+
30+
```dotenv
31+
FALLBACK_MODEL=mmangkad/Qwen3.6-35B-A3B-NVFP4
32+
FALLBACK_SERVED_NAME=mmangkad/Qwen3.6-35B-A3B-NVFP4
33+
FALLBACK_MAX_MODEL_LEN=32768
34+
FALLBACK_GPU_MEM_UTIL=0.35 # both models warm: keep primary+fallback well under 1.0
35+
FALLBACK_TOOL_CALL_PARSER=qwen3_coder
36+
FALLBACK_QUANTIZATION=modelopt_fp4
37+
```
38+
39+
Address it through the gateway by name (or set `GATEWAY_ALIASES` for a short
40+
alias):
41+
42+
```bash
43+
curl -s http://localhost:8000/v1/chat/completions \
44+
-d '{"model":"mmangkad/Qwen3.6-35B-A3B-NVFP4","messages":[{"role":"user","content":"hi"}]}'
45+
```
46+
47+
## Caveats to confirm on first load
48+
49+
1. **Tool-call format.** Qwen3.6 emits the Qwen3-Coder **XML** function format, so
50+
the backend is served with `--tool-call-parser=qwen3_coder` (not the `hermes`
51+
parser the dense Qwen3-32B uses). `model_gear.runtime._parser.infer_parser`
52+
already maps `qwen3.6``qwen3_coder`. Verify a `tool_choice:"auto"` probe
53+
returns a `finish` tool call.
54+
2. **Quantization format.** The fleet defaults `FALLBACK_QUANTIZATION=modelopt_fp4`
55+
(as for the `nvidia/` checkpoints). This community (`mmangkad`) checkpoint may
56+
instead be a compressed-tensors NVFP4 — if vLLM rejects `modelopt_fp4`, drop or
57+
change `FALLBACK_QUANTIZATION`.
58+
3. **`--trust-remote-code`.** The fleet compose omits it (as the single-model
59+
template does). If this checkpoint ships custom modeling code, vLLM will say so
60+
on load; add it back deliberately (it lets repo code run in-container alongside
61+
`HF_TOKEN` and the mounted cache).
62+
4. **Architecture support.** Confirm the engine registers the checkpoint's
63+
architecture, as done for the 27B sibling:
64+
`docker exec model-gear-vllm-fallback python3 -c "from
65+
vllm.model_executor.models.registry import ModelRegistry;
66+
print(ModelRegistry.get_supported_archs())"`.
67+
68+
## Benchmark — pending
69+
70+
Fill from a live run (`model fleet up --apply`, then `model assess` / `model
71+
benchmark` against `:8000` with this model's name):
72+
73+
| Property | Value |
74+
|---|---|
75+
| Health / `max_model_len` | *pending* |
76+
| Correctness (`17×23`, train 14:45→17:10) | *pending* |
77+
| Reasoning trace field | *pending* |
78+
| Tool calling (`tool_choice:auto`, `qwen3_coder`) | *pending* |
79+
| **Decode throughput** | *pending* (expected ≫ the dense 32B's ~9.7 tok/s) |
80+
| Prefill (~2K tokens) | *pending* |
81+
| GPU memory reserved (at `FALLBACK_GPU_MEM_UTIL`) | *pending* |
82+
| Co-resident total (primary + fallback) | *pending* — watch for OOM |

model_gear/cli/__init__.py

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -66,6 +66,7 @@ def _build_parser() -> argparse.ArgumentParser:
6666
from model_gear.cli._commands import cli as _cli_group
6767
from model_gear.cli._commands import doctor as _doctor_cmd
6868
from model_gear.cli._commands import explain as _explain_cmd
69+
from model_gear.cli._commands import fleet as _fleet_cmd
6970
from model_gear.cli._commands import init as _init_cmd
7071
from model_gear.cli._commands import learn as _learn_cmd
7172
from model_gear.cli._commands import overview as _overview_cmd
@@ -96,6 +97,7 @@ def _build_parser() -> argparse.ArgumentParser:
9697
_assess_cmd.register(sub)
9798
_benchmark_cmd.register(sub)
9899
_init_cmd.register(sub)
100+
_fleet_cmd.register(sub)
99101

100102
# Agent-first / introspection verbs (sibling rubric).
101103
_whoami_cmd.register(sub)

0 commit comments

Comments
 (0)