|
| 1 | +# zhub Phase 1.7 + 1.2 + 1.1 + 1.8 design |
| 2 | + |
| 3 | +> Autonomous batch authorized 2026-05-09 ("1.7 + 1.2 + 1.1 in order amd many more"). User-approval gates skipped per autonomous override. Discipline preserved: explore context, propose, write spec, self-review, plan, TDD, verification. |
| 4 | +
|
| 5 | +## Context |
| 6 | + |
| 7 | +zhub at commit `f7a13f3`. 36/36 pytest, CI green (Python 3.10/3.11/3.12 + Kotlin job). Phases 0+1 → 1.0b → 1.5 → 1.3 shipped. Pocket has native zhub provider with manifest preview. Federation reads peer registries; chat routing is local-only. |
| 8 | + |
| 9 | +**This batch ships:** rate-limit enforcement (1.7), JS/TS client lib (1.2), cross-hub call routing (1.1), tool calls (1.8). "Many more" past 1.8 attempted if context permits — honest scope-down otherwise. |
| 10 | + |
| 11 | +## Phase 1.7 — Rate-limit enforcement |
| 12 | + |
| 13 | +### Problem |
| 14 | + |
| 15 | +Manifests advertise `rate_limit: "60/min"` today; the hub ignores it. Public endpoints can be hammered. Real public-deployment readiness needs server-side enforcement. |
| 16 | + |
| 17 | +### Approach |
| 18 | + |
| 19 | +In-memory sliding-window counter keyed by `api_key`. Lost on restart — that's fine for v0; clients that hit limits retry, no persistent damage. |
| 20 | + |
| 21 | +Parse the same string format manifests use: `"60/min"`, `"10/s"`, `"1000/day"`. Default fallback: `"60/min"` if manifest doesn't specify. |
| 22 | + |
| 23 | +Enforce on the HTTP `/v1/chat/completions` route (and `/registry/global`'s aggregator if useful — Phase 1.7b). When over limit, return HTTP 429 with `Retry-After: <seconds>` header, JSON body `{"error": {"code": "rate_limited", "message": "...", "retry_after": N}}`. |
| 24 | + |
| 25 | +Public endpoints (no auth header): use a per-IP fallback window — generous default (`60/min`) so casual `curl /healthz` isn't blocked. Authenticated calls use the publisher's manifest rate. |
| 26 | + |
| 27 | +### Out of scope |
| 28 | + |
| 29 | +- Persistent rate-limit state across restart (Phase 1.7b) |
| 30 | +- Per-connection-id rate limits (the publisher can decide) |
| 31 | +- Distributed rate limits across federated hubs (Phase 1.7c — needs a shared store) |
| 32 | + |
| 33 | +### TDD |
| 34 | + |
| 35 | +`tests/test_rate_limit.py`: |
| 36 | +1. Parse fixture: `"60/min"` → `(60, 60s)`, `"10/s"` → `(10, 1s)`, `"1000/day"` → `(1000, 86400s)`. Bad strings raise. |
| 37 | +2. Sliding-window counter over a fake clock: 3 hits in window → allow; 4th hit at same instant → deny; window expires → next hit allows. |
| 38 | +3. e2e: publish AI with `rate_limit: "3/s"` in manifest. Hit `/v1/chat/completions` 4 times rapidly. First 3 = 200, 4th = 429 with `Retry-After`. |
| 39 | + |
| 40 | +## Phase 1.2 — JS/TS client library |
| 41 | + |
| 42 | +### Goal |
| 43 | + |
| 44 | +Browsers and Node services connect to zhub via a clean SDK without going through `openai` library or hand-coding WebSockets. Same primitives as Python: `publish()` and `connect()`. |
| 45 | + |
| 46 | +### Layout |
| 47 | + |
| 48 | +``` |
| 49 | +js/ |
| 50 | + package.json — name "@zawwarsami/zhub", workspaces nothing |
| 51 | + tsconfig.json — strict, ESM-only target |
| 52 | + src/ |
| 53 | + index.ts — re-exports |
| 54 | + manifest.ts — Manifest, Capability types |
| 55 | + protocol.ts — Envelope + helpers (mirrors Python) |
| 56 | + client.ts — publish(), connect(), ZhubPublication, ZhubConnection |
| 57 | + errors.ts |
| 58 | + test/ |
| 59 | + protocol.test.ts — envelope round-trip |
| 60 | + e2e.test.ts — spin up the Python hub via fixture, run JS client against it |
| 61 | + README.md |
| 62 | +``` |
| 63 | + |
| 64 | +### Behaviors |
| 65 | + |
| 66 | +`publish(opts)`: |
| 67 | +- `name`, `description`, `chatHandler` (sync function or async generator). |
| 68 | +- Opens WebSocket. Auto-reconnect on disconnect (mirrors Phase 1.5). |
| 69 | +- Returns `ZhubPublication` with `apiKey`, `name`, `baseUrl`, `listConnections()`, `findCapability()`, `invoke()`. |
| 70 | + |
| 71 | +`connect(opts)`: |
| 72 | +- `aiName`, `apiKey`, `capabilities` map (name → {schema, handler}). |
| 73 | +- Returns `ZhubConnection` with `chat(messages)`, `chatStream(messages)` (async iterator), `close()`. |
| 74 | + |
| 75 | +### Browser caveat |
| 76 | + |
| 77 | +Browsers can't write raw TCP, but native `WebSocket` works. Browser builds therefore work for `connect()` mode (the most useful client-side direction). `publish()` from browser is theoretically supported (everything is WS) — useful for Pocket-as-publisher (a browser AI). Tests cover Node; browser path verified manually. |
| 78 | + |
| 79 | +### Build |
| 80 | + |
| 81 | +`tsc` produces ESM in `dist/`. Single bundle for browser is unnecessary at this stage — tree-shaken imports work via bundlers (Vite, esbuild, webpack). |
| 82 | + |
| 83 | +### TDD |
| 84 | + |
| 85 | +- `protocol.test.ts`: envelope JSON round-trips identical to Python's protocol.py output for at least register-publisher, chat-request, chat-response, invoke-request, invoke-result. |
| 86 | +- `e2e.test.ts`: a Vitest suite that spawns `python -m zhub.server` as a subprocess, waits for `/healthz`, then runs the JS client against it. publish + connect + chat + invoke happy paths. |
| 87 | + |
| 88 | +### CI |
| 89 | + |
| 90 | +Add `js-test` job to `.github/workflows/ci.yml`. Uses `actions/setup-node@v4`, runs `npm ci && npm test`. |
| 91 | + |
| 92 | +## Phase 1.1 — Cross-hub call routing |
| 93 | + |
| 94 | +### Goal |
| 95 | + |
| 96 | +Hub A receives a chat request for an AI registered on hub B. Hub A proxies through to B via HTTP, returns the response. Real federation — not just `/registry/global` discovery. |
| 97 | + |
| 98 | +### Approach |
| 99 | + |
| 100 | +When `POST /<name>/v1/chat/completions` arrives at Hub A: |
| 101 | + |
| 102 | +1. Look up `<name>` locally. If found → existing local path. |
| 103 | +2. Not found locally → check peer registry. If a peer hub has `<name>`, proxy. |
| 104 | +3. Build the proxy request: forward the body + `Authorization: Bearer <api_key>` to peer's `/<name>/v1/chat/completions`. |
| 105 | +4. Stream the response back if `stream:true`, else single-shot. |
| 106 | +5. If multiple peers claim the name, prefer the local match (already covered) → first peer in `peers` list otherwise. |
| 107 | +6. Origin annotation: include `X-Zhub-Origin: <peer_url>` header on the response so callers can see where it came from. |
| 108 | + |
| 109 | +### Auth |
| 110 | + |
| 111 | +The user's `api_key` is the publisher's key on hub B. Hub A doesn't know the key — it just forwards what the client sent. Hub B validates locally. **Hub A never sees the bearer key as anything but an opaque blob.** |
| 112 | + |
| 113 | +### Federation gotchas |
| 114 | + |
| 115 | +- **Loop prevention:** if hub A proxies to hub B which then tries to proxy back to hub A, infinite loop. Mitigation: `X-Zhub-Forwarded-By: <hub-id>` header. If hub sees its own id in this header, refuse with 508. |
| 116 | +- **Latency:** every hop = network round-trip. Document the cost. |
| 117 | +- **Streaming:** SSE bytes flow through hub A → hub B verbatim. Use `httpx.AsyncClient.stream()` for proxying; pump bytes one chunk at a time. |
| 118 | + |
| 119 | +### Out of scope |
| 120 | + |
| 121 | +- WebSocket connections from a connect()-side client to a federated AI (that requires either WS-tunneling or moving the connection state across hubs — Phase 1.1b). |
| 122 | +- Auth-server-mediated federation (signed peer relationships) — Phase 1.2 of the federation track, not this batch. |
| 123 | + |
| 124 | +### TDD |
| 125 | + |
| 126 | +`tests/test_cross_hub_routing.py`: |
| 127 | +1. Two hubs in-process (A, B). A peers B. |
| 128 | +2. Publish AI `bob` on B with chat handler that returns `"from-b"`. |
| 129 | +3. POST to A: `/bob/v1/chat/completions` → expect `"from-b"` response (proxied). |
| 130 | +4. POST to A with stream:true → expect SSE chunks arriving from B through A. |
| 131 | +5. Loop case: configure A to peer B, B to peer A, publish only on B. Calling A → A forwards to B → B serves locally (no loop). Then publish nothing on either, call A → 404 with peer-checked-but-not-found. |
| 132 | + |
| 133 | +## Phase 1.8 — Tool calls (function calling) end-to-end |
| 134 | + |
| 135 | +### Goal |
| 136 | + |
| 137 | +Standard OpenAI function-calling works through zhub. The AI's `tool_calls` in a chat response automatically map to invoke-request envelopes against connected clients' capabilities. Tool result returns to the AI as a `role: tool` message. |
| 138 | + |
| 139 | +### Why this composes naturally |
| 140 | + |
| 141 | +zhub already has bidirectional invoke. OpenAI tool calls are exactly: "AI wants the runtime to call tool X with args Y, then continue." That maps to: "AI emits invoke-request through hub, gets back invoke-result, continues conversation." |
| 142 | + |
| 143 | +### Architecture |
| 144 | + |
| 145 | +``` |
| 146 | +[client app] |
| 147 | + POST /<name>/v1/chat/completions |
| 148 | + body has tool definitions (or capabilities discovered from connected client manifest) |
| 149 | + │ |
| 150 | + ▼ |
| 151 | +[hub] forwards chat-request to publisher with tools array |
| 152 | + │ |
| 153 | + ▼ |
| 154 | +[AI publisher] returns chat-response with tool_calls (instead of plain content) |
| 155 | + │ |
| 156 | + ▼ |
| 157 | +[hub] sees tool_calls in publisher's response |
| 158 | + - if tool name matches a connected client's capability: |
| 159 | + → automatic: invoke-request to that client |
| 160 | + → wait for invoke-result |
| 161 | + → append role=tool message |
| 162 | + → forward back to publisher (recursive chat) |
| 163 | + - else: return tool_calls to client app as normal OpenAI shape |
| 164 | + │ |
| 165 | + ▼ |
| 166 | +[publisher] continues with tool result in conversation, returns final assistant message |
| 167 | +``` |
| 168 | + |
| 169 | +### Wire format |
| 170 | + |
| 171 | +Capability declarations from connect()-side already include the JSON schema. Hub turns each capability into a `tool` entry: |
| 172 | + |
| 173 | +```json |
| 174 | +{ |
| 175 | + "type": "function", |
| 176 | + "function": { |
| 177 | + "name": "send_whatsapp", |
| 178 | + "description": "Send WhatsApp via Loki", |
| 179 | + "parameters": {...JSON schema...} |
| 180 | + } |
| 181 | +} |
| 182 | +``` |
| 183 | + |
| 184 | +These get injected into the chat-request envelope so the AI sees them. AI replies with `tool_calls`; hub maps each to the right connection_id and invokes. |
| 185 | + |
| 186 | +### Operator opt-out |
| 187 | + |
| 188 | +By default: hub auto-resolves tool_calls that match capabilities (saves operator a round-trip). Opt-out: client request adds `X-Zhub-Tool-Resolve: client` header → hub returns tool_calls verbatim and the operator handles them. |
| 189 | + |
| 190 | +### Out of scope |
| 191 | + |
| 192 | +- Parallel tool calls (OpenAI now supports multiple tool_calls per response — phase 1.8b) |
| 193 | +- Tool streaming (chunks of tool_calls arriving via SSE) — phase 1.8c |
| 194 | +- Tool retries on transient invoke errors — phase 1.8d |
| 195 | + |
| 196 | +### TDD |
| 197 | + |
| 198 | +`tests/test_tool_calls.py`: |
| 199 | +1. Publish AI whose chat handler returns a chat-response with `tool_calls: [{name: "send_whatsapp", args: {to: "Ammi", message: "ok"}}]` on first call, then a normal text response on second call (mock the LLM with a deterministic two-step handler). |
| 200 | +2. Connect a client exposing `send_whatsapp` capability (returns `{ok: true, delivered: true}`). |
| 201 | +3. POST to `/<name>/v1/chat/completions`. |
| 202 | +4. Expect: hub auto-resolves the tool_call, appends role=tool message with the result, sends back to publisher, returns publisher's final text response — **client app sees normal chat completion with no tool_calls in the visible response**, but a `tool_results` log in the response usage block. |
| 203 | +5. Same flow with `X-Zhub-Tool-Resolve: client` → expect tool_calls returned verbatim, no automatic invoke. |
| 204 | + |
| 205 | +## Self-review |
| 206 | + |
| 207 | +**Placeholder scan:** none. Each phase has concrete deliverables, TDD plan, and out-of-scope cuts. |
| 208 | + |
| 209 | +**Internal consistency:** |
| 210 | +- Phase 1.7 rate-limit format matches manifest.rate_limit type already declared (string like `"60/min"`). |
| 211 | +- Phase 1.2 JS client follows the same Envelope schema as Python — wire-compatible. |
| 212 | +- Phase 1.1 routing reuses Phase 1.0b's PeerRegistry for peer URL discovery. |
| 213 | +- Phase 1.8 tool calls layer on top of Phase 1.0/1.5 invoke-request path — same connection_id routing, same wire envelope. |
| 214 | + |
| 215 | +**Scope check:** four phases is a lot. Each has an honest size. 1.1 (cross-hub) is the biggest — if it bumps past time budget, ship 1.1a (non-streaming proxy) and defer 1.1b (streaming proxy) to follow-up. The other three should land cleanly. |
| 216 | + |
| 217 | +**Ambiguity check:** |
| 218 | +- 1.7: "60/min" parsing — clarified to support `s/min/hour/day` units, fallback `"60/min"`. |
| 219 | +- 1.2: ESM-only emit — explicit; CJS path deferred. |
| 220 | +- 1.1: loop prevention via `X-Zhub-Forwarded-By` header — explicit. |
| 221 | +- 1.8: opt-out via `X-Zhub-Tool-Resolve: client` header — explicit. |
| 222 | + |
| 223 | +## Transition |
| 224 | + |
| 225 | +Skipping the user-review gate per autonomous override. Going directly to TDD execution per phase. |
0 commit comments