Skip to content

Commit b0e168c

Browse files
Zawwarsami16claude
andcommitted
docs: design spec for phase 1.7 + 1.2 + 1.1 + 1.8 batch
per superpowers brainstorming — explore, propose, write spec, self-review. user-approval gate skipped per father's "1.7 + 1.2 + 1.1 + many more" autonomous directive. scope of the batch: 1.7 rate-limit enforcement (in-memory sliding window, 429 with Retry-After, parsed from manifest.rate_limit string) 1.2 JS/TS client library (js/ workspace, mirrors Python publish/ connect, ESM-only, CI parallel job) 1.1 cross-hub call routing (real federation — hub A proxies chat to AI on hub B via HTTP, streaming preserved, loop prevention via X-Zhub-Forwarded-By header) 1.8 tool calls end-to-end (OpenAI function-calling: AI's tool_calls map to bidirectional invoke-request on connected client capabilities; opt-out via X-Zhub-Tool-Resolve header) self-review surfaced + clarified inline: - 1.7 unit set: s/min/hour/day. fallback "60/min". - 1.2 emit format: ESM-only first, CJS deferred. - 1.1 loop prevention header explicit. - 1.8 opt-out header explicit. scope honesty: 1.1 is the biggest. 1.1a (non-stream proxy) ships in this batch. 1.1b (stream proxy) defers to follow-up if time runs out. next: writing-plans skill, then TDD execution. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
1 parent f7a13f3 commit b0e168c

1 file changed

Lines changed: 225 additions & 0 deletions

File tree

Lines changed: 225 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,225 @@
1+
# zhub Phase 1.7 + 1.2 + 1.1 + 1.8 design
2+
3+
> Autonomous batch authorized 2026-05-09 ("1.7 + 1.2 + 1.1 in order amd many more"). User-approval gates skipped per autonomous override. Discipline preserved: explore context, propose, write spec, self-review, plan, TDD, verification.
4+
5+
## Context
6+
7+
zhub at commit `f7a13f3`. 36/36 pytest, CI green (Python 3.10/3.11/3.12 + Kotlin job). Phases 0+1 → 1.0b → 1.5 → 1.3 shipped. Pocket has native zhub provider with manifest preview. Federation reads peer registries; chat routing is local-only.
8+
9+
**This batch ships:** rate-limit enforcement (1.7), JS/TS client lib (1.2), cross-hub call routing (1.1), tool calls (1.8). "Many more" past 1.8 attempted if context permits — honest scope-down otherwise.
10+
11+
## Phase 1.7 — Rate-limit enforcement
12+
13+
### Problem
14+
15+
Manifests advertise `rate_limit: "60/min"` today; the hub ignores it. Public endpoints can be hammered. Real public-deployment readiness needs server-side enforcement.
16+
17+
### Approach
18+
19+
In-memory sliding-window counter keyed by `api_key`. Lost on restart — that's fine for v0; clients that hit limits retry, no persistent damage.
20+
21+
Parse the same string format manifests use: `"60/min"`, `"10/s"`, `"1000/day"`. Default fallback: `"60/min"` if manifest doesn't specify.
22+
23+
Enforce on the HTTP `/v1/chat/completions` route (and `/registry/global`'s aggregator if useful — Phase 1.7b). When over limit, return HTTP 429 with `Retry-After: <seconds>` header, JSON body `{"error": {"code": "rate_limited", "message": "...", "retry_after": N}}`.
24+
25+
Public endpoints (no auth header): use a per-IP fallback window — generous default (`60/min`) so casual `curl /healthz` isn't blocked. Authenticated calls use the publisher's manifest rate.
26+
27+
### Out of scope
28+
29+
- Persistent rate-limit state across restart (Phase 1.7b)
30+
- Per-connection-id rate limits (the publisher can decide)
31+
- Distributed rate limits across federated hubs (Phase 1.7c — needs a shared store)
32+
33+
### TDD
34+
35+
`tests/test_rate_limit.py`:
36+
1. Parse fixture: `"60/min"``(60, 60s)`, `"10/s"``(10, 1s)`, `"1000/day"``(1000, 86400s)`. Bad strings raise.
37+
2. Sliding-window counter over a fake clock: 3 hits in window → allow; 4th hit at same instant → deny; window expires → next hit allows.
38+
3. e2e: publish AI with `rate_limit: "3/s"` in manifest. Hit `/v1/chat/completions` 4 times rapidly. First 3 = 200, 4th = 429 with `Retry-After`.
39+
40+
## Phase 1.2 — JS/TS client library
41+
42+
### Goal
43+
44+
Browsers and Node services connect to zhub via a clean SDK without going through `openai` library or hand-coding WebSockets. Same primitives as Python: `publish()` and `connect()`.
45+
46+
### Layout
47+
48+
```
49+
js/
50+
package.json — name "@zawwarsami/zhub", workspaces nothing
51+
tsconfig.json — strict, ESM-only target
52+
src/
53+
index.ts — re-exports
54+
manifest.ts — Manifest, Capability types
55+
protocol.ts — Envelope + helpers (mirrors Python)
56+
client.ts — publish(), connect(), ZhubPublication, ZhubConnection
57+
errors.ts
58+
test/
59+
protocol.test.ts — envelope round-trip
60+
e2e.test.ts — spin up the Python hub via fixture, run JS client against it
61+
README.md
62+
```
63+
64+
### Behaviors
65+
66+
`publish(opts)`:
67+
- `name`, `description`, `chatHandler` (sync function or async generator).
68+
- Opens WebSocket. Auto-reconnect on disconnect (mirrors Phase 1.5).
69+
- Returns `ZhubPublication` with `apiKey`, `name`, `baseUrl`, `listConnections()`, `findCapability()`, `invoke()`.
70+
71+
`connect(opts)`:
72+
- `aiName`, `apiKey`, `capabilities` map (name → {schema, handler}).
73+
- Returns `ZhubConnection` with `chat(messages)`, `chatStream(messages)` (async iterator), `close()`.
74+
75+
### Browser caveat
76+
77+
Browsers can't write raw TCP, but native `WebSocket` works. Browser builds therefore work for `connect()` mode (the most useful client-side direction). `publish()` from browser is theoretically supported (everything is WS) — useful for Pocket-as-publisher (a browser AI). Tests cover Node; browser path verified manually.
78+
79+
### Build
80+
81+
`tsc` produces ESM in `dist/`. Single bundle for browser is unnecessary at this stage — tree-shaken imports work via bundlers (Vite, esbuild, webpack).
82+
83+
### TDD
84+
85+
- `protocol.test.ts`: envelope JSON round-trips identical to Python's protocol.py output for at least register-publisher, chat-request, chat-response, invoke-request, invoke-result.
86+
- `e2e.test.ts`: a Vitest suite that spawns `python -m zhub.server` as a subprocess, waits for `/healthz`, then runs the JS client against it. publish + connect + chat + invoke happy paths.
87+
88+
### CI
89+
90+
Add `js-test` job to `.github/workflows/ci.yml`. Uses `actions/setup-node@v4`, runs `npm ci && npm test`.
91+
92+
## Phase 1.1 — Cross-hub call routing
93+
94+
### Goal
95+
96+
Hub A receives a chat request for an AI registered on hub B. Hub A proxies through to B via HTTP, returns the response. Real federation — not just `/registry/global` discovery.
97+
98+
### Approach
99+
100+
When `POST /<name>/v1/chat/completions` arrives at Hub A:
101+
102+
1. Look up `<name>` locally. If found → existing local path.
103+
2. Not found locally → check peer registry. If a peer hub has `<name>`, proxy.
104+
3. Build the proxy request: forward the body + `Authorization: Bearer <api_key>` to peer's `/<name>/v1/chat/completions`.
105+
4. Stream the response back if `stream:true`, else single-shot.
106+
5. If multiple peers claim the name, prefer the local match (already covered) → first peer in `peers` list otherwise.
107+
6. Origin annotation: include `X-Zhub-Origin: <peer_url>` header on the response so callers can see where it came from.
108+
109+
### Auth
110+
111+
The user's `api_key` is the publisher's key on hub B. Hub A doesn't know the key — it just forwards what the client sent. Hub B validates locally. **Hub A never sees the bearer key as anything but an opaque blob.**
112+
113+
### Federation gotchas
114+
115+
- **Loop prevention:** if hub A proxies to hub B which then tries to proxy back to hub A, infinite loop. Mitigation: `X-Zhub-Forwarded-By: <hub-id>` header. If hub sees its own id in this header, refuse with 508.
116+
- **Latency:** every hop = network round-trip. Document the cost.
117+
- **Streaming:** SSE bytes flow through hub A → hub B verbatim. Use `httpx.AsyncClient.stream()` for proxying; pump bytes one chunk at a time.
118+
119+
### Out of scope
120+
121+
- WebSocket connections from a connect()-side client to a federated AI (that requires either WS-tunneling or moving the connection state across hubs — Phase 1.1b).
122+
- Auth-server-mediated federation (signed peer relationships) — Phase 1.2 of the federation track, not this batch.
123+
124+
### TDD
125+
126+
`tests/test_cross_hub_routing.py`:
127+
1. Two hubs in-process (A, B). A peers B.
128+
2. Publish AI `bob` on B with chat handler that returns `"from-b"`.
129+
3. POST to A: `/bob/v1/chat/completions` → expect `"from-b"` response (proxied).
130+
4. POST to A with stream:true → expect SSE chunks arriving from B through A.
131+
5. Loop case: configure A to peer B, B to peer A, publish only on B. Calling A → A forwards to B → B serves locally (no loop). Then publish nothing on either, call A → 404 with peer-checked-but-not-found.
132+
133+
## Phase 1.8 — Tool calls (function calling) end-to-end
134+
135+
### Goal
136+
137+
Standard OpenAI function-calling works through zhub. The AI's `tool_calls` in a chat response automatically map to invoke-request envelopes against connected clients' capabilities. Tool result returns to the AI as a `role: tool` message.
138+
139+
### Why this composes naturally
140+
141+
zhub already has bidirectional invoke. OpenAI tool calls are exactly: "AI wants the runtime to call tool X with args Y, then continue." That maps to: "AI emits invoke-request through hub, gets back invoke-result, continues conversation."
142+
143+
### Architecture
144+
145+
```
146+
[client app]
147+
POST /<name>/v1/chat/completions
148+
body has tool definitions (or capabilities discovered from connected client manifest)
149+
150+
151+
[hub] forwards chat-request to publisher with tools array
152+
153+
154+
[AI publisher] returns chat-response with tool_calls (instead of plain content)
155+
156+
157+
[hub] sees tool_calls in publisher's response
158+
- if tool name matches a connected client's capability:
159+
→ automatic: invoke-request to that client
160+
→ wait for invoke-result
161+
→ append role=tool message
162+
→ forward back to publisher (recursive chat)
163+
- else: return tool_calls to client app as normal OpenAI shape
164+
165+
166+
[publisher] continues with tool result in conversation, returns final assistant message
167+
```
168+
169+
### Wire format
170+
171+
Capability declarations from connect()-side already include the JSON schema. Hub turns each capability into a `tool` entry:
172+
173+
```json
174+
{
175+
"type": "function",
176+
"function": {
177+
"name": "send_whatsapp",
178+
"description": "Send WhatsApp via Loki",
179+
"parameters": {...JSON schema...}
180+
}
181+
}
182+
```
183+
184+
These get injected into the chat-request envelope so the AI sees them. AI replies with `tool_calls`; hub maps each to the right connection_id and invokes.
185+
186+
### Operator opt-out
187+
188+
By default: hub auto-resolves tool_calls that match capabilities (saves operator a round-trip). Opt-out: client request adds `X-Zhub-Tool-Resolve: client` header → hub returns tool_calls verbatim and the operator handles them.
189+
190+
### Out of scope
191+
192+
- Parallel tool calls (OpenAI now supports multiple tool_calls per response — phase 1.8b)
193+
- Tool streaming (chunks of tool_calls arriving via SSE) — phase 1.8c
194+
- Tool retries on transient invoke errors — phase 1.8d
195+
196+
### TDD
197+
198+
`tests/test_tool_calls.py`:
199+
1. Publish AI whose chat handler returns a chat-response with `tool_calls: [{name: "send_whatsapp", args: {to: "Ammi", message: "ok"}}]` on first call, then a normal text response on second call (mock the LLM with a deterministic two-step handler).
200+
2. Connect a client exposing `send_whatsapp` capability (returns `{ok: true, delivered: true}`).
201+
3. POST to `/<name>/v1/chat/completions`.
202+
4. Expect: hub auto-resolves the tool_call, appends role=tool message with the result, sends back to publisher, returns publisher's final text response — **client app sees normal chat completion with no tool_calls in the visible response**, but a `tool_results` log in the response usage block.
203+
5. Same flow with `X-Zhub-Tool-Resolve: client` → expect tool_calls returned verbatim, no automatic invoke.
204+
205+
## Self-review
206+
207+
**Placeholder scan:** none. Each phase has concrete deliverables, TDD plan, and out-of-scope cuts.
208+
209+
**Internal consistency:**
210+
- Phase 1.7 rate-limit format matches manifest.rate_limit type already declared (string like `"60/min"`).
211+
- Phase 1.2 JS client follows the same Envelope schema as Python — wire-compatible.
212+
- Phase 1.1 routing reuses Phase 1.0b's PeerRegistry for peer URL discovery.
213+
- Phase 1.8 tool calls layer on top of Phase 1.0/1.5 invoke-request path — same connection_id routing, same wire envelope.
214+
215+
**Scope check:** four phases is a lot. Each has an honest size. 1.1 (cross-hub) is the biggest — if it bumps past time budget, ship 1.1a (non-streaming proxy) and defer 1.1b (streaming proxy) to follow-up. The other three should land cleanly.
216+
217+
**Ambiguity check:**
218+
- 1.7: "60/min" parsing — clarified to support `s/min/hour/day` units, fallback `"60/min"`.
219+
- 1.2: ESM-only emit — explicit; CJS path deferred.
220+
- 1.1: loop prevention via `X-Zhub-Forwarded-By` header — explicit.
221+
- 1.8: opt-out via `X-Zhub-Tool-Resolve: client` header — explicit.
222+
223+
## Transition
224+
225+
Skipping the user-review gate per autonomous override. Going directly to TDD execution per phase.

0 commit comments

Comments
 (0)