|
| 1 | +# zhub Phase 0.7 — 1.0 design |
| 2 | + |
| 3 | +> Autonomous batch authorized by operator on 2026-05-09 ("use super power plugin plan and do everything autonomously, yes to all"). User-approval gates inside the brainstorming skill are skipped per that instruction; the rest of the discipline (explore context, propose approaches, written design, self-review, plan, TDD, verification) is preserved. |
| 4 | +
|
| 5 | +## 1. Where zhub stands today (2026-05-09) |
| 6 | + |
| 7 | +Phase 0+1 through 0.6 shipped. Latest commit `d8affa8`. CI green on Python 3.10/3.11/3.12. 17/17 pytest. Hand-tested live by Claude on a local hub: bidirectional invoke, SSE streaming, persistence + re-registration with stored key, registry HTML page, all working. Two real bugs fixed during that hand-test (invoke double-wrap; generator-vs-non-streaming timeout). Two regression tests committed. |
| 8 | + |
| 9 | +Roadmap remaining (per `/root/zhub/CLAUDE.md` §5): |
| 10 | + |
| 11 | +> **Phase 1.0** — Signed manifests, hub federation. |
| 12 | +
|
| 13 | +Production-readiness gaps not on the roadmap but worth closing in this batch: |
| 14 | + |
| 15 | +- **Kotlin module isn't tested in CI.** Today only Python tests run. The kotlin/ subproject has unit tests for the protocol round-trip (`kotlin/src/test/kotlin/com/zawwar/zhub/ProtocolTest.kt`) — Loki will rely on this code, so silent regressions there ship to phones. |
| 16 | +- **Cloudflare Tunnel path is code-correct but un-exercised live** (no `cloudflared` binary in proot). One unverified moving piece between Father and a real public URL. |
| 17 | +- **Multi-AI orchestration (council pattern) has no example.** zhub's bidirectional design literally enables it but nothing in `examples/` demonstrates more than one AI at a time. |
| 18 | + |
| 19 | +## 2. Scope of this batch |
| 20 | + |
| 21 | +Four sub-projects, each with its own deliverable. Sequential because each builds on what's there. |
| 22 | + |
| 23 | +### Phase 0.7 — Kotlin gradle CI job |
| 24 | + |
| 25 | +**Goal:** parallel CI job that compiles `kotlin/` and runs its unit tests on every push to `main`. If the protocol envelope schema drifts (Python vs Kotlin) or a Kotlin compile error sneaks in, CI fails. |
| 26 | + |
| 27 | +**Approach:** add a `kotlin-test` job to `.github/workflows/ci.yml` using `actions/setup-java@v4` (Temurin 17) and `gradle/actions/setup-gradle@v3`. Run `./gradlew test`. Cache dependencies. Job runs in parallel with the Python matrix; fails the workflow if either side breaks. |
| 28 | + |
| 29 | +**Stretch:** also run `./gradlew build` to catch packaging issues. Likely free given setup-gradle's caching. |
| 30 | + |
| 31 | +**Out of scope:** publishing the Kotlin artifact to Maven Central. That's Phase 0.10+ and requires repository credentials Father will set up when there's a downstream consumer. |
| 32 | + |
| 33 | +### Phase 0.8 — Cloudflare Tunnel live verification |
| 34 | + |
| 35 | +**Goal:** prove `zhub-server --public-tunnel` actually produces a working public URL end-to-end, on the same machine where Father will run it (Linux/proot). |
| 36 | + |
| 37 | +**Approach:** install `cloudflared` in this proot via the official `.deb` (Cloudflare's recommended distribution method). Start the hub with `--public-tunnel`. Verify: |
| 38 | + |
| 39 | +1. `cloudflared` spawns and reaches a "Your tunnel is at https://*.trycloudflare.com" line within 30s. |
| 40 | +2. `tunnel.start()` parses that URL correctly. |
| 41 | +3. `curl <public-url>/healthz` from the same proot returns 200 (proves egress + ingress through CF works). |
| 42 | +4. Publish a stub AI through the tunnel; hit it with curl; verify the round-trip. |
| 43 | +5. Clean shutdown — `cloudflared` exits when hub exits. |
| 44 | + |
| 45 | +**Document install + caveats** in `docs/CLOUDFLARED.md` so Father has the same path. |
| 46 | + |
| 47 | +**Out of scope:** named tunnels with stable subdomains. Ephemeral trycloudflare.com is fine for development; named tunnels are operator-side configuration, not a code concern. |
| 48 | + |
| 49 | +### Phase 0.9 — Multi-AI council example |
| 50 | + |
| 51 | +**Goal:** demonstrate bidirectional power that no other tool in this space has — three AIs published, one orchestrates the other two through the hub. |
| 52 | + |
| 53 | +**Approach:** new `examples/council_demo.py`: |
| 54 | + |
| 55 | +- Publishes 3 AIs: |
| 56 | + - `claude-stub` — pretends to be Claude, returns thoughtful-toned replies |
| 57 | + - `gpt-stub` — pretends to be GPT, returns concise-toned replies |
| 58 | + - `gemini-stub` — pretends to be Gemini, returns analytical-toned replies |
| 59 | +- A 4th publisher, `coordinator`, is the council itself. Its chat handler: |
| 60 | + 1. Receives operator's question |
| 61 | + 2. Connects (via `zhub.connect`) to each of the 3 AIs and asks them in parallel |
| 62 | + 3. Synthesizes a single response that quotes each |
| 63 | + 4. Returns the synthesis as its own chat-response |
| 64 | + |
| 65 | +**Why this matters:** ZAI's existing `zai-council` plugin uses bespoke OpenRouter routing. With zhub, this pattern works for any published AIs uniformly. Demonstrates the substrate's value beyond "expose one AI." |
| 66 | + |
| 67 | +**Out of scope:** real LLM calls in the example (operator-side keys, costs). Stubs make the orchestration shape visible without spending on tokens. |
| 68 | + |
| 69 | +### Phase 1.0 — Signed manifests + lightweight federation |
| 70 | + |
| 71 | +This is the biggest piece. Honest scope first. |
| 72 | + |
| 73 | +**Goal:** publishers cryptographically sign their manifests; consumers can verify identity without trusting the hub. Hubs can advertise registries to each other so a global directory is possible. |
| 74 | + |
| 75 | +**Two-part decomposition:** |
| 76 | + |
| 77 | +**1.0a — Signed manifests (must-have):** |
| 78 | + |
| 79 | +- Each publisher generates an ed25519 keypair on first publish (or supplies an existing private key) |
| 80 | +- The manifest published to the hub includes a `signature` field — Ed25519 signature over `name + canonical_json(manifest_minus_signature)` |
| 81 | +- The publisher also publishes its public key as part of the manifest |
| 82 | +- A separate verification path: any consumer can fetch `/<name>/manifest.json`, verify the signature, and confirm the manifest hasn't been tampered with by the hub |
| 83 | +- Store the key fingerprint in `pyproject` and pin it across re-registrations (so re-using `api_key=...` validates that the same private key is signing — prevents key takeover via stolen api_key alone) |
| 84 | + |
| 85 | +**1.0b — Federation (stretch):** |
| 86 | + |
| 87 | +- A hub can declare peer hubs in config: `peers: ["https://hub-b.example.com", ...]` |
| 88 | +- Federated `/registry` returns local listings + a `peers` block linking to peer hubs' `/registry` endpoints |
| 89 | +- A new `/registry/global` endpoint walks one hop of peers and returns the union (with origin annotations) |
| 90 | +- Federation is **read-only / discovery-only** in 1.0b — no cross-hub call routing, no shared registry state. That's 1.1+. |
| 91 | + |
| 92 | +**Approach:** |
| 93 | + |
| 94 | +For 1.0a: |
| 95 | +- Add `cryptography` (already common dep) to optional `[crypto]` extras |
| 96 | +- New `zhub/signing.py`: `generate_keypair()`, `sign_manifest(manifest, private_key)`, `verify_manifest(signed_manifest)` — small surface |
| 97 | +- Modify `register_publisher` envelope to carry `signed_manifest` instead of bare `manifest`. Hub validates signature before accepting registration. Hub stores the public key alongside the manifest (already in storage; just an extra field). |
| 98 | +- `manifest.json` HTTP endpoint returns the signed form; consumers can verify |
| 99 | +- New test: `tests/test_signing.py` covering happy path + tamper detection |
| 100 | + |
| 101 | +For 1.0b: |
| 102 | +- New `zhub/federation.py`: `PeerRegistry` class, periodic refresh of peer registries |
| 103 | +- Hub config gains `peers: list[str]` (CLI flag or env var: `ZHUB_PEERS=url1,url2`) |
| 104 | +- `/registry/global` aggregator |
| 105 | +- Test: spin up two hubs, register one publisher on each, verify global view |
| 106 | + |
| 107 | +**Honest expectation:** 1.0a will land. 1.0b may land if context permits; if not, it gets a written plan and is the obvious next session. Federation is a bigger design surface (TLS verification, peer trust, refresh cadence) than signing. |
| 108 | + |
| 109 | +**Out of scope:** |
| 110 | +- Cross-hub call routing (call AI on hub-b through hub-a) — Phase 1.1. |
| 111 | +- Private peering (signed peer relationships) — Phase 1.2. |
| 112 | +- Anchored identities (DNS-bound or DID-bound publisher names) — Phase 2.0. |
| 113 | + |
| 114 | +## 3. Architecture impact summary |
| 115 | + |
| 116 | +Most of this batch is additive — new files, new tests, new examples. Three things change existing code paths: |
| 117 | + |
| 118 | +1. **CI workflow** gains a parallel job (Phase 0.7). |
| 119 | +2. **`server.py` register-publisher path** validates signature when present, falls back to legacy unsigned for backwards compatibility (Phase 1.0a). Existing tests keep passing because they don't sign. |
| 120 | +3. **`client.py` `publish()`** gains a `private_key` parameter (None means unsigned, opt-in). When present, signs the manifest before send. |
| 121 | + |
| 122 | +No breaking changes to v0 callers. v0.1 manifests remain valid alongside v0.2 signed manifests. |
| 123 | + |
| 124 | +## 4. Test plan (TDD throughout) |
| 125 | + |
| 126 | +Per the test-driven-development skill — every change writes a failing test first, then makes it pass. |
| 127 | + |
| 128 | +**0.7:** the CI job itself is the test. Verify by pushing and watching the new job appear, pass, and gate the workflow. Optional: a `kotlin/src/test/kotlin/.../ManifestSerializationTest.kt` that round-trips a Manifest through Python's wire format (regression for Python ↔ Kotlin schema drift). |
| 129 | + |
| 130 | +**0.8:** a real curl-against-tunnel test, manually verified. No CI test (CF tunnels aren't a CI runner concern). |
| 131 | + |
| 132 | +**0.9:** `tests/test_council.py` — the council coordinator's chat handler is exercised end-to-end. Stub AIs respond; coordinator synthesizes; assertion: synthesized output mentions each stub. |
| 133 | + |
| 134 | +**1.0a:** `tests/test_signing.py`: |
| 135 | +- happy-path sign + verify round-trip |
| 136 | +- tamper detection (mutate manifest after sign → verify fails) |
| 137 | +- public-key extraction matches generated key |
| 138 | +- legacy unsigned manifest still accepted (backwards compatibility) |
| 139 | +- re-registration with `api_key=` and a different private key is rejected (key pinning) |
| 140 | + |
| 141 | +**1.0b:** `tests/test_federation.py`: |
| 142 | +- two hubs in-process, each with one publisher, peering enabled |
| 143 | +- `GET /registry/global` on hub A includes hub B's listing |
| 144 | +- non-peered hub returns only its own |
| 145 | +- offline peer is gracefully omitted |
| 146 | + |
| 147 | +## 5. Verification before completion |
| 148 | + |
| 149 | +For every phase: actual tests pass, actual CI green, hand-test the changed code path before claiming done. The verification-before-completion skill says: evidence before assertions always. The two bugs caught in last session's hand-test prove this rule's value. |
| 150 | + |
| 151 | +For Phase 0.8 specifically — must include a real `curl https://*.trycloudflare.com/...` round trip succeeding. Just installing `cloudflared` and reading the URL doesn't count. |
| 152 | + |
| 153 | +## 6. Self-review (fresh eyes) |
| 154 | + |
| 155 | +**Placeholder scan:** none. All four phases have concrete deliverables. |
| 156 | + |
| 157 | +**Internal consistency:** Phase 1.0a backwards-compat claim cross-checks: "v0 unsigned manifests remain valid" + "test: legacy unsigned manifest still accepted." Consistent. |
| 158 | + |
| 159 | +**Scope check:** four sub-projects in one design doc is acceptable here because each is small (0.7, 0.8, 0.9 are < 1 day each) and 1.0 is decomposed into 1.0a + 1.0b with a stop-line. If 1.0a alone is going to take a full day, that's fine — it ships before 1.0b is touched. |
| 160 | + |
| 161 | +**Ambiguity check:** |
| 162 | + |
| 163 | +- "Hand-test" in §5 was vague — added: must include a real curl round-trip for 0.8. |
| 164 | +- "Federation discovery" in §2 was vague — clarified: read-only, one-hop, no shared state, no cross-hub call routing. |
| 165 | +- "Key pinning" in §4 was unstated — added explicit test: re-register with same api_key but different private key → reject. |
| 166 | + |
| 167 | +Fixed inline; nothing else surfaces. |
| 168 | + |
| 169 | +## 7. Transition |
| 170 | + |
| 171 | +This design doc proceeds directly to writing-plans for an implementation plan, then executing-plans / subagent-driven-development for the actual work. Per "yes to all" the user-review gate (brainstorming step 8) is skipped. |
0 commit comments