diff --git a/docs/formal/nip-broker/NOTE.md b/docs/formal/nip-broker/NOTE.md new file mode 100644 index 00000000000..78593994791 --- /dev/null +++ b/docs/formal/nip-broker/NOTE.md @@ -0,0 +1,233 @@ +# NIP-BA assurance and compatibility note + +Status: draft, awaiting independent redteam. Normative text is +[`../../nips/NIP-BA.md`](../../nips/NIP-BA.md). This note is evidence, not extra +normative protocol hidden to improve the word count. + +## Sources and scope + +This is a separate proposal on main, not a change to runtime behavior or a +claim that the existing SDK/host conforms: + +- Original nine-operation prose: block/buzz#6790, + `804ce76c167d0f435cbff65e7e3f4aad720915f6`, `docs/agent-broker.md`. +- Fifteen-operation Rust contract: block/buzz#6922, + `115e7975a11d7d4d95043cb847abd13c1a32f370`, + `crates/buzz-sdk/src/broker/{actions,mod.rs,wire.rs,correlate.rs}`. +- Client integration: block/buzz#6967, + `3d2ea5b89adccf980879b9841e9a6b517b588b57`. +- Buzz-local NIPs from this branch's main base + `7a9a5233d9d755e715be0c585cf7850e935d28cf`. + +The remote-agent vision presently hands a signing key to the substrate. This +proposes optional different custody, not a replacement deployment management +plane. All durable user-visible state remains relay-scoped; the host's retry +journal is execution safety state, not an alternative source of channel truth. + +## Deliberate clarifications beyond editorial compression + +The wire names, argument/outcome members, 15 action versions, and numerical +limits follow #6922. The following need host/client agreement before adoption; +they are **not assertions of already-implemented behavior**: + +1. Retry key includes community as well as principal. Atomic durable admission, + in-flight joining, crash fencing, evidence-only reconciliation, and permanent + retry tombstones close unspecified safety gaps. Availability/storage cost is + intentional: finite retention without a new wire epoch cannot safely permit + an old ID to execute again. +2. Authenticate/authorize release on every retry, including revoked sessions. + A retry refusal is attempt-local and must not clear earlier uncertainty. + This corrects the original blanket “failed means never happened” wording. +3. Specify memory core mapping to `profile`, absence/tombstones, exact compact + size encoding, and observer partial-acceptance non-identifiability. No prefix + guarantee was present in #6922; none is invented here. +4. Cursor filter binding/invalid-cursor refusal, thread ancestry validation, + exact unsigned integer syntax, and current membership are explicit host + obligations. They are not all enforced by the baseline SDK deserializer. +5. Clarify publication acknowledgement, partial lifecycle/watchdog effects, + sorted changed-field subset, ambiguous-name refusal, and creation not implying + a booted runtime. +6. Remove false claims that NIP-46 cannot inspect intent or that closed string + schemas prevent all secret transmission. Storage reads intentionally return + decrypted application content. + +Remaining flexibility is explicit: deployment resource limits, initial read +window/order/cursor lifetime, runtime/provider/model defaults, provisioning and +deletion machinery, runtime telemetry body, and ownership-depth policy. This +spec does not standardize runtime-to-owner telemetry interpretation. A claim of +full cross-runtime interoperability would need that companion profile. + +## Finite transition system + +Run with Python 3.10+ (standard library only): + +```sh +python3 docs/formal/nip-broker/model.py +``` + +`model.py` defines an explicit transition relation and breadth-first explores +all reachable states until a fixed point, not random traces or a depth cutoff. +A state is `(records, allowed, uncertain)`. Each record is +`(digest, phase, dispatches, effects, final)`, with final as a ghost observation +that survives result erasure. One shared request ID, two unequal body digests, +three contexts `(community,principal) = (0,0),(0,1),(1,0)` exercise collisions. +One ID is a symmetry abstraction, not proof for arbitrarily many IDs. Work on +different IDs can interact through real operation state, which is outside this +model. `effects` is a Boolean abstraction of “any effect took hold,” not the +number of relay events. + +Transitions: admission, digest conflict, wait timeout, dispatch, effect, +completion, crash, reconciliation, result erasure, retry, and revocation. +Dispatch is atomically admitted once. Revocation prevents future dispatch and +stored-result release; it cannot recall already dispatched work. Reconciliation +may reveal real effects but never runs the operation again. Crashed work can +remain unknown forever. Lost responses are modeled by allowing completion to +occur without client observation and later retry; HTTP packet order is not +modeled. Reconciliation evidence is assumed truthful. + +### Checked invariants and mutation witnesses + +| Property | Normative seam | Deliberately broken transition | +|---|---|---| +| At most one dispatch per context/ID | Execution 2–5, retention | Concurrent redispatch, restart executor, evict protection | +| Known failure has no effects | Result and Execution 6 | Crash misreported as failure | +| Success has effect evidence | Published / completion | Baseline assertion (no separate mutant yet) | +| No cross-context stored-result release | Session / K definition | Drop community or principal | +| No release after revocation | Session / retry authorization | Replay bypasses revocation | +| Different bytes do not replay | Execution 1/3 | Ignore digest | +| Refusal retains previous uncertainty | Results, bold retry caveat | Clear uncertainty on refusal | + +Nine deliberate mutations must each produce a counterexample for their expected +property; the process exits nonzero if the baseline violates an invariant or a +mutation survives. This tests model guards, **not production guards**. Claims +about production regressions require binding the actual host implementation, +which this PR does not supply. + +### What this does not prove + +Not a proof of JSON-parser correctness, cryptography, credential entropy, TLS, +Nostr event construction, thread ancestry resolution, pagination completeness, +NIP-AE convergence, lifecycle atomicity, or actual host behavior. Authorization +is abstracted to a Boolean; no policy engine is modeled. No fairness or liveness +claim: permanent partitions and unavailable reconciliation can remain unknown. +The at-most-once dispatch invariant does not establish exactly-once remote +effects. Multi-step operation adapters must separately establish their effect +and failure semantics. No malicious-host security theorem is possible when the +host holds the identity key. + +## Upstream comparison protocol + +Fixed before drafting: upstream NIPs 01, 05, 07, 09, 10, 29, 42, 44, 46, 98 at +`488b787848fcf1c6c3498c253264b8121b1a9692`. This is a purposive dependency/API/ +security sample, not a random sample supporting a 90th-percentile assertion. + +Dimensions: normative precision; independent implementability; failure/retry +clarity; security boundaries; reproducible verification; economy of expression. +Each gets 0 absent, 1 major gaps, 2 usable with questions, 3 explicit adequate, +4 unusually strong. N/A must not be scored zero. Independent review must cite +evidence and may reject the rubric. No averages may hide a weaker dimension. +Strictly better on every dimension requires every score to exceed the +comparator's; Pareto superiority (no weaker, at least one stronger) is a +separate, weaker criterion. Adoption, deployed interoperability, and ecosystem +maturity are additional dimensions that an unimplemented draft cannot beat by +editing prose. No “superior to 9/10” conclusion is asserted here. + + +## Executable wire examples + +```sh +python3 -m unittest discover -s docs/formal/nip-broker -v +``` + +`vectors.json` contains a request and corresponding result for each of the 15 +actions. `wire.py` is a dependency-free, independently written **partial wire +oracle**, not a production implementation or a full conformance certificate. +It checks closed shapes, scalar normalization, integer widths, UTF-8, identity +syntax, size limits, status/code pairs and selected response correlations. +`test_wire.py` systematically inserts unknown/null/duplicate members into the +example objects and exercises boundaries and byte-distinct equivalent JSON. +Example event IDs and d-tags are illustrative, not cryptographic vectors; the +checker does not verify signatures, +read-filter membership, ancestry, address derivation, or operation execution. + +A deliberately bypassed closed-object guard admits a forbidden `scope` member; +the normal guard rejects it. An encoding regression was reproduced before the +fix: Python's JSON decoder accepted UTF-16 input. The oracle now decodes bytes +as UTF-8 explicitly. Neither experiment establishes a production SDK regression. + +## Comparison findings (author assessment; independent review pending) + +The fixed sample is not exchangeable: a browser capability, an event deletion +request and a distributed action service solve different problems. Absence of +an execution journal in a signature or serialization standard is **not a defect**. +Consequently, numerical totals would reward our chosen problem and hide tradeoffs. +The six dimensions above are inspection questions, not measured universal ranks. +These are concrete strengths to preserve or learn from, and limits to our claim: + +| NIP at the pinned revision | Evidence in that document | NIP-BA comparison boundary | +|---|---|---| +| [01][n01] | Events/signatures specifies serialization; relay flow defines OK, CLOSED, EOSE and tie ordering | We add operation uncertainty, but depend on its event machinery; not strictly more precise in every dimension | +| [05][n05] | Security Constraints forbids redirects; Notes distinguishes identification from verification and preserves pubkey identity across remapping | Comparable explicit trust boundary; much narrower and economical protocol | +| [07][n07] | Two required browser methods, optional encryption methods, extension timing and implementation link | Our failure contract is fuller; its tiny API is easier to implement and explain for its task | +| [09][n09] | Client Usage requires author matching; warns deletion cannot be guaranteed; deleting a deletion has no effect | Both explicitly bound promises; no reason to demand an action journal from a deletion-request event | +| [10][n10] | Marked e tags distinguish parent/root and document legacy ambiguity | We reuse this idea rather than surpass it; its kind-1 scope is not our kind-9 profile | +| [29][n29] | Relay-scoped group identity, forks/migrations, independent subgroup membership, reconstruction events | Our retry namespace is explicit, but UUID-only actions and out-of-band provisioning are less general | +| [42][n42] | Connection-scoped challenges, request retry examples, auth-required/restricted distinction | We add revoked-result rules; our bearer provisioning is less specified than its challenge exchange | +| [44][n44] | Limitations, exact algorithm/pseudocode, external audit, published positive/negative vectors | A finite retry model is not a cryptographic audit; no verification superiority claimed | +| [46][n46] | Two connection flows, permission requests, secret validation, logout limitations, auth challenge examples | We add reads/execution safety; it specifies discovery and connection establishment that we leave out of band | +| [98][n98] | URL/method/time checks, optional body binding, wire example and reference implementation | We require retry-byte identity but solve a different authentication problem and are less compact | + +NIP-BA's current strengths are explicit attempt-local failure semantics, durable +retry protection, a small executable safety model with negative witnesses, and +all-action wire examples. Its weaknesses remain companion runtime telemetry +semantics, out-of-band provisioning/policy, unbounded lifetime journal growth +controlled only by admission quotas, no production host, and no independent +client/host interoperability run. Economy must be judged relative to scope, +not by rewarding the document with the fewest absolute words. + +**The requested “superior in all dimensions to at least 9/10” target is not +established.** No editing-only stopping rule can supply deployment maturity or +an independent implementation. This proposal should be judged on an auditable +contract and resolved review findings, not a fabricated league table. + +[n01]: https://github.com/nostr-protocol/nips/blob/488b787848fcf1c6c3498c253264b8121b1a9692/01.md + +[n05]: https://github.com/nostr-protocol/nips/blob/488b787848fcf1c6c3498c253264b8121b1a9692/05.md + +[n07]: https://github.com/nostr-protocol/nips/blob/488b787848fcf1c6c3498c253264b8121b1a9692/07.md + +[n09]: https://github.com/nostr-protocol/nips/blob/488b787848fcf1c6c3498c253264b8121b1a9692/09.md + +[n10]: https://github.com/nostr-protocol/nips/blob/488b787848fcf1c6c3498c253264b8121b1a9692/10.md + +[n29]: https://github.com/nostr-protocol/nips/blob/488b787848fcf1c6c3498c253264b8121b1a9692/29.md + +[n42]: https://github.com/nostr-protocol/nips/blob/488b787848fcf1c6c3498c253264b8121b1a9692/42.md + +[n44]: https://github.com/nostr-protocol/nips/blob/488b787848fcf1c6c3498c253264b8121b1a9692/44.md + +[n46]: https://github.com/nostr-protocol/nips/blob/488b787848fcf1c6c3498c253264b8121b1a9692/46.md + +[n98]: https://github.com/nostr-protocol/nips/blob/488b787848fcf1c6c3498c253264b8121b1a9692/98.md + + +## Reproduction result before independent review (2026-09-03) + +Python 3.14, three-context baseline: **1,481,544 reachable states and 16,102,044 +transitions**, exhausted with no invariant violation. All nine mutations yielded +a counterexample for their designated property. Wire suite: **9 test groups +passed**, including positive request/result vectors for every action. +Model source SHA-256: `0f87f39917e044c4767be3d9a75fe1aed65325be9316dbebdab7c6847c7447a5`. +Rerun the commands above; model stdout contains the complete mutation traces. + +`just ci` was attempted on base `7a9a5233d9d755e715be0c585cf7850e935d28cf` +with only these documentation/model additions. The initial attempt timed out; +a subsequent invocation selected an old PATH compiler and failed the MSRV check. +With the installed Rust 1.95.0 toolchain selected explicitly, repository checks +and the full 463-test CLI package passed, but `test-unit` stopped in buzz-acp at +`acp::tests::keepalive_resets_idle_past_deadline` (82 passed, one failed before +fail-fast). Later CI stages did not run. **Full repository CI is not green**; +this note does not diagnose that failure as a flake or as caused by this change. + +Independent redteam requested from Eva; findings and disposition remain pending. +This is a reviewable draft, not approval of a production broker. diff --git a/docs/formal/nip-broker/model.py b/docs/formal/nip-broker/model.py new file mode 100644 index 00000000000..bc25ce23d82 --- /dev/null +++ b/docs/formal/nip-broker/model.py @@ -0,0 +1,200 @@ +#!/usr/bin/env python3 +"""Finite-state NIP-BA retry model; exhaustive reachable-state search. + +No runtime dependency. This is a specification model, not a broker emulator. +See NOTE.md for the abstraction, properties, and explicit non-claims. +""" +from collections import deque +from dataclasses import dataclass, replace +import json + + +@dataclass(frozen=True) +class Record: + digest: int = -1 + phase: str = "absent" + dispatches: int = 0 + effects: int = 0 + # Ghost evidence: final verdict before optional result erasure. + final: str = "" + + +@dataclass(frozen=True) +class State: + records: tuple + # Current auth/release authorization for each context; revocation is monotone. + allowed: tuple + # Whether a caller still lacks conclusive evidence of its original attempt. + uncertain: tuple + + +CONTEXTS = ((0, 0), (0, 1), (1, 0)) # community, principal + + +def slot(context, mutant): + if mutant == "omit_community": + return CONTEXTS.index((0, CONTEXTS[context][1])) + if mutant == "omit_principal": + return CONTEXTS.index((CONTEXTS[context][0], 0)) + return context + + +def successors(state, mutant=""): + """(label, next state, observation) transitions, all choices enumerated. + + Observations: (caller context, owner slot, verdict, releases stored data, + newly introduced effects). Attempt IDs are fixed to one shared ID; digests + 0 and 1 represent unequal complete request bytes. + """ + def changed(i, record, *, uncertain=None): + records = list(state.records) + records[i] = record + return replace(state, records=tuple(records), uncertain=( + state.uncertain if uncertain is None else uncertain)) + + for c in range(len(CONTEXTS)): + if state.allowed[c]: + allowed = list(state.allowed) + allowed[c] = False + yield f"revoke({c})", replace(state, allowed=tuple(allowed)), None + i = slot(c, mutant) + r = state.records[i] + for digest in (0, 1): + prefix = f"submit(context={c},digest={digest})" + if not state.allowed[c] and mutant != "replay_after_revoke": + uncertain = state.uncertain + if mutant == "refusal_clears_uncertainty": + uncertain = tuple(False if j == c else u for j, u in enumerate(uncertain)) + yield prefix + ":unauthorized", replace(state, uncertain=uncertain), (c, i, "refusal", False, 0) + elif r.phase == "absent": + if state.allowed[c]: + yield prefix + ":claim", changed(i, Record(digest, "ready")), None + elif r.digest != digest: + if mutant == "ignore_digest": + yield prefix + ":wrong-replay", state, (c, i, "wrong_digest", True, 0) + else: + yield prefix + ":conflict", state, (c, i, "refusal", False, 0) + elif r.phase in ("ready", "running"): + uncertain = tuple(True if j == c else u for j, u in enumerate(state.uncertain)) + if mutant == "duplicate_dispatch" and r.phase == "running": + yield prefix + ":redispatch", changed(i, replace(r, dispatches=r.dispatches + 1)), None + else: + yield prefix + ":wait-timeout", replace(state, uncertain=uncertain), (c, i, "unknown", False, 0) + else: + verdict = r.phase if r.phase in ("succeeded", "failed") else "unknown" + uncertain = tuple((verdict == "unknown") if j == c else u for j, u in enumerate(state.uncertain)) + yield prefix + ":replay", replace(state, uncertain=uncertain), (c, i, verdict, True, 0) + + for i, r in enumerate(state.records): + if r.phase == "ready": + if state.allowed[i]: + yield f"dispatch({i})", changed(i, replace(r, phase="running", dispatches=r.dispatches + 1)), None + else: + yield f"deny-before-dispatch({i})", changed(i, replace(r, phase="failed", final="failed")), None + if r.phase == "running": + if r.effects == 0: + # Revocation after authorized dispatch does not recall work already begun. + yield f"effect({i})", changed(i, replace(r, effects=1)), None + yield f"finish-no-effects({i})", changed(i, replace(r, phase="failed", final="failed")), None + else: + yield f"finish-success({i})", changed(i, replace(r, phase="succeeded", final="succeeded")), None + yield f"crash({i})", changed(i, replace(r, phase="crashed")), None + if r.phase == "crashed": + if mutant == "restart_executor": + yield f"unsafe-restart({i})", changed(i, replace(r, phase="ready")), None + if mutant == "false_failure": + yield f"unsafe-failure({i})", changed(i, replace(r, phase="failed", final="failed")), None + yield f"persist-unknown({i})", changed(i, replace(r, phase="unknown")), None + if r.phase in ("crashed", "unknown"): + # Abstract external reconciliation evidence: it can reveal reality, + # not create effects. In a real host it may never become available. + verdict = "succeeded" if r.effects else "failed" + yield f"reconcile({i},{verdict})", changed(i, replace(r, phase=verdict, final=verdict)), None + if r.phase in ("succeeded", "failed", "unknown"): + yield f"erase-result({i})", changed(i, replace(r, phase="tombstone")), None + if r.phase == "tombstone" and mutant == "evict_protection": + # Preserve ghost counters so clearing the journal cannot hide a repeat. + yield f"unsafe-evict({i})", changed(i, replace(r, phase="ready")), None + + +def violation(before, after, observation): + for r in after.records: + if r.dispatches > 1: + return "at_most_once_dispatch" + if r.final == "failed" and r.effects: + return "no_false_failure" + if r.final == "succeeded" and not r.effects: + return "success_has_effect_evidence" + if observation: + c, owner, verdict, released, effects = observation + if released and owner != c: + return "context_isolation" + if released and not before.allowed[c]: + return "no_release_after_revocation" + if verdict == "wrong_digest": + return "byte_identity" + if effects: + return "replay_has_no_effects" + if verdict == "refusal" and before.uncertain[c] and not after.uncertain[c]: + return "refusal_preserves_prior_uncertainty" + return None + + +def check(mutant="", contexts=1): + # Smaller runs isolate each mutation; the baseline is also run with all + # contexts. Unused contexts start revoked, avoiding irrelevant interleavings. + initial = State(tuple(Record() for _ in CONTEXTS), + tuple(i < contexts for i in range(len(CONTEXTS))), + (False,) * len(CONTEXTS)) + queue = deque([initial]) + parents = {initial: None} + edges = 0 + while queue: + state = queue.popleft() + for label, nxt, observation in successors(state, mutant): + edges += 1 + broken = violation(state, nxt, observation) + if broken: + trace = [label] + while parents[state]: + state, step = parents[state] + trace.append(step) + return {"mutant": mutant or "baseline", "property": broken, + "states": len(parents), "edges": edges, + "counterexample": list(reversed(trace))} + if nxt not in parents: + parents[nxt] = (state, label) + queue.append(nxt) + return {"mutant": mutant or "baseline", "states": len(parents), + "edges": edges, "counterexample": None} + + +def main(): + import argparse + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument("--contexts", type=int, choices=(1, 2, 3), default=3) + args = parser.parse_args() + baseline = check(contexts=args.contexts) + print(json.dumps(baseline, sort_keys=True)) + if baseline["counterexample"]: + raise SystemExit("baseline invariant failure") + mutations = { + "omit_community": (3, "context_isolation"), + "omit_principal": (2, "context_isolation"), + "replay_after_revoke": (1, "no_release_after_revocation"), + "ignore_digest": (1, "byte_identity"), + "duplicate_dispatch": (1, "at_most_once_dispatch"), + "restart_executor": (1, "at_most_once_dispatch"), + "false_failure": (1, "no_false_failure"), + "evict_protection": (1, "at_most_once_dispatch"), + "refusal_clears_uncertainty": (1, "refusal_preserves_prior_uncertainty"), + } + for mutant, (contexts, expected) in mutations.items(): + result = check(mutant, contexts) + print(json.dumps(result, sort_keys=True)) + if not result["counterexample"] or result["property"] != expected: + raise SystemExit(f"mutation did not falsify expected property: {mutant}") + + +if __name__ == "__main__": + main() diff --git a/docs/formal/nip-broker/test_wire.py b/docs/formal/nip-broker/test_wire.py new file mode 100644 index 00000000000..822998516d1 --- /dev/null +++ b/docs/formal/nip-broker/test_wire.py @@ -0,0 +1,172 @@ +"""Run: python3 -m unittest discover -s docs/formal/nip-broker -v""" +import copy +import json +from pathlib import Path +import unittest +from unittest.mock import patch + +import wire + +PUB = "79be667ef9dcbbac55a06295ce870b07029bfcdb2dce28d959f2815b16f81798" +CHANNEL = "5df7dfa8-e919-43df-8efd-f1dcb8af7071" +EVENT = "a" * 64 + + +def envelope(action, args): + return dict(type="broker_request", protocolVersion=1, requestId="example-1", + actionVersion=1, action=action, args=args) + + +def success(req, outcome): + return dict(type="broker_result", protocolVersion=1, requestId=req["requestId"], + status="succeeded", action=req["action"], outcome=outcome) + + +class WireTests(unittest.TestCase): + def setUp(self): + self.examples = json.loads(Path(__file__).with_name("vectors.json").read_text()) + + def test_all_fifteen_actions(self): + self.assertEqual(set(wire.SCHEMAS), {v["request"]["action"] for v in self.examples}) + for v in self.examples: + with self.subTest(action=v["request"]["action"]): + req = wire.request(wire.compact(v["request"])) + wire.result(wire.compact(v["result"]), req) + + def test_unknown_null_and_duplicate_at_every_object(self): + def objects(value, path=()): + if isinstance(value, dict): + yield path + for key, child in value.items(): + yield from objects(child, path + (key,)) + elif isinstance(value, list): + for i, child in enumerate(value): + yield from objects(child, path + (i,)) + for v in self.examples: + for side in ("request", "result"): + def check(raw): + return wire.request(raw) if side == "request" else wire.result(raw, v["request"]) + for path in objects(v[side]): + for field, value in (("unknown", True), ("unknown", None)): + changed = copy.deepcopy(v[side]) + target = changed + for key in path: + target = target[key] + target[field] = value + with self.assertRaises(ValueError): + check(wire.compact(changed)) + target = v[side] + for key in path: + target = target[key] + if target: + first = next(iter(target)) + encoded = wire.compact(target) + duplicate = b"{" + wire.compact(first) + b":" + wire.compact(target[first]) + b"," + encoded[1:] + raw = wire.compact(v[side]).replace(encoded, duplicate, 1) + with self.assertRaises(ValueError): + check(raw) + for field in target: + changed = copy.deepcopy(v[side]) + item = changed + for key in path: + item = item[key] + item[field] = None + with self.assertRaises(ValueError): + check(wire.compact(changed)) + + def test_error_matrix(self): + req = self.examples[0]["request"] + for code in wire.CODES: + for status in ("failed", "indeterminate"): + r = dict(type="broker_result", protocolVersion=1, requestId=req["requestId"], + status=status, error=dict(code=code, message="no details")) + valid = code == "internal" or ((code == "outcome_unknown") == (status == "indeterminate")) + if valid: + wire.result(wire.compact(r), req) + else: + with self.assertRaises(ValueError): + wire.result(wire.compact(r), req) + + def test_correlation(self): + for v in self.examples: + for field, value in (("requestId", "other"), ("action", "unknown"), ("protocolVersion", 2)): + changed = copy.deepcopy(v["result"]) + changed[field] = value + with self.assertRaises(ValueError): + wire.result(wire.compact(changed), v["request"]) + + def test_boundaries(self): + def valid(action, args): + return wire.request(wire.compact(envelope(action, args))) + for n in (1, 500): + valid("channel.read", dict(channelId=CHANNEL, limit=n)) + for n in (0, 501, True, 1.0): + with self.assertRaises(ValueError): + valid("channel.read", dict(channelId=CHANNEL, limit=n)) + for content in ("", " \n", "a" * 65537, "é" * 32769): + with self.assertRaises(ValueError): + valid("message.post", dict(channelId=CHANNEL, content=content)) + valid("message.post", dict(channelId=CHANNEL, content="é" * 32768)) + with self.assertRaises(ValueError): + valid("message.post", dict(channelId=CHANNEL, content="x", mentions=[])) + with self.assertRaises(ValueError): + valid("message.post", dict(channelId=CHANNEL, content="x", mentions=["f" * 64])) + for action, args in (("profile.set", {}), ("agents.update", {"target": {"pubkey": PUB}}), + ("observer.emit", {"frames": []})): + with self.assertRaises(ValueError): + valid(action, args) + # Complete NIP-AE body, not just raw value length. + overhead = len(wire.compact({"slug": "core", "profile": ""})) + valid("storage.put", {"slug": "core", "value": "x" * (65535 - overhead)}) + with self.assertRaises(ValueError): + valid("storage.put", {"slug": "core", "value": "x" * (65536 - overhead)}) + with self.assertRaises(ValueError): + valid("storage.put", {"slug": "core", "value": "\n" * 33000 + "x"}) + req = valid("observer.emit", {"frames": [{"kind": "x", "payload": "{}"}]}) + with self.assertRaises(ValueError): + wire.result(wire.compact(success(req, {"accepted": 2})), req) + + def test_absence_and_opaque_content(self): + req = envelope("storage.get", {"slug": "core"}) + for outcome in ({}, {"value": ""}, {"value": '{"anything":null}'}): + wire.result(wire.compact(success(req, outcome)), req) + wire.request(wire.compact(envelope("observer.emit", {"frames": [{"kind": "x", "payload": '{"owner":null}'}]}))) + + def test_retry_is_bytes_not_equivalent_json(self): + import hashlib + request = self.examples[0]["request"] + frozen = wire.compact(request) + other = json.dumps(request, indent=2).encode() + self.assertEqual(wire.request(frozen), wire.request(other)) + self.assertNotEqual(hashlib.sha256(frozen).digest(), hashlib.sha256(other).digest()) + + def test_encoding_and_canonical_output(self): + raw = wire.compact(self.examples[0]["request"]) + with self.assertRaises(ValueError): + wire.request(raw.decode().encode("utf-16")) + for action, args in (("storage.get", {"slug": " core "}), + ("presence.set", {"status": " online "}), + ("agents.delete", {"target": {"pubkey": " " + PUB.upper() + " "}})): + wire.request(wire.compact(envelope(action, args))) + for invalid in (CHANNEL.replace("-", "", 1), "{{" + CHANNEL + "}}"): + with self.assertRaises(ValueError): + wire.request(wire.compact(envelope("typing.set", {"channelId": invalid}))) + req = envelope("message.post", {"channelId": CHANNEL, "content": "x"}) + for event_id in (EVENT.upper(), " " + EVENT + " "): + with self.assertRaises(ValueError): + wire.result(wire.compact(success(req, {"eventId": event_id, "kind": 9, "createdAt": 1})), req) + + def test_mutation_guard_is_load_bearing(self): + # Deliberately bypass closed-object validation in the oracle, then show + # the exact negative vector is accepted. This is not a production test. + changed = copy.deepcopy(self.examples[0]["request"]) + changed["scope"] = "invented-authority" + raw = wire.compact(changed) + with self.assertRaises(ValueError): + wire.request(raw) + with patch.object(wire, "obj", lambda *args: None): + self.assertEqual(wire.request(raw)["scope"], "invented-authority") + + +if __name__ == "__main__": + unittest.main() diff --git a/docs/formal/nip-broker/vectors.json b/docs/formal/nip-broker/vectors.json new file mode 100644 index 00000000000..ae452ddb6e4 --- /dev/null +++ b/docs/formal/nip-broker/vectors.json @@ -0,0 +1,380 @@ +[ + { + "request": { + "type": "broker_request", + "protocolVersion": 1, + "requestId": "example-1", + "actionVersion": 1, + "action": "channel.read", + "args": { + "channelId": "5df7dfa8-e919-43df-8efd-f1dcb8af7071", + "limit": 1 + } + }, + "result": { + "type": "broker_result", + "protocolVersion": 1, + "requestId": "example-1", + "status": "succeeded", + "action": "channel.read", + "outcome": { + "messages": [] + } + } + }, + { + "request": { + "type": "broker_request", + "protocolVersion": 1, + "requestId": "example-2", + "actionVersion": 1, + "action": "message.post", + "args": { + "channelId": "5df7dfa8-e919-43df-8efd-f1dcb8af7071", + "content": "hello", + "mentions": [ + "79be667ef9dcbbac55a06295ce870b07029bfcdb2dce28d959f2815b16f81798" + ] + } + }, + "result": { + "type": "broker_result", + "protocolVersion": 1, + "requestId": "example-2", + "status": "succeeded", + "action": "message.post", + "outcome": { + "eventId": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", + "kind": 9, + "createdAt": 1787675471 + } + } + }, + { + "request": { + "type": "broker_request", + "protocolVersion": 1, + "requestId": "example-3", + "actionVersion": 1, + "action": "message.reply", + "args": { + "channelId": "5df7dfa8-e919-43df-8efd-f1dcb8af7071", + "content": "reply", + "replyToEventId": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa" + } + }, + "result": { + "type": "broker_result", + "protocolVersion": 1, + "requestId": "example-3", + "status": "succeeded", + "action": "message.reply", + "outcome": { + "eventId": "bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb", + "kind": 9, + "createdAt": 1787675471 + } + } + }, + { + "request": { + "type": "broker_request", + "protocolVersion": 1, + "requestId": "example-4", + "actionVersion": 1, + "action": "reaction.add", + "args": { + "channelId": "5df7dfa8-e919-43df-8efd-f1dcb8af7071", + "targetEventId": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", + "reaction": "👍" + } + }, + "result": { + "type": "broker_result", + "protocolVersion": 1, + "requestId": "example-4", + "status": "succeeded", + "action": "reaction.add", + "outcome": { + "eventId": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", + "kind": 7, + "createdAt": 1787675471 + } + } + }, + { + "request": { + "type": "broker_request", + "protocolVersion": 1, + "requestId": "example-5", + "actionVersion": 1, + "action": "profile.set", + "args": { + "displayName": "Example" + } + }, + "result": { + "type": "broker_result", + "protocolVersion": 1, + "requestId": "example-5", + "status": "succeeded", + "action": "profile.set", + "outcome": { + "eventId": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", + "kind": 0, + "createdAt": 1787675471 + } + } + }, + { + "request": { + "type": "broker_request", + "protocolVersion": 1, + "requestId": "example-6", + "actionVersion": 1, + "action": "storage.address", + "args": { + "slug": "mem/example" + } + }, + "result": { + "type": "broker_result", + "protocolVersion": 1, + "requestId": "example-6", + "status": "succeeded", + "action": "storage.address", + "outcome": { + "authorPubkey": "79be667ef9dcbbac55a06295ce870b07029bfcdb2dce28d959f2815b16f81798", + "kind": 30174, + "dTag": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa" + } + } + }, + { + "request": { + "type": "broker_request", + "protocolVersion": 1, + "requestId": "example-7", + "actionVersion": 1, + "action": "storage.get", + "args": { + "slug": "core" + } + }, + "result": { + "type": "broker_result", + "protocolVersion": 1, + "requestId": "example-7", + "status": "succeeded", + "action": "storage.get", + "outcome": { + "value": "example profile" + } + } + }, + { + "request": { + "type": "broker_request", + "protocolVersion": 1, + "requestId": "example-8", + "actionVersion": 1, + "action": "storage.put", + "args": { + "slug": "mem/example", + "value": "hello" + } + }, + "result": { + "type": "broker_result", + "protocolVersion": 1, + "requestId": "example-8", + "status": "succeeded", + "action": "storage.put", + "outcome": { + "eventId": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", + "kind": 30174, + "createdAt": 1787675471 + } + } + }, + { + "request": { + "type": "broker_request", + "protocolVersion": 1, + "requestId": "example-9", + "actionVersion": 1, + "action": "presence.set", + "args": { + "status": "online" + } + }, + "result": { + "type": "broker_result", + "protocolVersion": 1, + "requestId": "example-9", + "status": "succeeded", + "action": "presence.set", + "outcome": { + "eventId": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", + "kind": 20001, + "createdAt": 1787675471 + } + } + }, + { + "request": { + "type": "broker_request", + "protocolVersion": 1, + "requestId": "example-10", + "actionVersion": 1, + "action": "typing.set", + "args": { + "channelId": "5df7dfa8-e919-43df-8efd-f1dcb8af7071" + } + }, + "result": { + "type": "broker_result", + "protocolVersion": 1, + "requestId": "example-10", + "status": "succeeded", + "action": "typing.set", + "outcome": { + "eventId": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", + "kind": 20002, + "createdAt": 1787675471 + } + } + }, + { + "request": { + "type": "broker_request", + "protocolVersion": 1, + "requestId": "example-11", + "actionVersion": 1, + "action": "observer.emit", + "args": { + "frames": [ + { + "kind": "acp_write", + "payload": "{\"message\":\"example\"}" + } + ] + } + }, + "result": { + "type": "broker_result", + "protocolVersion": 1, + "requestId": "example-11", + "status": "succeeded", + "action": "observer.emit", + "outcome": { + "accepted": 1 + } + } + }, + { + "request": { + "type": "broker_request", + "protocolVersion": 1, + "requestId": "example-12", + "actionVersion": 1, + "action": "liveness.ping", + "args": { + "channelId": "5df7dfa8-e919-43df-8efd-f1dcb8af7071", + "turnId": "turn-1" + } + }, + "result": { + "type": "broker_result", + "protocolVersion": 1, + "requestId": "example-12", + "status": "succeeded", + "action": "liveness.ping", + "outcome": { + "eventId": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", + "kind": 24200, + "createdAt": 1787675471 + } + } + }, + { + "request": { + "type": "broker_request", + "protocolVersion": 1, + "requestId": "example-13", + "actionVersion": 1, + "action": "agents.create", + "args": { + "channelId": "5df7dfa8-e919-43df-8efd-f1dcb8af7071", + "displayName": "Example", + "systemPrompt": "Help with tests." + } + }, + "result": { + "type": "broker_result", + "protocolVersion": 1, + "requestId": "example-13", + "status": "succeeded", + "action": "agents.create", + "outcome": { + "agentPubkey": "79be667ef9dcbbac55a06295ce870b07029bfcdb2dce28d959f2815b16f81798", + "displayName": "Example", + "channelId": "5df7dfa8-e919-43df-8efd-f1dcb8af7071" + } + } + }, + { + "request": { + "type": "broker_request", + "protocolVersion": 1, + "requestId": "example-14", + "actionVersion": 1, + "action": "agents.update", + "args": { + "target": { + "pubkey": "79be667ef9dcbbac55a06295ce870b07029bfcdb2dce28d959f2815b16f81798" + }, + "displayName": "Example" + } + }, + "result": { + "type": "broker_result", + "protocolVersion": 1, + "requestId": "example-14", + "status": "succeeded", + "action": "agents.update", + "outcome": { + "agentPubkey": "79be667ef9dcbbac55a06295ce870b07029bfcdb2dce28d959f2815b16f81798", + "displayName": "Example", + "updatedFields": [ + "displayName" + ] + } + } + }, + { + "request": { + "type": "broker_request", + "protocolVersion": 1, + "requestId": "example-15", + "actionVersion": 1, + "action": "agents.delete", + "args": { + "target": { + "name": "Example" + } + } + }, + "result": { + "type": "broker_result", + "protocolVersion": 1, + "requestId": "example-15", + "status": "succeeded", + "action": "agents.delete", + "outcome": { + "agentPubkey": "79be667ef9dcbbac55a06295ce870b07029bfcdb2dce28d959f2815b16f81798", + "displayName": "Example" + } + } + } +] diff --git a/docs/formal/nip-broker/wire.py b/docs/formal/nip-broker/wire.py new file mode 100644 index 00000000000..79612e9c425 --- /dev/null +++ b/docs/formal/nip-broker/wire.py @@ -0,0 +1,288 @@ +"""Executable NIP-BA wire examples; independent spec oracle, NOT production SDK. + +Intentionally no networking or cryptographic signature verification. Validates +normalized request/result shapes and basic correlation. See test_wire.py. +""" +import json +import re +import uuid + +HEX = re.compile(r"[0-9a-f]{64}\Z") +SLUG = re.compile(r"(?:core|mem/[a-z0-9][a-z0-9_-]{0,63}(?:/[a-z0-9][a-z0-9_-]{0,63})*)\Z") +WHITE_SPACE = "\u0009\u000a\u000b\u000c\u000d\u0020\u0085\u00a0\u1680\u2000\u2001\u2002\u2003\u2004\u2005\u2006\u2007\u2008\u2009\u200a\u2028\u2029\u202f\u205f\u3000" + + +def require(condition): + if not condition: + raise ValueError("invalid wire value") + + +def obj(value, required, optional=()): + require(type(value) is dict and set(required) <= value.keys() + and value.keys() <= set(required) | set(optional)) + + +def integer(value, bits=32): + require(type(value) is int and 0 <= value < 2 ** bits) + + +def string(value): + require(type(value) is str) + value.encode("utf-8") # also reject lone surrogates + + +def scalar(value, maximum): + string(value) + require(bool(value.strip(WHITE_SPACE)) and len(value.strip(WHITE_SPACE)) <= maximum) + + +def opaque(value, maximum): + string(value) + require(1 <= len(value) <= maximum and all(0x21 <= ord(c) <= 0x7e for c in value)) + + +def hex64(value, key=False): + string(value) + require(bool(HEX.fullmatch(value))) + if key: + x = int(value, 16) + p = 2 ** 256 - 2 ** 32 - 977 + y2 = (x ** 3 + 7) % p + require(x < p and pow(y2, (p - 1) // 2, p) == 1) + + +def channel(value): + string(value) + require(str(uuid.UUID(value)) == value) + + +def payload(value, maximum): + scalar(value, maximum) + require(len(value.encode("utf-8")) <= maximum) + + +def compact(value): + return json.dumps(value, ensure_ascii=False, separators=(",", ":")).encode("utf-8") + + +def parse(raw): + def pairs(items): + result = {} + for key, value in items: + require(key not in result) + result[key] = value + return result + + def reject(_): + raise ValueError("non-integer number") + + if isinstance(raw, bytes): + raw = raw.decode("utf-8") + value = json.loads(raw, object_pairs_hook=pairs, parse_float=reject, parse_constant=reject) + + def walk(v): + require(v is not None) + if type(v) is dict: + for k, child in v.items(): + string(k) + walk(child) + elif type(v) is list: + for child in v: + walk(child) + elif type(v) is str: + string(v) + walk(value) + return value + + +SCHEMAS = { + "channel.read": ("channelId", "rootEventId mentionsOnly cursor limit"), + "message.post": ("channelId content", "mentions"), + "message.reply": ("channelId content replyToEventId", "mentions"), + "reaction.add": ("channelId targetEventId reaction", ""), + "profile.set": ("", "displayName about picture"), + "storage.address": ("slug", ""), + "storage.get": ("slug", ""), + "storage.put": ("slug value", ""), + "presence.set": ("status", ""), + "typing.set": ("channelId", ""), + "observer.emit": ("frames", ""), + "liveness.ping": ("channelId turnId", ""), + "agents.create": ("channelId displayName systemPrompt", "runtime provider model respondTo"), + "agents.update": ("target", "displayName systemPrompt runtime provider model respondTo"), + "agents.delete": ("target", ""), +} +MUTABLE = set("displayName systemPrompt runtime provider model respondTo".split()) +PUBLISHED = {"message.post": 9, "message.reply": 9, "reaction.add": 7, + "profile.set": 0, "storage.put": 30174, "presence.set": 20001, + "typing.set": 20002, "liveness.ping": 24200} +CODES = set("invalid_request unsupported_protocol_version unknown_action unsupported_action_version unsupported unauthenticated unauthorized request_id_conflict action_failed outcome_unknown internal".split()) + + +def request(raw): + r = parse(raw) + obj(r, "type protocolVersion requestId actionVersion action args".split()) + require(r["type"] == "broker_request") + opaque(r["requestId"], 128) + for field in ("protocolVersion", "actionVersion"): + integer(r[field], 16) + require(r[field] == 1) + require(type(r["action"]) is str and r["action"] in SCHEMAS) + action, a = r["action"], r["args"] + required, optional = SCHEMAS[action] + obj(a, required.split(), optional.split()) + # Normalize only argument scalars. Keep raw body bytes outside this oracle + # for retry hashing; never apply this to signed events or opaque content. + def normalize(k, v): + if k in ("content", "value", "payload", "cursor") or type(v) is not str: + return v + v = v.strip(WHITE_SPACE) + if k in ("pubkey", "rootEventId", "replyToEventId", "targetEventId"): + return v.lower() + if k == "channelId": + h = r"[0-9a-fA-F]" + u = f"{h}{{8}}-{h}{{4}}-{h}{{4}}-{h}{{4}}-{h}{{12}}" + require(bool(re.fullmatch(f"(?:{u}|{h}{{32}}|\\{{{u}\\}}|urn:uuid:{u})", v))) + return str(uuid.UUID(v)) + return v + for k, v in a.items(): + if k == "target" and type(v) is dict: + a[k] = {f: normalize(f, item) for f, item in v.items()} + elif k == "mentions" and type(v) is list: + a[k] = [normalize("pubkey", item) for item in v] + elif k == "frames" and type(v) is list: + a[k] = [{f: normalize(f, item) for f, item in frame.items()} + if type(frame) is dict else frame for frame in v] + else: + a[k] = normalize(k, v) + for k, v in a.items(): + if k == "channelId": + channel(v) + elif k in ("rootEventId", "replyToEventId", "targetEventId"): + hex64(v) + elif k in ("displayName", "systemPrompt", "about", "picture", "runtime", "provider", "model", "turnId", "reaction"): + scalar(v, {"displayName": 120, "systemPrompt": 20000, "about": 2000, "reaction": 66}.get(k, 300)) + elif k == "respondTo": + require(v in ("owner-only", "anyone")) + elif k == "status": + require(v in ("online", "away", "offline")) + elif k == "mentionsOnly": + require(type(v) is bool) + elif k == "limit": + integer(v) + require(1 <= v <= 500) + elif k == "cursor": + opaque(v, 256) + elif k == "content": + payload(v, 65536) + elif k == "mentions": + require(type(v) is list and 1 <= len(v) <= 50) + for pubkey in v: + hex64(pubkey, True) + elif k == "slug": + string(v) + require(len(v) <= 255 and bool(SLUG.fullmatch(v))) + elif k == "target": + require(type(v) is dict and set(v) in ({"name"}, {"pubkey"})) + if "name" in v: + scalar(v["name"], 120) + else: + hex64(v["pubkey"], True) + elif k == "value": + payload(v, 65535) + elif k == "frames": + require(type(v) is list and 1 <= len(v) <= 256) + for frame in v: + obj(frame, ("kind", "payload")) + scalar(frame["kind"], 300) + payload(frame["payload"], 65535) + require(len(compact(a)) <= 65535) + if action == "profile.set": + require(bool(a)) + if action == "agents.update": + require(bool(set(a) & MUTABLE)) + if action == "storage.put": + body = {"slug": a["slug"], "profile" if a["slug"] == "core" else "value": a["value"]} + require(len(compact(body)) <= 65535) + return r + + +def result(raw, req): + r = parse(raw) + common = "type protocolVersion requestId status".split() + require(type(r) is dict and r.get("status") in ("succeeded", "failed", "indeterminate")) + succeeded = r["status"] == "succeeded" + obj(r, common + (["action", "outcome"] if succeeded else ["error"]), ["replayed"]) + require(r["type"] == "broker_result") + integer(r["protocolVersion"], 16) + require(r["protocolVersion"] == 1 and r["requestId"] == req["requestId"]) + if "replayed" in r: + require(type(r["replayed"]) is bool) + if not succeeded: + e = r["error"] + obj(e, ("code", "message")) + require(type(e["code"]) is str and e["code"] in CODES) + string(e["message"]) + require((r["status"] == "indeterminate" and e["code"] in ("outcome_unknown", "internal")) + or (r["status"] == "failed" and e["code"] != "outcome_unknown")) + return r + action, o, a = r["action"], r["outcome"], req["args"] + require(action == req["action"]) + if action in PUBLISHED: + obj(o, ("eventId", "kind", "createdAt")) + hex64(o["eventId"]) + integer(o["kind"]) + integer(o["createdAt"], 64) + require(o["kind"] == PUBLISHED[action]) + elif action == "channel.read": + obj(o, ("messages",), ("nextCursor",)) + require(type(o["messages"]) is list and len(o["messages"]) <= a.get("limit", 100)) + if "nextCursor" in o: + opaque(o["nextCursor"], 256) + for event in o["messages"]: + obj(event, "id pubkey created_at kind tags content sig".split()) + hex64(event["id"]) + hex64(event["pubkey"], True) + integer(event["created_at"], 64) + integer(event["kind"]) + string(event["content"]) + string(event["sig"]) + require(bool(re.fullmatch(r"[0-9a-f]{128}", event["sig"]))) + require(type(event["tags"]) is list) + for tag in event["tags"]: + require(type(tag) is list) + for value in tag: + string(value) + elif action == "storage.get": + obj(o, (), ("value",)) + if "value" in o: + string(o["value"]) + elif action == "storage.address": + obj(o, ("authorPubkey", "kind", "dTag")) + hex64(o["authorPubkey"], True) + hex64(o["dTag"]) + integer(o["kind"]) + require(o["kind"] == 30174) + elif action == "observer.emit": + obj(o, ("accepted",)) + integer(o["accepted"]) + require(o["accepted"] <= len(a["frames"])) + else: + fields = ["agentPubkey", "displayName"] + if action == "agents.create": + fields.append("channelId") + if action == "agents.update": + fields.append("updatedFields") + obj(o, fields) + hex64(o["agentPubkey"], True) + scalar(o["displayName"], 120) + if action == "agents.create": + channel(o["channelId"]) + require(uuid.UUID(o["channelId"]) == uuid.UUID(a["channelId"])) + elif "pubkey" in a["target"]: + require(o["agentPubkey"].lower() == a["target"]["pubkey"].lower()) + if action == "agents.update": + changed = o["updatedFields"] + require(type(changed) is list and all(type(v) is str for v in changed)) + require(changed == sorted(set(changed)) and set(changed) <= set(a) & MUTABLE) + return r diff --git a/docs/nips/NIP-BA.md b/docs/nips/NIP-BA.md new file mode 100644 index 00000000000..2142434c982 --- /dev/null +++ b/docs/nips/NIP-BA.md @@ -0,0 +1,361 @@ +NIP-BA +====== + +Brokered Agent Operations +------------------------- + +`draft` `optional` `Buzz-local, unassigned upstream` + +An agent delegates named application operations to a **host** that holds its +Nostr identity key. The agent holds a public identity and a sensitive session +credential, not that durable key. The host authenticates, authorizes, constructs, +signs, and publishes ordinary Nostr events; reads also pass through the host. +This specification defines their HTTP boundary, not a relay extension or a host +implementation. MUST, MUST NOT, SHOULD, and MAY are normative requirements. + +This is a closed Buzz capability profile. It relies on +[NIP-01](https://github.com/nostr-protocol/nips/blob/master/01.md), +[NIP-10](https://github.com/nostr-protocol/nips/blob/master/10.md), +[NIP-29](https://github.com/nostr-protocol/nips/blob/master/29.md), and the local +[NIP-AE](NIP-AE.md), [NIP-AO](NIP-AO.md), and [NIP-OA](NIP-OA.md) conventions. +No generic signing, publishing, encryption, decryption, or relay-authentication +action is exposed. Adding capabilities requires revising this specification. Channel UUIDs are a +Buzz restriction; upstream NIP-29 accepts arbitrary group IDs. The marked-root/ +reply convention from NIP-10 is reused for Buzz kind 9, not a claim that NIP-10 +itself defines kind 9 messaging. + +[NIP-46](https://github.com/nostr-protocol/nips/blob/master/46.md) already keeps +identity keys elsewhere and permits inspection and refusal of complete unsigned +events. The distinction here is **executing application operations and mediating +reads**, not an inability to constrain NIP-46 signing. Layering a NIP-46 signer +behind a broker is possible but not specified, including memory-key derivation. + +## Session and transport + +The endpoint and credential are provisioned out of band. Each credential MUST +bind to one agent principal, community, owner relationship, and permitted scope. +The host MUST derive these from authentication, never from request content. +Channel IDs and agent selectors are targets, not grants. Authorization MUST +include current community/channel membership and ownership where applicable; +the broker cannot expand relay authority. Credentials MUST be revocable and MUST +NOT be reassigned to another principal or community. + +```http +POST /v1/action +Authorization: Bearer +Content-Type: application/json + + +``` + +Use TLS with server authentication, except for a deliberately configured +loopback endpoint. Clients MUST NOT forward credentials on redirects and MUST +NOT send loopback traffic through a proxy. Hosts MUST authenticate every attempt, +including retries, and authorize both execution and release of stored results. +A revoked credential MUST NOT retrieve a cached read or outcome. + +Every correlated verdict MUST be a JSON result with HTTP 200 and +`Content-Type: application/json`. Clients MUST attempt envelope validation +regardless of HTTP status; a valid correlated result takes precedence. A proxy +401, missing route, timeout, truncated body, or invalid result is a **transport +failure**, not evidence of non-execution. No usable `requestId` means no +correlated verdict is possible; the host MUST NOT execute such a request. + +Hosts and clients MUST bound body sizes, concurrent work, timeouts, and retries. +Hosts MUST document deployment limits and reject oversized input before dispatch; +clients MUST use bounded backoff, not an unbounded retry loop. This version has +no discovery endpoint or automatic version negotiation. + +## Wire and normalization + +JSON is UTF-8. Objects are closed: unknown members, duplicate member names, +wrong types, and explicit `null` MUST be rejected at every structural depth. +This does not parse JSON embedded inside opaque strings. Integers MUST have +integer JSON syntax, be nonnegative, and fit the stated unsigned width; booleans +are not integers. All version fields are u16, counts/kinds u32, timestamps u64 +Unix seconds. Senders MUST preserve integers exactly, including in JavaScript. + +`?` below means optional by **omission**, never `null`. The optional `mentions` +array MUST be omitted when empty; explicit `[]` is invalid. Required arrays +(`messages`, `tags`, `updatedFields`) MAY be empty. `frames` MUST NOT be empty. +Optional booleans default false; senders SHOULD omit false, receivers accept it. + +Before freezing a request, clients MUST validate and normalize. Hosts MUST +validate independently; normalization MUST NOT change the body used for the +retry digest. Trim surrounding Unicode White_Space from scalar arguments, then +apply limits; required and supplied optional scalars MUST be nonempty. Do not +trim `requestId`, cursors, message `content`, storage `value`, or observer +`payload`. The latter three MUST contain a non-whitespace character but retain +all original bytes. Limits count Unicode scalar values unless marked **bytes**. +Action names and wire discriminators are exact, not trimmed. Signed event +fields MUST NOT be normalized or rewritten. + +| Type | Accepted form and canonical output | +|---|---| +| Channel | UUID, lowercase hyphenated output; input also accepts uppercase hex, 32 hex digits, braces around hyphenated form, or `urn:uuid:` plus hyphenated form | +| Hex | 64 hexadecimal characters, lowercase output | +| Pubkey | Hex encoding of a valid secp256k1 x-only public key; not an npub | +| Request ID | 1–128 bytes, each `0x21`–`0x7e` | +| Cursor | 1–256 bytes, each `0x21`–`0x7e`; opaque, retained verbatim | +| Name / prompt / scalar / about | At most 120 / 20,000 / 300 / 2,000 scalars respectively | +| Slug | `core` or `^mem/[a-z0-9][a-z0-9_-]{0,63}(/[a-z0-9][a-z0-9_-]{0,63})*$`, at most 255 bytes | + +### Request + +Exactly six required members: + +```json +{"type":"broker_request","protocolVersion":1,"requestId":"example-1","actionVersion":1,"action":"storage.get","args":{"slug":"core"}} +``` + +`type` is `broker_request`; `protocolVersion` is 1. All fifteen actions below +have `actionVersion` 1. `requestId` is caller-chosen and unique for each logical +operation in the host's community/principal namespace. `args` has exactly the +chosen action's shape. No requester, owner, credential, relay, or scope member +is permitted. Missing versions do not mean version 1. + +### Result + +Common required members: `type:"broker_result"`, `protocolVersion:1`, +`requestId` copied exactly, and `status`. Optional `replayed` is boolean. + +| Status | Additional required members | Promise | +|---|---|---| +| `succeeded` | `action`, `outcome` | Operation completed as defined by the action | +| `failed` | `error:{code,message}` | This submission caused no operation effects | +| `indeterminate` | `error:{code,message}` | Effects may have happened; reconciliation is required | + +Success MUST NOT contain `error`; other statuses MUST NOT contain `action` or +`outcome`. `message` is an operator-facing string, not machine-readable detail; +it MUST NOT disclose keys, credentials, or decrypted application payloads. + +| Error code | Meaning | Permitted status | +|---|---|---| +| `invalid_request` | Malformed envelope/arguments | failed | +| `unsupported_protocol_version` | Unsupported envelope version | failed | +| `unknown_action` | Unknown action name | failed | +| `unsupported_action_version` | Unsupported version of a known action | failed | +| `unsupported` | Known action not offered | failed | +| `unauthenticated` | Missing, invalid, expired, or revoked credential | failed | +| `unauthorized` | Session lacks permission | failed | +| `request_id_conflict` | Same retry key, different received bytes | failed | +| `action_failed` | Known domain failure with no effects | failed | +| `outcome_unknown` | Execution effects cannot be determined | indeterminate | +| `internal` | Unexpected host fault | failed or indeterminate | + +If several preflight checks fail, the host MAY choose any applicable code, +but MUST NOT expose authenticated state to an unauthenticated caller. +Authentication refusal is a result when correlation is possible, not a transport +error. **A refusal of a retry does not undo or disprove earlier execution.** In +particular, conflict, revocation, or permission loss only proves this attempt +introduced no new effects. Clients with earlier uncertainty MUST retain it. + +Clients MUST validate the complete shape, status/code pair, request ID, and +success action before accepting a verdict. Validate all outcome identities and +bounds, the create channel against the requested channel, and update/delete +pubkey against a pubkey selector. A name selector cannot prove identity by echo. +Read pages MUST fit the effective limit and observer receipts the submitted +frame count. A mismatch is a transport failure. Event signature verification +is separate; clients SHOULD verify NIP-01 IDs/signatures before trusting content. + +## Execution and retries + +Let `K = (community, authenticated principal, requestId)` and +`D = SHA-256(received HTTP body bytes)`. Community is host-derived; a shared host +MUST NOT allow one community to retrieve another's records. Credential rotation +within the same principal/community MUST retain this namespace. + +1. Client validates, normalizes, and serializes **once**. Every retry MUST send + the identical bytes and ID. Equivalent reserialized JSON is not a retry. +2. After authentication and permission checks, the host MUST atomically claim + an unused K with D in durable storage **before any operation effect**. Only + one executor may own K, including across replicas and restarts. +3. An existing K with a different D MUST return `request_id_conflict`, without + overwriting the record or executing. A matching completed record MUST return + its stored domain result, adding `replayed:true`. Delivery metadata is not + part of the stored result. Replaying MUST NOT execute, republish, or renew an + ephemeral signal. +4. For a matching in-flight record, the host MUST join/wait within its timeout + or return `indeterminate/outcome_unknown`; it MUST NOT start another executor. + A wait timeout need not finalize the record: the original executor can finish. +5. After a crash, a record whose executor may have acted MUST NOT be reclaimed + for fresh execution. Reconcile from durable execution evidence or retain an + indeterminate result. A lease expiring alone is not proof of non-execution. +6. Store the result durably before delivering it. `failed` requires proof that + no operation effects occurred; partial publication/provisioning is not a + failure with no effects. `indeterminate` may later resolve only from evidence, + never from re-executing the requested operation. + +The host MUST retain records for as long as that principal/community namespace +can submit requests. If full results are discarded, a durable tombstone MUST +retain K and D and prevent re-execution; a matching retry then returns +`indeterminate/outcome_unknown`, not fabricated success or failure. A host MAY +retire a namespace only if it permanently rejects all future submissions in it. +It MUST apply admission quotas rather than silently evict retry protection. +Clients MUST NOT reuse an ID, or switch to a new ID to retry an uncertain write. + +This is **at-most-once dispatch**, not exactly-once relay delivery. Hosts MUST +fence workers and any downstream retries so one dispatch cannot itself mint +duplicate effects. Persisting an event before publication can enable replay of +that same signed event; whether a relay accepts it remains a separate question. +Reconciliation can require operator intervention; termination is not guaranteed +through a permanent partition. New reads and periodic signals use new IDs. + +## Actions + +`Published` means exactly `{eventId:Hex,kind:u32,createdAt:u64}`. It describes the +host-built event, not a signed receipt. Success requires positive acceptance by +the configured community relay, not merely a local send; ephemeral acceptance +does not guarantee any subscriber saw it. The host chooses timestamps, signer, +relay, and tags, except values derived from explicit arguments below. + +| Action | Exact args (`?` optional) | Exact outcome | +|---|---|---| +| `channel.read` | `channelId:Channel, rootEventId?:Hex, mentionsOnly?:bool, cursor?:Cursor, limit?:u32` | `messages:Event[], nextCursor?:Cursor` | +| `message.post` | `channelId:Channel, content:string, mentions?:Pubkey[]` | Published | +| `message.reply` | post args plus `replyToEventId:Hex` | Published | +| `reaction.add` | `channelId:Channel, targetEventId:Hex, reaction:string` | Published | +| `profile.set` | `displayName?:Name, about?:about, picture?:scalar` | Published | +| `storage.address` | `slug:Slug` | `authorPubkey:Pubkey, kind:u32, dTag:Hex` | +| `storage.get` | `slug:Slug` | `value?:string` | +| `storage.put` | `slug:Slug, value:string` | Published | +| `presence.set` | `status:"online"\|"away"\|"offline"` | Published | +| `typing.set` | `channelId:Channel` | Published | +| `observer.emit` | `frames:{kind:scalar,payload:string}[]` | `accepted:u32` | +| `liveness.ping` | `channelId:Channel, turnId:scalar` | Published | +| `agents.create` | `channelId:Channel, displayName:Name, systemPrompt:prompt, runtime?:scalar, provider?:scalar, model?:scalar, respondTo?:mode` | `agentPubkey:Pubkey, displayName:Name, channelId:Channel` | +| `agents.update` | `target:Target, displayName?:Name, systemPrompt?:prompt, runtime?:scalar, provider?:scalar, model?:scalar, respondTo?:mode` | `agentPubkey:Pubkey, displayName:Name, updatedFields:string[]` | +| `agents.delete` | `target:Target` | `agentPubkey:Pubkey, displayName:Name` | + +### Channel and messages + +`limit` is 1–500; omission means 100. Filters intersect: channel, optional +thread root, and optional signed `p` mention of the authenticated agent. +`Event` has exactly the seven NIP-01 members `id,pubkey,created_at,kind,tags, +content,sig`; tags are arrays of string arrays, signature is 128 hex characters. +The host MUST return original signed events, not projections. All returned +messages MUST satisfy the requested channel/thread/mention filters. Ancestry +and mentions come from signed tags, not unsigned sibling metadata. + +The host MUST document initial window, ordering, and cursor lifetime. A cursor +MUST bind to the principal/community and query filters and preserve continuation +without skipping same-timestamp events. Invalid, expired, or mismatched cursors +MUST fail with `invalid_request`, never silently restart. Clients MUST round-trip +cursors verbatim, never parse, synthesize, or compare them for ordering. Continue +while `nextCursor` is present, even after a short page; omission means exhausted +for that traversal, not a promise that no future message can arrive. Each page +uses a new ID. Clients SHOULD deduplicate overlapping polling windows by event +ID. This is not a subscription or a completeness proof. + +Post/reply content is at most 65,536 bytes; mentions contain 1–50 pubkeys when +present. The host constructs kind 9 messages with the target channel's `h` tag +and supplied notification `p` tags. Mentions do not grant membership. For replies, +the host MUST fetch/validate the parent in that channel, derive its actual +NIP-10 root, and encode both ancestry and immediate parent consistently; it MUST +NOT silently treat a nested parent as the root. Missing, inaccessible, or +inconsistent ancestry MUST fail before publication. Reaction targets likewise +MUST belong to the channel; reaction text is trimmed, nonempty, at most 66 +scalars, published as kind 7. Hosts MAY restrict reaction vocabulary further. + +Profile setting publishes kind 0 for the requester, merging only supplied fields; +at least one field is required. Omitted fields remain unchanged. Empty/whitespace +values are invalid, not clear commands. Picture is a bounded string, not a +promise that its URL is safe to fetch. + +### Memory + +The host performs NIP-AE address derivation, validation, head selection, and +NIP-44 encryption/decryption for the session's agent/owner pair, restricted to +the session community. `storage.address` returns the agent's author pubkey, +kind 30174, and NIP-AE's HMAC-derived d-tag, never its conversation key. + +`storage.get` returns `{}` for a missing head or memory tombstone. Otherwise +`value` is the memory body's `value`, or **core body's `profile` string**, not +serialized core JSON. An existing empty string remains `{"value":""}`. +Unavailable reads MUST NOT masquerade as absent memory. + +`storage.put` replaces the selected record. Encode exactly +`{"slug":s,"value":v}` for memory or `{"slug":"core","profile":v}` for core. +The complete compact UTF-8 JSON plaintext, including escaping, MUST fit 65,535 +bytes. This is the Buzz NIP-AE/profile cap, not a claim about the maximum of +all upstream NIP-44 formats. For size measurement use the displayed member +order, no insignificant whitespace, literal non-ASCII scalars, escapes `\"`, `\\`, `\b`, `\t`, `\n`, +`\f`, `\r`, and lowercase `\u00xx` for other U+0000–001F controls. Do not escape +`/`. Publish kind 30174 following NIP-AE's monotonic timestamp rule. This API +has no delete, empty write, or core-clear operation. Concurrent distinct-ID +writes retain NIP-AE's eventual head semantics; success is not a lasting lock +on the head. The broker's no-null rule does not forbid NIP-AE tombstones inside +host-decrypted records. + +### Live signals + +Reaction and all four live actions are **best effort**: `unsupported` is a +normal refusal and clients MUST be able to continue without them. Other actions +can also be refused but callers MUST surface loss of the requested capability. +Best effort does not weaken authentication, validation, or retry guarantees. + +Presence publishes kind 20001 with content equal to `status`; offline clears +presence. Typing publishes kind 20002 with channel `h` tag and empty content. +There is no stop action: the indicator expires under relay policy. Each fresh +renewal is a new operation, not a replay of the preceding signal. + +Observer frame `kind` is a runtime discriminator, **not a Nostr kind**. Payload +is an opaque serialized string; the host MUST NOT interpret it as routing or +authority. There are 1–256 frames; each payload is at most 65,535 bytes, and the +whole normalized `{"frames":[{"kind":k,"payload":p},...]}` is at most 65,535 +bytes using the compact encoding rule above. The host encrypts accepted payloads +to the session owner using NIP-AO telemetry, deriving all Nostr metadata itself, +and MAY batch/pace publication. `accepted` is a count, 0 through input length, +not a list or prefix guarantee. It acknowledges intake for best-effort +publication, **not relay delivery**. Clients MUST NOT infer which frames were +accepted or resubmit a guessed remainder. Replaying a receipt does not enqueue +again. Drops after intake remain possible. + +Liveness refers to the current session's channel and process-local `turnId`. +The host MUST reject a turn not bound to that session/channel before effects. +It publishes an owner-encrypted NIP-AO telemetry event (kind 24200) describing +the keepalive and MAY also renew that turn's stall watchdog. Watchdog renewal +and relay publication can partially complete; that requires `indeterminate`, +not `failed`. The precise runtime telemetry payload is defined by the runtime +profile, not this action interface; neither it nor observer data proves actual +computation progress. + +### Managed agents + +`Target` is exactly `{"pubkey":Pubkey}` or `{"name":Name}`. The host resolves +names within authorized ownership scope and MUST reject missing or ambiguous +matches, never choose an arbitrary agent. `mode` is `owner-only` or `anyone`. +Create defaults to owner-only; optional runtime/provider/model defaults are +host-declared and unknown choices MUST be refused, not silently substituted. + +Create mints an identity, stores configuration and attaches it to `channelId`; +the authenticated requester is its owner. Ownership chains MUST terminate at +a human; allowed depth is host policy. No private key or credential is returned. +Success promises the managed record and attachment exist, not that the runtime +has booted. Hosts MUST document provisioning and deletion scope. + +Update requires at least one mutable field. Absent fields remain unchanged; +empty strings do not clear them. `updatedFields` lists actually changed wire +names in lexicographic order without duplicates; it is empty for an idempotent +no-op patch and MUST be a subset of supplied mutable fields. Delete removes the +managed record; it does not erase historical Nostr events or imply that an +unreachable process stopped. Outcomes report the resulting (or deleted) name. +Partial provisioning/deletion MUST be indeterminate unless all effects are +proven absent. Authorization is rechecked at the operation's effect boundary. + +## Security and conformance boundary + +Both sides MUST implement the wire, correlation, and retry rules; hosts MAY +refuse individual capabilities. These requirements are independent of language +and of any SDK. The [model and conformance note](../formal/nip-broker/NOTE.md) +records checked properties, proposed clarifications, tests, and exclusions; +it does not replace the normative rules above. + +The trusted host can impersonate its agents and suppress reads. Signatures prove +event authorship/content, not freshness, completeness, or benign intent. Memory +and observer payloads are plaintext at the broker boundary and MUST be protected +in transit, logs, and retry storage. Closed schemas remove key-export fields; +**they cannot prevent a string from containing a secret**. A compromised bearer +can exercise its permitted capabilities until revoked. Loopback alone does not +isolate same-user hostile processes. This contract is not a sandbox, a complete +information-flow policy, a cryptographic proof, or an upstream-accepted NIP.