Skip to content

feat(discovery): Phase 4 reservation voucher protocol surface (EXPERIMENTAL) - #240

Open
metaphorics wants to merge 8 commits into
discovery-phase-3-multihop-diversityfrom
discovery-phase-4-reservation-voucher
Open

feat(discovery): Phase 4 reservation voucher protocol surface (EXPERIMENTAL)#240
metaphorics wants to merge 8 commits into
discovery-phase-3-multihop-diversityfrom
discovery-phase-4-reservation-voucher

Conversation

@metaphorics

Copy link
Copy Markdown
Contributor

Phase 4 of the discovery rationalization

Builds on PR #239 (Phase 3 multi-hop diversity). Adds the reservation-voucher protocol surface mirroring libp2p Circuit Relay v2 semantics. Stacked PR: base = discovery-phase-3-multihop-diversity.

Plan-deferral caveat (READ THIS BEFORE ENABLING)

Phase 4 was specified to ship only if Phase 1-3 telemetry demonstrated the libp2p-v1 oversubscription failure mode (unbounded queue growth, late binding, retry storms). User has directed it ships ahead of that data anyway.

This PR delivers the negotiation surface only — NO dataplane enforcement. A relay that grants a voucher does not refuse traffic from clients without one. The voucher mechanism exists so operators can:

  1. Publish capacity-supports flags via RelayDescriptor.SupportsReservation.
  2. Issue signed vouchers via /admin/reserve with a per-process budget.
  3. Have clients prefer cached-voucher relays via the new voucher.Voucher selector wrapper.

Do NOT enable in production until Phase 1+ telemetry justifies it. Every public type carries an EXPERIMENTAL doc comment.

What landed (8 atomic commits)

  1. 5c7e9948 feat(types): add ReservationVoucher type + RelayDescriptor.SupportsReservation flagReservationVoucher{ClientAddress, RelayURL, IssuedAt, ExpiresAt, Signature} with CanonicalBytes(). RelayDescriptor.SupportsReservation is advisory (NOT in CanonicalBytes — legacy signed descriptors continue to verify).
  2. 60843a99 feat(auth): SignReservationVoucher + VerifyReservationVoucher — secp256k1 signing/verifying mirroring the descriptor pattern. Recoverable signatures (no out-of-band public key needed).
  3. 26394ec0 feat(discovery/voucher): in-memory voucher cache with expiry evictionvoucher.Cache with sync.RWMutex-protected map; Get and Has evict expired entries on read.
  4. 47b647bc feat(discovery/voucher): selector wrapper with three-bucket partitionvoucher.Voucher wraps an inner Selector. Algorithm partitions candidates into A (legacy bypass), B (cached preferred), C (uncached supports-reservation, demoted). Stable sort keeps MOLS order within each bucket.
  5. 8fbfd980 feat(relay-server): /admin/reserve handler with capacity budget — POST /admin/reserve (auth-gated under types.PathAdminPrefix); request {client_address, requested_duration_seconds}; response 200 with signed voucher, or 503 capacity-exhausted. Per-process atomic counter; default budget 100; tracks vouchers issued (not active-tunnel count).
  6. b40c897b test(loadtest): -reservation flag pre-seeds synthetic voucher cachecmd/portal-loadtest -reservation mints a synthetic signing identity, signs all relay descriptors, pre-seeds the voucher cache, and wraps the selector chain voucher.New(diversity.New(weighted.New(mols.New()))).
  7. 6c947c75 fix(relay-server): use APIHTTPSAddr (not secp256k1 addr) as voucher.RelayURL — semantic-correctness fix: voucher.RelayURL should be the API URL clients connect to, not the relay's secp256k1 identity address. Pre-existing tests didn't catch this; new assertion voucher.RelayURL != address added in admin_test.go.
  8. 0f3f7e0b chore(discovery): drop dead relayAddress param + stale nolint hint — cleanup-codebase post-review tidy: removes unused relayAddress string parameter from handleAdminReserve (left over after feat: add WASM client web server and improve service worker #7 stopped using it) and replaces a stale //nolint:errcheck pragma in cache_test.go (Cache.Get returns (T, bool), no error to check).

Verification (local)

  • go build ./... exit 0
  • go vet ./... exit 0
  • make lint exit 0 (0 issues)
  • go test -count=1 ./... exit 0 (273 tests pass across 25 packages, up from Phase 3's 243 in 24)
  • go mod verify clean

Public API additions

Type / Function Purpose EXPERIMENTAL?
types.ReservationVoucher Signed claim YES
types.RelayDescriptor.SupportsReservation bool Capability flag YES
auth.SignReservationVoucher / auth.VerifyReservationVoucher secp256k1 round-trip YES
voucher.Cache (portal/discovery/voucher/) Client-side cache YES
voucher.Voucher selector wrapper Three-bucket partition YES
/admin/reserve HTTP route RPC surface YES
cmd/portal-loadtest -reservation Synthetic test mode YES

No public API breakage. All Phase 1-3 selectors and seams continue to work unchanged (voucher.New(...) is opt-in via the SDK config, not wired by default).

What this PR does NOT do (deliberate scope cap)

  • No dataplane enforcement. A relay that issues a voucher does NOT reject traffic from un-vouchered clients. The mechanism is negotiation-only.
  • No client-side automatic acquisition. The voucher.Voucher selector demotes uncached candidates but never calls /admin/reserve itself. A future phase would wire a background acquirer that polls the cache.
  • No persistence. Voucher cache and capacity budget are in-memory per process; relay restart resets the budget.
  • No per-tunnel budget tracking. The /admin/reserve counter increments per voucher issued, not per active tunnel. Voucher expiry is the only release mechanism.

Acceptance evidence

go run ./cmd/portal-loadtest -clients 100 -relays 5 -reservation (cache pre-populated; all relays SupportsReservation=true):

  • Vouchers granted: 100 (one per client × all relays cached at startup, no /admin/reserve calls)
  • Capacity-exhausted: 0
  • Legacy-bypass: 0
  • Selector chain: mols+diversity+voucher

The synthetic load-test does NOT exercise /admin/reserve (cache is pre-seeded). Production wiring would require:

  1. Relay-server reachable at the URL each synthetic relay descriptor advertises.
  2. A background acquirer goroutine polling the cache and calling /admin/reserve for each SupportsReservation=true relay missing a fresh voucher.

This is mechanism-only by design.

Atomic-commit gate

git log --oneline 5c37195d..HEAD shows 35 commits across all 4 phases (22 Phase 2 + 5 Phase 3 + 8 Phase 4). Each with a single concern; no behavior+cleanup mixing. Phase 4 includes one explicit fix: commit (#7 — semantic correctness) and one chore: commit (#8 — cleanup-codebase post-review).

Out of scope (entirely future)

  • Background voucher acquirer in the SDK.
  • Dataplane enforcement (refusing un-vouchered tunnels).
  • Voucher persistence across relay restarts.
  • Per-tunnel budget tracking that releases on tunnel close (the current per-process counter only releases on voucher expiry).
  • Production deployment of any of Phase 2-4 mechanisms — telemetry from Phase 1 (PR feat(discovery): Phase 1 telemetry — measure non-uniform relay binding before changing logic #235) should validate which mechanisms are actually load-bearing before deploy.

…elayURL

handleAdminReserve was setting RelayURL to the relay's secp256k1/EVM address
instead of the relay's API HTTPS URL (APIHTTPSAddr).  The client-side voucher
cache keys on APIHTTPSAddr, so vouchers issued with the wrong URL would never
match a cached entry and the three-bucket partition would always demote the
relay to bucket C.

Fix: add relayAPIURL string parameter, plumb f.server.PortalURL() through from
serveAdmin, and assert voucher.RelayURL == expectedAPIURL in the round-trip
test.
handleAdminReserve received a relayAddress parameter that was never read
inside the function body (the relay's secp256k1 address is not needed for
voucher signing — only the private key + APIHTTPSAddr are). Remove the
parameter and update both call sites and the test helper.

cache_test.go's TestCacheConcurrent had a //nolint:errcheck pragma against
Cache.Get, which returns (ReservationVoucher, bool) — no error; the pragma
was stale from an earlier iteration of the API. Replace with a plain blank
identifier discard.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 0f3f7e0bd9

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread portal/auth/voucher.go
Comment on lines +77 to +78
func VerifyReservationVoucherFromDescriptor(v types.ReservationVoucher, desc types.RelayDescriptor) error {
return VerifyReservationVoucher(v, desc.Address)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Bind voucher URL when verifying from a descriptor

VerifyReservationVoucherFromDescriptor only checks the recovered signer address and never verifies that v.RelayURL matches desc.APIHTTPSAddr, so a relay can sign a voucher that names a different relay URL and still pass verification. In flows that cache by RelayURL, this enables cross-relay voucher poisoning/misbinding (the voucher is cryptographically valid for the signer but semantically bound to the wrong relay endpoint).

Useful? React with 👍 / 👎.

Comment on lines +59 to +63
if stored, still := c.store[relayURL]; still && time.Now().After(stored.ExpiresAt) {
delete(c.store, relayURL)
}
c.mu.Unlock()
return types.ReservationVoucher{}, false

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Return refreshed entry after expired-read recheck

When Get sees an expired entry on the read path, it rechecks under the write lock, but it always returns (zero, false) even if another goroutine replaced that key with a fresh voucher before the lock was acquired. This produces false cache misses under concurrent refresh and can incorrectly demote reservation-capable relays despite a valid voucher being present.

Useful? React with 👍 / 👎.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant