bedrock-q67.9 (3/4): clients read the shard map from commit proxies - #174
Open
jallum wants to merge 1 commit into
Open
bedrock-q67.9 (3/4): clients read the shard map from commit proxies#174jallum wants to merge 1 commit into
jallum wants to merge 1 commit into
Conversation
jallum
pushed a commit
that referenced
this pull request
Aug 20, 2026
…onto feature/q67-phase-b-prereqs, per the audited design. #172 materializers/<tag> keyspace family (serverList analogue: string {worker_id,node} refs; recovery clear+rewrite gated on shard_materializers; RoutingData fold incl. clear_range; seed extension = q67.23 formalized; audit PASS-WITH-NOTES, gate-asymmetry fix applied). #173 CommitProxy.fetch_routing/2 (GetKeyServerLocations: client projection = shards+materializers only; :locked refusal = validState; cadence-preserving reply; audit PASS-WITH-NOTES, cadence test added in #174's layer). #174 client switch (Link = node locationCache, dropped on wiring push; LAZY routing_fn - builder indexes on first read, read-free txs never fetch; string->callable ref conversion documented atom exception; retry reclassification: layout_lookup_failed/no_servers_to_race/version_too_old/locked retryable + invalidate-on-routing-shaped-failure only; audit PASS-WITH-NOTES). #175 the headline kill (TSL = wiring only, ClientDBInfo shape: epoch/sequencer/proxies/resolvers/logs/services; shard_layout/shard_materializers/metadata_materializer/id/director/rate_keeper DELETED; recovery derives keyspace writes + unlock seed from RecoveryAttempt via one materializer_refs/2 helper; RecoveryAttempt.metadata_materializer deleted; LayoutIndex segmenting collapsed to direct end_key tree per #174 audit - old boundary sort fed duplicate keys into gb_trees.from_orddict at seams; Link.invalidate_routing made synchronous - ordering by construction not by accident; audit in flight). Exit criterion status: broadcast no longer carries shard topology (fields gone); clients read the map from proxies; Distributor-publishes-coverage remains q67.21's half. Audit-recorded precluded/known items: materializer parse_key non-injective on non-canonical decimals (writers all go through materializer_key/1, injective - recorded, no guard); {:error,:unknown} rescued-exception shape escapes the retry loop (bug path, arguably right)."
The client read path moves off the recovery-frozen TSL shard fields and onto proxy-served routing - FDB's client location model, whole: - The Link is the node's locationCache (FDB DatabaseContext): it stores the raw projection, no TTL; a wiring push (tsl_updated) drops it so new-epoch wiring can never pair with old-epoch routing. - Routing is fetched LAZILY, like the read version: the builder builds its LayoutIndex on first read via a routing_fn (Link cache first, fetch-through to a random commit proxy on a miss, cast the projection back). A transaction that never reads never fetches routing - FDB fetches locations per read, not per transaction. - String refs become callable refs at the client: worker OTP names are deterministic in the worker id; the node-atom conversion is the documented exception to the no-atoms-on-decode rule (system-mode-gated writers, count bounded by cluster membership). - Retry reclassification - the load-bearing piece: layout_lookup_failed, no_servers_to_race, version_too_old, and :locked join the retryable set (resolving the old get/get_range inconsistency), and the retry invalidates the routing cache on routing-shaped failures (a dead pid is exactly what a stale snapshot looks like). Version-window misses keep the cache - they are not routing's fault. FDB parity: routing staleness costs the user a retry, never an error, never a wrong answer. - LayoutIndex now takes shard boundaries + materializer refs directly; the TSL-field derivation survives only for caller-provided layouts and dies with q67.9's final layer. Also pins the fetch_routing cadence as direct handler assertions (from the PR #173 audit): a routing fetch must not swallow an open batch's pending timeout or the heartbeat.
jallum
added a commit
that referenced
this pull request
Aug 20, 2026
…q67.9) The Phase B headline deletion. The TransactionSystemLayout shrinks to FDB's ClientDBInfo shape - epoch, sequencer, proxies, resolvers, logs, services - and shard topology has exactly one home: the \xFF/system keyspace, projected to proxies by the metadata pipeline and served to clients by fetch_routing. The tsl_updated broadcast still exists (FDB keeps ClientDBInfo too - runtime wiring legitimately rides a coordinator push) but it can no longer carry a shard map: the fields are gone. - TSL loses shard_layout, shard_materializers, metadata_materializer, and the dead fields id, director, rate_keeper (never read; random_id deleted with its last caller). - Recovery keeps shard state on the RecoveryAttempt, where it always lived: persistence writes the keyspace families from the attempt, topology builds the unlock seed from the attempt. The unread RecoveryAttempt.metadata_materializer field is deleted too - tag 0's materializer is just shard_materializers[0]. - TransactionBuilder's TSL-derivation fallback is deleted as promised: without a routing_fn the index is empty and reads fail as layout_lookup_failed. - LayoutIndex's segmenting pass collapses to a direct end_key tree (from the PR #174 audit): the input is provably non-overlapping, and the old boundary-sort fed duplicate keys into gb_trees.from_orddict at adjacent seams - an accident waiting for a contract check. - Link.invalidate_routing becomes synchronous (same audit): the retry that invalidates must not be able to read the stale projection back; a cast was ordered only by accident of the intervening wiring call. The invalidate-exactly-once-then-refetch path is now pinned by an end-to-end test.
jallum
force-pushed
the
feature/q67.9-client-read-path
branch
from
August 20, 2026 19:26
bd6e81c to
d97fc0e
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Third layer of the q67.9 stack, on #172/#173. The load-bearing layer: the client read path moves off the recovery-frozen TSL shard fields onto proxy-served routing — FDB's client location model, whole.
What: the Link becomes the node's locationCache (stores the raw projection, no TTL, dropped on every wiring push so new-epoch wiring never pairs with old-epoch routing). Routing is fetched lazily like the read version — the builder builds its LayoutIndex on first read through a
routing_fn(Link cache → fetch-through to a random proxy → cast back); a transaction that never reads never fetches. String refs become callable refs at the client (deterministic worker OTP names; node-atom conversion is the documented no-atoms-on-decode exception).The load-bearing piece (per the design audit): retry reclassification.
layout_lookup_failed/no_servers_to_race/version_too_old/:lockedjoin the retryable set (resolving the old get/get_range inconsistency), and retries invalidate the routing cache on routing-shaped failures only — version-window misses keep it. Staleness costs the user a retry, never an error, never a wrong answer.Also pins the fetch_routing cadence as direct handler assertions (the #173 audit's one test gap).
Full suite green (2,563), credo --strict + dialyzer clean.