Skip to content

Commit 95a3a54

Browse files
frristclaude
andcommitted
docs(s3-versioning): explain the lookup flow and field duplication up front
Review response for PR #49: add a §2.0 end-to-end lookup walkthrough (including what the token-verify guard protects against), document why VersionNode caches two manifest fields and why the manifest stays a separate block, and scope the §2.2 'id lives in the manifest' claim to prev-tree entries. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
1 parent 4247a04 commit 95a3a54

1 file changed

Lines changed: 60 additions & 3 deletions

File tree

docs/s3-versioning.md

Lines changed: 60 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -36,6 +36,41 @@ object keys.
3636

3737
## 2. Data model
3838

39+
### 2.0 How a lookup works
40+
41+
Three facts carry the design:
42+
43+
1. **The top-level bucket MST still maps plain object keys.** The value now points at a small
44+
per-key `ObjectLeaf` block (§2.1) instead of pointing straight at an `ObjectManifest`.
45+
2. **The leaf holds the current version inline.** A plain GET, HEAD, or list reads the leaf and
46+
is done: one block more than today, and history is never touched. Only when a key has
47+
noncurrent versions does the leaf also carry the root of a second, per-key MST that holds
48+
them (§2.2, the prev tree).
49+
3. **A numbered version id carries its own position.** The id is a ULID token whose low 64
50+
entropy bits hold the version's `seq` (§3). Parsing the token recovers the seq, and the seq
51+
is the lookup key in the prev tree.
52+
53+
Take one read end to end: `GET photos/cat.jpg?versionId=<token>`, against a key holding
54+
versions with seqs 9 (current), 7, 4, and 2, where the token names seq 7:
55+
56+
1. Read the key's leaf from the top MST.
57+
2. Parse the token; its low 64 bits give 7.
58+
3. `Current.Seq` is 9, so seek the prev tree directly: `prev.Get(revSeqKey(7))`. The seek is
59+
O(log n); nothing is scanned.
60+
4. Fetch the manifest that entry points at and confirm its stored `VersionID` equals the token
61+
(§6.1). This guard exists because step 2 trusts nothing: any well-formed ULID parses, so a
62+
client can send a token this bucket never minted (an id copied from another bucket, or
63+
fabricated) whose low 64 bits nonetheless equal the seq of a real version of this key. Step
64+
3 would then land on that version even though the token does not name it. Comparing the full
65+
26-character token against the manifest's stored `VersionID` catches the mismatch, and the
66+
read returns `NoSuchVersion` instead of serving a version the caller never asked for.
67+
5. Serve the body through the same manifest-to-blobs path a current read uses.
68+
69+
`versionId=null` is the one token that carries no position, because S3 fixes the null version's
70+
id to the literal string `"null"` (§1). The leaf's `NullSeq` field answers exactly that lookup:
71+
it names the null version's prev-tree slot while the null version is noncurrent (§6.1 has the
72+
full resolution rules).
73+
3974
### 2.1 The leaf
4075

4176
The top-level (per-bucket) MST continues to map **plain object keys** to CIDs. The value changes:
@@ -69,6 +104,20 @@ type ObjectLeaf struct {
69104
}
70105
```
71106

107+
**Why `VersionNode` repeats `Seq` and `VersionID` when the manifest also stores them (§2.3):**
108+
the copies let the leaf answer questions without fetching a manifest. The write rule decides
109+
whether a displaced current version is retained or discarded by reading `displaced.VersionID`
110+
(§5.2), and resolution short-circuits on `Current.Seq` / `Current.VersionID` (§6.1); both work
111+
from the leaf alone. Manifests are immutable, so the copies cannot drift: the manifest stays
112+
the authoritative record, and the `VersionNode` fields are a two-field cache of it.
113+
114+
**Why `Manifest` is a pointer rather than an inlined manifest:** the manifest CID is the
115+
version's stable identity (invariant 5, §2.4). Displacement pushes that same CID into the prev
116+
tree and promotion pulls it back; nothing is rewritten, and the reference index (§8) counts
117+
claims per version on the strength of that stability. An inlined manifest would have to be
118+
re-serialized as a standalone block when displaced, which mints a new CID, and it would put the
119+
full blob list on a block that every read and list fetches.
120+
72121
### 2.2 The prev tree
73122

74123
Noncurrent versions live in a **per-key sub-MST** (the same forked `mst` package, same space, same
@@ -79,9 +128,11 @@ blockstore — its nodes are ordinary catalog-plane blocks):
79128
iteration **newest-first**. Scoped to one object key, the key is *just an integer rendered
80129
order-preservingly* — no composite-string escaping. Valid under `mst.IsValidKey` (hex is
81130
UTF-8, NUL-free, short).
82-
- **value** — the version's `ObjectManifest` CID. `Seq` is recoverable from the key;
83-
`VersionID` comes from the manifest (§2.3), which every consumer of a prev entry fetches
84-
anyway (list rendering needs ETag/size/mtime; version-scoped reads need the body).
131+
- **value** — the version's `ObjectManifest` CID and nothing else: a prev entry stores no
132+
`VersionNode`. For these entries the seq is recoverable from the MST key, and the version id
133+
lives only in the manifest (§2.3). That costs nothing in practice, because every consumer of
134+
a prev entry fetches the manifest anyway (list rendering needs ETag/size/mtime;
135+
version-scoped reads need the body).
85136

86137
A direct version-scoped seek is `prev.Get(revSeqKey(seq))` — O(log n), no scan.
87138

@@ -95,6 +146,12 @@ Seq uint64 `cborgen:"sq"` // the version's ordinal (== leaf/prev position)
95146
VersionID string `cborgen:"vi"` // "null" or the ULID token; "" only in pre-versioning blocks
96147
```
97148

149+
Both fields also exist on the leaf's `VersionNode` (§2.1 explains the split). The manifest is
150+
the self-contained record: a prev entry stores only the manifest CID (§2.2), so for noncurrent
151+
versions the manifest is the only place the id is stored. List rendering reads it there, the
152+
token check in §6.1 verifies against it, and promotion (§7.2) rebuilds the leaf's `Current`
153+
node from these two fields.
154+
98155
A **delete marker** is an `ObjectManifest` with `DeleteMarker: true`, a zero `Body` (no blobs, no
99156
digests — it contributes nothing to `blob_refs`), no ETag, and `Created` = marker time. Markers
100157
are versions like any other: they occupy `Current` or a `Prev` slot and carry a `Seq`/`VersionID`.

0 commit comments

Comments
 (0)