Branched databases: blob store sharing, allocator, and GC safety

[kriszyp]

Summary

Wire blob handling for branched databases (#642). Records in a branch's checkpointed RocksDB reference blob file IDs that resolve under the base's blob directory, so branches must share the base's blob store and ID allocator — and must avoid corrupting the base by deleting blobs the base still references.

Background

Today (per `core/resources/blob.ts`):

• Blobs live at `$HARPER_HOME/blobs//` (or under `STORAGE_BLOBPATHS`), keyed by a 64-bit `fileId`. Records reference blobs as an `(storageIndex, fileId)` pair — no path embedded. (`blob.ts:681–699`, `:1080–1136`)
• The allocator is a per-database monotonic counter held in a shared `BigInt64Array` registered as `'blob-file-id'` on the root store; bumped atomically (`blob.ts:775–815`).
• Blobs are immutable on disk; deleted only when a record updates (sweep via `deleteBlobsInObject`, `blob.ts:614`) or by orphan GC (`cleanupOrphans`, `blob.ts:1218–1337`).
• Replication sends blob content, not IDs; receiver assigns a fresh local ID. IDs are not cluster-stable. (`replication/replicationConnection.ts:1828–1920`)

A naive checkpoint-based branch would have its own `blobs//` directory and its own allocator, breaking every existing blob reference in the cloned records.

Requirements

1. Shared blob directory

The branch's blob path resolver must key off the base dbName, not the branch's namespaced name. `STORAGE_BLOBPATHS` multi-path handling carries through unchanged. The branch reads and writes into the same directory tree as the base.

2. Shared ID allocator

The branch's blob writer must increment the base's `'blob-file-id'` shared `BigInt64Array` (registered on the base's root store), not a branch-local one. Atomics already handle concurrent allocation; the only change is sourcing the SAB from the base. This guarantees that base writes, branch-A writes, and branch-B writes never assign the same ID into the shared directory.

3. High-water mark (HWM)

At branch creation, record the base allocator's current value as `hwm` on the `ApplicationScope.branches` entry. Any blob with `fileId ≤ hwm` is base-origin and existed at checkpoint time. Any `fileId > hwm` referenced by a branch record was allocated by that branch.

4. Gated deletion in branches

When a branch sweeps an old blob during a record update (`deleteBlobsInObject`, `blob.ts:614`, and any other sweep call sites on the branch write path):

• If `fileId ≤ hwm` — no-op. The base (or other branches) may still reference it.
• If `fileId > hwm` — normal delete is safe; only the branch can reference IDs it allocated.

5. Orphan-GC safety

`cleanupOrphans` (`blob.ts:1218–1337`) reaps any blob not referenced by the database's records. Branch-allocated blobs (ID > hwm) are by definition not referenced by base records, so a base GC pass would delete them out from under the branch.

Simplest correct behavior: suspend base orphan GC while any branch of that base is alive. Track live branches via a small registry on the base's root store. On branch teardown the branch's blobs become true orphans and the next GC pass sweeps them.

A future optimization could union the referenced-set across base + all live branches instead of suspending GC, but is not required for the QA use case.

6. Replication

Out of scope for the branch itself (branches are excluded from replication per the umbrella). Base replication continues normally. Branch-allocated blobs never leave the local instance.

Acceptance criteria

• Branch can read all blobs that existed in the base at checkpoint time.
• Branch write that creates a new blob produces a file with a unique ID across base + all live branches (no collisions in the shared directory).
• Branch update that overwrites a blob reference: file with `fileId ≤ hwm` is not removed from disk; file with `fileId > hwm` originating from this branch is removed.
• While at least one branch of base `X` is alive, `cleanupOrphans` on `X` is suspended (or otherwise made branch-aware).
• After all branches of `X` close, the next `cleanupOrphans` pass reclaims their leftover blobs.
• Replication: branch writes do not propagate to peers (covered by umbrella).

Verification

Add to the integration test from #643:

1. Base `data` contains a row whose value includes a blob (e.g. 16 KB payload, above the inline threshold).
2. Two branched apps `appA`, `appB` boot.
3. Both apps read the row; both successfully stream the blob content.
4. `appA` overwrites the row with a new blob — assert the original base blob file still exists on disk; the new blob has `fileId > hwm_A` and lives in the shared directory.
5. `appB` reads the original row again — still sees the original blob, unchanged.
6. The base reads the original row — still sees the original blob, unchanged.
7. Trigger base `cleanupOrphans` while `appA` is alive — assert it is suspended / does not delete `appA`'s new blob.
8. Close `appA`; run `cleanupOrphans` — assert `appA`'s leftover blob is reclaimed.

Key files

• `core/resources/blob.ts` lines 614, 681–699, 775–815, 1080–1136, 1218–1337
• `core/components/ApplicationScope.ts` — extend `branches` entry: `{ branchName: string; hwm: bigint }`
• `core/components/Application.ts` — capture HWM at branch creation

🤖 Filed by Claude on behalf of Kris