Public anchor for the migration-safety engineering standards that govern flair's zero-touch migrations (#690) and their CI enforcement (#692), so code comments and PRs can reference a public source.
The four invariants (every migration, every release)
I. Source data is never modified in place. Derived data (embeddings, indexes, stamps) is recomputable and may be rewritten freely. Schema changes are additive-first: new fields land nullable, old fields keep working; destructive contraction ships no earlier than the release AFTER the expansion proved out. Content transforms write new-alongside-old and flip a read pointer (supersession) — never in-place rewrites. Config files are state too (#694 taught us this live): persisted config keys follow the same discipline — the previous release must always be able to boot.
II. Halt, don't brick — and no auto-rollback. A failed migration's worst case is: still serving on the pre-migration shape, halted loudly, doctor explaining the exact remedy. Rollback logic that rewrites data is itself a migration that can fail, so it doesn't exist; downgrade = reinstall the old binary, which MUST boot against a partially-migrated store (enforced by the downgrade-and-revert CI lane, which catches violations before any user can — see #694).
III. Snapshot before mutate — and restore is a drilled path. The runner takes an automatic risk-scoped pre-flight snapshot (no operator flag); flair snapshot create/restore is the byte-exact content-recovery floor; a scheduled plain-file export means memories can always leave the system intact. CI drills the restore path (snapshot → migrate → corrupt → restore → verify) — a backup that has never been restored is a prayer.
IV. Idempotent, resumable, observable — and verified before declaring success. Per-row progress markers live in the data (no journal to corrupt; crash-resume is free). Completion requires an integrity envelope: source-field content hashes computed before the first write and after the last must match. Every migration records a ledger event (structural metadata only — never memory content) and reports progress via health.
The CI lanes that enforce them (#692)
- downgrade-and-revert — proof of invariant I/II: old release → seed → new build mid-migration kill → old release must boot and serve byte-identically.
- snapshot-restore drill — proof of invariant III.
- upgrade-lane assertions — auto-migration ran,
hashEnvelopeMatch === true, recall parity on migrated rows.
- Risk-class governs batch size, snapshot scope, and completion-gate strictness mechanically.
🤖 Generated with Claude Code
§A — the boot-keyed runner (design summary)
A MigrationRegistry where each migration declares: id, cheap read-only detect(), risk class, idempotent run(batch), and a per-row progress marker. Boot serves FIRST on the old shape; the pre-flight integrity hash runs async after ready; the migration is deferred until it completes. Pre-flight ladder: space check (with index-rebuild overhead) → snapshot prune → risk-scoped snapshot → content-only export fallback → halt with an exact actionable reason. Single-flight via in-process mutex + stale-tolerant file lock. Snapshot/export dirs are created 0700 and stat-verified at creation and before writes.
§B — the CLI↔server version handshake
The server reports its running version via health; the CLI compares CLI / installed-package / running-server versions on invocation (cached ~60s per instance, 1–2s timeout, never blocks or fails the underlying command) and prints a one-line restart nudge on mismatch. flair doctor shows the version triple plus migration state; doctor --fix offers the restart. This is what rescues the bare-npm install -g user who never runs a built-in upgrade command.
Space-pressure ladder (invariant III detail)
- Pre-flight estimate (snapshot + migration working set + index rebuild) against a ~90% disk headroom floor — a full disk is worse than any deferred migration. 2. Prune old snapshots (keep-last-3 / 30-day). 3. Snapshot scope proportional to risk class (derived-only → metadata; schema-additive → schema+metadata; content transforms retain old rows in-store via supersession). 4. Content-only plain-file export fallback when the physical snapshot cannot fit (embeddings dominate size; the irreplaceable content exports small). 5. No safe path → halt-don't-brick, with the exact bytes needed and remedies named. Mid-run disk exhaustion is a clean resumable halt.
Public anchor for the migration-safety engineering standards that govern flair's zero-touch migrations (#690) and their CI enforcement (#692), so code comments and PRs can reference a public source.
The four invariants (every migration, every release)
I. Source data is never modified in place. Derived data (embeddings, indexes, stamps) is recomputable and may be rewritten freely. Schema changes are additive-first: new fields land nullable, old fields keep working; destructive contraction ships no earlier than the release AFTER the expansion proved out. Content transforms write new-alongside-old and flip a read pointer (supersession) — never in-place rewrites. Config files are state too (#694 taught us this live): persisted config keys follow the same discipline — the previous release must always be able to boot.
II. Halt, don't brick — and no auto-rollback. A failed migration's worst case is: still serving on the pre-migration shape, halted loudly,
doctorexplaining the exact remedy. Rollback logic that rewrites data is itself a migration that can fail, so it doesn't exist; downgrade = reinstall the old binary, which MUST boot against a partially-migrated store (enforced by the downgrade-and-revert CI lane, which catches violations before any user can — see #694).III. Snapshot before mutate — and restore is a drilled path. The runner takes an automatic risk-scoped pre-flight snapshot (no operator flag);
flair snapshot create/restoreis the byte-exact content-recovery floor; a scheduled plain-file export means memories can always leave the system intact. CI drills the restore path (snapshot → migrate → corrupt → restore → verify) — a backup that has never been restored is a prayer.IV. Idempotent, resumable, observable — and verified before declaring success. Per-row progress markers live in the data (no journal to corrupt; crash-resume is free). Completion requires an integrity envelope: source-field content hashes computed before the first write and after the last must match. Every migration records a ledger event (structural metadata only — never memory content) and reports progress via
health.The CI lanes that enforce them (#692)
hashEnvelopeMatch === true, recall parity on migrated rows.🤖 Generated with Claude Code
§A — the boot-keyed runner (design summary)
A
MigrationRegistrywhere each migration declares: id, cheap read-onlydetect(), risk class, idempotentrun(batch), and a per-row progress marker. Boot serves FIRST on the old shape; the pre-flight integrity hash runs async after ready; the migration is deferred until it completes. Pre-flight ladder: space check (with index-rebuild overhead) → snapshot prune → risk-scoped snapshot → content-only export fallback → halt with an exact actionable reason. Single-flight via in-process mutex + stale-tolerant file lock. Snapshot/export dirs are created 0700 and stat-verified at creation and before writes.§B — the CLI↔server version handshake
The server reports its running version via health; the CLI compares CLI / installed-package / running-server versions on invocation (cached ~60s per instance, 1–2s timeout, never blocks or fails the underlying command) and prints a one-line restart nudge on mismatch.
flair doctorshows the version triple plus migration state;doctor --fixoffers the restart. This is what rescues the bare-npm install -guser who never runs a built-in upgrade command.Space-pressure ladder (invariant III detail)