Skip to content

Phase C — memory as a verifiable contract + the front door (Move #4) - #21

Merged
logannye merged 32 commits into
mainfrom
rosalind/phase-c-contract
Jun 2, 2026
Merged

Phase C — memory as a verifiable contract + the front door (Move #4)#21
logannye merged 32 commits into
mainfrom
rosalind/phase-c-contract

Conversation

@logannye

@logannye logannye commented Jun 2, 2026

Copy link
Copy Markdown
Owner

Summary

Turns the existing memory receipt into a contract on the bounded variants --index path — declare → predict → honor-or-refuse → verify — the property no incumbent caller offers. Decomposed into three sub-stages (one design doc + three plans under docs/superpowers/), each landing green.

rosalind plan    --index g.idx --max-depth 1000 --budget-mb 2048   # predict before committing a byte
rosalind variants --index g.idx --alignments s.bam \
                 --memory-budget-mb 2048 --enforce -o s.vcf        # honor: refuse(3) / fail(4), never silent OOM
rosalind verify  --manifest s.vcf.manifest.json                    # re-check the receipt, no re-run

What changed

C1 — working-set soundness (the hard correctness core). PileupEngine::current_working_set() now counts the resident reference + each active read's seq/qual buffers (it previously counted only the projection map — a real under-count of the dominant terms). Adds a deterministic --max-depth cap; streams each call to the VCF writer (no genome-wide row buffer); decodes-then-moves each contig reference (2×→1×). Proven by a library test that the reported working set is a true upper bound and is flat as read count grows.

C2 — rosalind plan + --enforce. A pure estimator (src/call/plan.rs) sharing C1's cost constants so the prediction and the realized accountant cannot drift. plan predicts the variants peak (--index) or the build (--reference). --enforce refuses up front (exit 3) when the predicted peak won't fit and fails loud post-run (exit 4) if the realized peak does; record-only behavior is preserved without --enforce. The predicted peak = a process baseline measured at call time + the estimated working set, so it is comparable to the realized peak_rss the post-run check uses (no guessed baseline constant).

C3 — rosalind verify + self-describing receipts. Every receipt now carries contract_verdict/enforced/memory_budget_mb/max_depth/max_read_len. A small canonical-JSON hand-parser (RunManifest::from_canonical_json, round-trip property-tested, no serde_json) backs rosalind verify, which re-hashes the recorded inputs/outputs and re-checks the recorded peak vs the budget without re-running (exit 5 on mismatch). A stdout run without --manifest writes no file but prints how to persist one (avoids polluting a pipe user's cwd / racing on a fixed filename).

Test plan

  • Full suite green (26 sections), cargo fmt --all -- --check clean, cargo build 0 warnings (debug and release).
  • Determinism + golden-VCF snapshots preserved (the writer split + per-call streaming are byte-identical).
  • New gates in tests/plan_enforce.rs: plan breakdown, --enforce refuse (exit 3), within-budget pass, stdout receipt, verify round-trip + tamper (exit 5), and the estimator-upper-bounds-realized contract gate. New provenance round-trip parser test (escapes + multibyte).

Non-goals (kept out by design)

Index-build enforcement (build is O(reference); Phase D), somatic whole-genome bounding, graceful degrade/spill (Phase D √t ladder), and thread-invariance claims (engine is single-threaded). MSRV 1.72 preserved; no new dependencies.

🤖 Generated with Claude Code

logannye and others added 30 commits June 1, 2026 16:43
Design for the Phase-C contract increment on the bounded `variants --index`
path: declare → predict → honor-or-refuse → verify. Decomposed C1 (working-set
soundness fix + deterministic depth cap + per-call VCF row flush + decode-then-
move reference) → C2 (`rosalind plan` + `--enforce`) → C3 (`rosalind verify` +
receipt-on-stdout + CI contract gate). Index build stays record-only; somatic
and graceful-degrade/spill out of scope (Phase D/E).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
8-task TDD plan for the C1 soundness core: corrected current_working_set
(reference + per-read seq/qual), deterministic max_depth cap (mechanism, default
off), incremental VCF writer (header + row split), streaming sink on
call_germline_whole_genome + decode-then-move reference, and the main.rs wiring.
Output-preserving by default; the soundness proof is a library-level bounded-
working-set test (process-RSS CI gate deferred to C3).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
6-task TDD plan: shared pileup cost consts (core/budget.rs) used by both the C1
accountant and a new pure estimator (call/plan.rs); `rosalind plan` (--index
variants peak / --reference build peak); --max-depth (default 1000) / --max-read-len
/ --enforce on variants --index with exit 3 (refuse pre-run) / 4 (fail post-run).
Resolves the spec's working-set-vs-RSS gap by measuring the process baseline at
call time (no guessed constant); post-run peak_rss is the backstop.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…plan

5-task TDD plan: always-write a self-describing receipt (+ --manifest; new
contract_verdict/enforced/budget/max_depth/max_read_len params); a hand-parser
RunManifest::from_canonical_json (round-trip property-tested, no serde_json);
rosalind verify (re-hash inputs/outputs + re-check peak vs budget, exit 5 on
mismatch); and a deterministic CI gate that the pure estimator upper-bounds the
realized working set.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…on (C3)

Adds a shared unique_dir(atomic counter + nanos) so concurrent plan_enforce
tests never share a directory, and routes the generous-budget enforce test's
receipt to its temp dir instead of the cwd-default sidecar.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…a notice (C3)

Revises the spec §7.1 cwd-sidecar default: silently dropping rosalind.variants.
manifest.json into the caller's cwd pollutes pipe users' dirs and races on a
fixed filename across concurrent stdout runs. Instead, persist a receipt only
when there is a destination (--manifest or a -o sidecar); a stdout run without
--manifest prints how to get one (honest, not silent; no pollution, no race).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Converts the watching fork wave into building forkers: crate-root re-exports of
the genomics product surface, a contract-first lib.rs/cargo-doc rewrite (√t as an
honest research footer), CONTRACT.md, a README rewrite routing to the bounded
substrate (plugin lineage demoted+labeled, not removed), a multi-contig demo
fixture, a PileupColumn-iterator cookbook example, and an issue-3 reframe
(confirm-first). Pure positioning/docs/re-exports/examples — no behavior changes.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
7-task plan: crate-root genomics re-exports + contract-first lib.rs rustdoc
(runnable doctest, √t demoted) + pileup docline fix; CONTRACT.md; README rewrite
(contract-first lead, substrate-first Extend, honest brand); PileupColumn cookbook
example; an end-to-end smoke test of the README in-house demo; full verification;
and a confirm-first issue-3 reframe. §3.5 refined: runnable demo is single-contig
in-house (aligner is single-contig), multi-contig flagship documented as a command.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@logannye logannye changed the title Phase C — memory as a verifiable contract (variants path) Phase C — memory as a verifiable contract + the front door (Move #4) Jun 2, 2026
@logannye

logannye commented Jun 2, 2026

Copy link
Copy Markdown
Owner Author

Folded in Move #4 — the front door (8 commits on top of Phase C), so this PR is now the complete "shipped contract + discoverable front door" story:

  • Crate-root re-exportsuse rosalind::{PileupEngine, PileupColumn, ReadSource, MemoryBudget, GenomeIndex, RunManifest, …} now lands on the genomics product surface; the √t theory types are regrouped under a "research layer" comment.
  • lib.rs / cargo doc rewritten contract-first with a runnable doctest (was "O(√t) Space Simulation"); √t demoted to an honest "Research direction (Phase D)" footer.
  • CONTRACT.md (the four verbs + the honest "never silently OOM-kills" brand) and a contract-first README (substrate-first Extend; the GenomicPlugin/Python lineage labeled legacy/non-bounded, not removed; stale "does not yet enforce" claims fixed; a runnable bundled in-house demo).
  • examples/custom_pileup_analytics.rs — a non-caller PileupColumn-iterator cookbook.
  • tests/frontdoor_demo.rs — smoke-tests the README's in-house demo end-to-end on the bundled fixture.

Public roadmap issue #3 reframed to match (contract-first; √t as the future Phase-D knob). Full suite green (27 sections), 0 warnings (debug + release), fmt clean.

@logannye
logannye merged commit 5164724 into main Jun 2, 2026
2 of 3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant