Phase C — memory as a verifiable contract + the front door (Move #4) - #21
Merged
Conversation
Design for the Phase-C contract increment on the bounded `variants --index` path: declare → predict → honor-or-refuse → verify. Decomposed C1 (working-set soundness fix + deterministic depth cap + per-call VCF row flush + decode-then- move reference) → C2 (`rosalind plan` + `--enforce`) → C3 (`rosalind verify` + receipt-on-stdout + CI contract gate). Index build stays record-only; somatic and graceful-degrade/spill out of scope (Phase D/E). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
8-task TDD plan for the C1 soundness core: corrected current_working_set (reference + per-read seq/qual), deterministic max_depth cap (mechanism, default off), incremental VCF writer (header + row split), streaming sink on call_germline_whole_genome + decode-then-move reference, and the main.rs wiring. Output-preserving by default; the soundness proof is a library-level bounded- working-set test (process-RSS CI gate deferred to C3). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…/qual (C1 soundness core)
…e_vcf wraps them (C1)
…; tracked wraps it (C1)
…ove reference (C1)
6-task TDD plan: shared pileup cost consts (core/budget.rs) used by both the C1 accountant and a new pure estimator (call/plan.rs); `rosalind plan` (--index variants peak / --reference build peak); --max-depth (default 1000) / --max-read-len / --enforce on variants --index with exit 3 (refuse pre-run) / 4 (fail post-run). Resolves the spec's working-set-vs-RSS gap by measuring the process baseline at call time (no guessed constant); post-run peak_rss is the backstop. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
… accountant) (C2)
…t-run (4); --max-depth default 1000 (C2)
…plan 5-task TDD plan: always-write a self-describing receipt (+ --manifest; new contract_verdict/enforced/budget/max_depth/max_read_len params); a hand-parser RunManifest::from_canonical_json (round-trip property-tested, no serde_json); rosalind verify (re-hash inputs/outputs + re-check peak vs budget, exit 5 on mismatch); and a deterministic CI gate that the pure estimator upper-bounds the realized working set. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…+ --manifest) (C3)
…hout re-running (C3)
…on (C3) Adds a shared unique_dir(atomic counter + nanos) so concurrent plan_enforce tests never share a directory, and routes the generous-budget enforce test's receipt to its temp dir instead of the cwd-default sidecar. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…a notice (C3) Revises the spec §7.1 cwd-sidecar default: silently dropping rosalind.variants. manifest.json into the caller's cwd pollutes pipe users' dirs and races on a fixed filename across concurrent stdout runs. Instead, persist a receipt only when there is a destination (--manifest or a -o sidecar); a stdout run without --manifest prints how to get one (honest, not silent; no pollution, no race). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Converts the watching fork wave into building forkers: crate-root re-exports of the genomics product surface, a contract-first lib.rs/cargo-doc rewrite (√t as an honest research footer), CONTRACT.md, a README rewrite routing to the bounded substrate (plugin lineage demoted+labeled, not removed), a multi-contig demo fixture, a PileupColumn-iterator cookbook example, and an issue-3 reframe (confirm-first). Pure positioning/docs/re-exports/examples — no behavior changes. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
7-task plan: crate-root genomics re-exports + contract-first lib.rs rustdoc (runnable doctest, √t demoted) + pileup docline fix; CONTRACT.md; README rewrite (contract-first lead, substrate-first Extend, honest brand); PileupColumn cookbook example; an end-to-end smoke test of the README in-house demo; full verification; and a confirm-first issue-3 reframe. §3.5 refined: runnable demo is single-contig in-house (aligner is single-contig), multi-contig flagship documented as a command. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…brand; in-house demo (Move #4)
…over the substrate (Move #4)
Owner
Author
|
Folded in Move #4 — the front door (8 commits on top of Phase C), so this PR is now the complete "shipped contract + discoverable front door" story:
Public roadmap issue #3 reframed to match (contract-first; √t as the future Phase-D knob). Full suite green (27 sections), 0 warnings (debug + release), |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Turns the existing memory receipt into a contract on the bounded
variants --indexpath — declare → predict → honor-or-refuse → verify — the property no incumbent caller offers. Decomposed into three sub-stages (one design doc + three plans underdocs/superpowers/), each landing green.What changed
C1 — working-set soundness (the hard correctness core).
PileupEngine::current_working_set()now counts the resident reference + each active read'sseq/qualbuffers (it previously counted only the projection map — a real under-count of the dominant terms). Adds a deterministic--max-depthcap; streams each call to the VCF writer (no genome-wide row buffer); decodes-then-moves each contig reference (2×→1×). Proven by a library test that the reported working set is a true upper bound and is flat as read count grows.C2 —
rosalind plan+--enforce. A pure estimator (src/call/plan.rs) sharing C1's cost constants so the prediction and the realized accountant cannot drift.planpredicts the variants peak (--index) or the build (--reference).--enforcerefuses up front (exit 3) when the predicted peak won't fit and fails loud post-run (exit 4) if the realized peak does; record-only behavior is preserved without--enforce. The predicted peak = a process baseline measured at call time + the estimated working set, so it is comparable to the realizedpeak_rssthe post-run check uses (no guessed baseline constant).C3 —
rosalind verify+ self-describing receipts. Every receipt now carriescontract_verdict/enforced/memory_budget_mb/max_depth/max_read_len. A small canonical-JSON hand-parser (RunManifest::from_canonical_json, round-trip property-tested, noserde_json) backsrosalind verify, which re-hashes the recorded inputs/outputs and re-checks the recorded peak vs the budget without re-running (exit 5 on mismatch). A stdout run without--manifestwrites no file but prints how to persist one (avoids polluting a pipe user's cwd / racing on a fixed filename).Test plan
cargo fmt --all -- --checkclean,cargo build0 warnings (debug and release).tests/plan_enforce.rs:planbreakdown,--enforcerefuse (exit 3), within-budget pass, stdout receipt,verifyround-trip + tamper (exit 5), and the estimator-upper-bounds-realized contract gate. Newprovenanceround-trip parser test (escapes + multibyte).Non-goals (kept out by design)
Index-build enforcement (build is O(reference); Phase D), somatic whole-genome bounding, graceful degrade/spill (Phase D √t ladder), and thread-invariance claims (engine is single-threaded). MSRV 1.72 preserved; no new dependencies.
🤖 Generated with Claude Code