fix(contract): predicted peak RSS is a true upper bound on real genomes (Act-1 PR-A) - #33
Merged
Conversation
…C scope Derived from the 60-agent reflection audit. PR-A makes the predicted peak a true upper bound on real genomes: kill the Arc::from reference-decode transient, record the prediction in the receipt, add an honest RSS margin, and add the missing real peak-RSS soundness test. PR-B (real-genome correctness) and PR-C (trust on-ramp + hygiene) scoped for follow-on. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The predicted-peak contract held on the synthetic/single-contig demo path but could be breached at runtime on a real multi-contig reference: the per-contig `Arc::from(decoded)` reallocation briefly held TWO ASCII-decoded copies of the largest contig, while both the estimator (call/plan.rs) and the realized accountant (pileup/engine.rs) counted the reference exactly once. A `[FITS]` plan could therefore breach, caught only post-hoc by the exit-4 check — and no test compared predicted *peak RSS* to realized *peak RSS* (the working-set tests are blind to the transient by construction). This makes the up-front claim sound: - decode_window_arc: decode straight into the Arc (TrustedLen collect, no intermediate Vec) — one resident reference copy, not the transient two. Used by both the germline and features whole-genome drivers. - record predicted_peak_rss_bytes in the receipt, computed unconditionally (also what the Act-2 plan --fleet scheduler needs). - PILEUP_IO_RSS_OVERHEAD: an honest RSS-level margin (htslib/BGZF buffers + BufWriter + allocator slack) added to the prediction only — the working-set estimate stays comparable to the realized accountant. - the missing soundness test: predicted peak RSS >= realized peak RSS on a 4 MiB-contig shallow-coverage run (RED before the fix by ~3.4 MiB; the transient drops realized 22.5 -> 15.4 MiB, headroom +10.6 MiB after). Byte-identical calls (decode_window_arc == decode_window unit-tested; golden VCF + whole-genome-equals-per-contig pin output). Full suite green, rustc 0 warnings debug+release. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This was referenced Jun 2, 2026
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Act-1 hardening, PR-A (soundness keystone). The predicted-peak memory contract held on the synthetic/single-contig demo path but could be breached at runtime on a real multi-contig reference — the highest-severity finding from the reflection audit.
The gap (mechanism confirmed + quantified):
call_germline_whole_genomedecoded each contig withArc::from(decoded), which reallocates — two ASCII-decoded copies of the largest contig briefly co-resident. But both the estimator (call/plan.rs) and the realized accountant (pileup/engine.rs) counted the reference once. So a[FITS]plan + the pre-run exit-3 gate could pass, then the run spikes past budget — caught only after the fact by the exit-4 check, i.e. after the allocation the contract promises to prevent up front. And no test compared predicted peak RSS to realized peak RSS (the existing soundness test is working-set-vs-working-set, both modeling the reference once → blind to the transient by construction).What changed
decode_window_arc— decode straight into theArc<[u8]>via aTrustedLencollect (no intermediateVec), so peak resident reference memory is one copy, not the transient two. Used by both the germline and features whole-genome drivers. Byte-identical todecode_window(unit-tested).predicted_peak_rss_bytesin the receipt, computed unconditionally — the contract's up-front claim, now carried in the manifest (also what the Act-2plan --fleetscheduler needs).PILEUP_IO_RSS_OVERHEAD— an honest RSS-level margin (htslib/BGZF buffers +BufWriter+ allocator slack) added to the prediction only; the working-set estimate stays comparable to the realized accountant. The transient is eliminated at the source, so this margin covers fixed I/O buffers + slack, not a per-contig reference copy (so it need not scale with the genome).predicted_peak_rss_upper_bounds_realized_peak: a 4 MiB-contig, shallow-coverage run asserting predicted peak RSS ≥ realized peak RSS, read from the receipt.Measured (the TDD red→green)
The realized peak dropping by ~one contig copy confirms
decode_window_arceliminates the transient at scale — so the contract is sound on chr1-sized references by elimination, not by a fixed margin trying to cover a 237 MiB transient.Test plan
plan_enforce11/11decode_window_arc==decode_windowbyte-equivalence (unit)whole_genome_equals_per_contig_callsgreencargo fmtcleanplan/--enforce/verifyre-run on the bundled toy fixture; receipt carriespredicted_peak_rss_bytesAct-1 sequence
This is PR-A of 3. Next: PR-B (real-genome correctness — multi-contig
eval, contig-naming guard, IUPAC→N) and PR-C (trust on-ramp + hygiene — Action snippet,install.shchecksum, CI tautology,verifyself-hash, clippy gate). Plan:docs/superpowers/plans/2026-06-02-act1-contract-hardening.md.🤖 Generated with Claude Code