Skip to content

CONTRACT.md

Latest commit

 

History

History
107 lines (77 loc) · 4.89 KB

CONTRACT.md

File metadata and controls

107 lines (77 loc) · 4.89 KB
Raw

The memory contract

Memory is a contract, not a hope.

Every other variant caller treats RAM as an emergent property you guess at (-Xmx…, --target-mem "heuristics may not work well") and then crash on. Rosalind treats it as a contract:

Rosalind never silently OOM-kills you — it fits, or it tells you up front, and it proves the realized peak with a receipt.

(It does not claim to "never refuse": when a budget is genuinely too small the run declines cleanly rather than crashing. Graceful degrade-don't-die — sliding down a space/time curve to finish anyway — is the Phase-D research direction, not a present claim.)

The four verbs

The contract applies to the bounded whole-genome germline path, rosalind variants --index.

1. Declare

State the RAM you have. --memory-budget-mb N on variants, --budget-mb N on plan/verify.

2. Predict — rosalind plan

Ask before committing a byte whether the job fits. plan reads only the index header (plus your declared depth/read-length assumptions) — it never opens the BAM:

rosalind plan --index genome.idx --max-depth 1000 --max-read-len 250 --budget-mb 2048

It prints a breakdown — reference decode + active set @ max-depth + engine overhead, atop a measured process baseline — and a verdict: [FITS] or [REFUSE].

3. Honor — rosalind variants … --enforce

rosalind variants --index genome.idx --alignments sample.sorted.bam \
  --memory-budget-mb 2048 --enforce -o sample.vcf

With --enforce:

  • predicted peak > budget → refuse up front (exit 3), before doing any work, with an actionable message (raise the budget, lower --max-depth, or drop --enforce);
  • realized peak > budget → fail loud (exit 4) after writing the VCF + receipt (you keep the data and the proof it overran) — never a silent overrun;
  • otherwise the run completes within budget.

Without --enforce, the budget is record-only: the run always completes and the verdict is recorded in the receipt. The active read set is capped at --max-depth (default 1000; 0 = uncapped) — deterministic downsampling that bounds the working set; output changes only at sites deeper than the cap.

4. Verify — rosalind verify

Re-check a receipt without re-running:

rosalind verify --manifest sample.vcf.manifest.json

It re-hashes the recorded inputs and outputs (BLAKE3) and re-checks the recorded peak against the budget (supplied via --budget-mb, or read from the manifest). Exit 0 if everything matches and fits; non-zero (exit 5) with a per-check report on any drift, missing file, or over-budget peak. This is the auditability story containers can't give you for a non-deterministic caller.

What's bounded (honest scope)

  • Germline variants --index is the bounded path: peak ≈ the largest contig's reference + the depth-capped active set, independent of BAM size. Reads stream one record at a time.
  • Somatic (somatic) is region-bounded, not whole-genome-bounded (it collects both pileup streams for the region).
  • Index build (rosalind index) is O(reference) in RAM today; plan --reference reports an advisory estimate. Sublinear-space construction is the Phase-D research direction (see docs/OPEN_PROBLEMS.md).
  • The engine is single-threaded — outputs are deterministic, but there is no thread-invariance claim yet.

Extend — build on the bounded substrate

Rosalind's kernel is a bounded, deterministic PileupColumn stream. Compute your own per-locus analytics over it (coverage, QC, methylation, ML features) and inherit bounded memory + determinism for free — no variant calling required:

use rosalind::{PileupEngine, PileupParams};
// PileupEngine<S: ReadSource> is an Iterator<Item = Result<PileupColumn, _>>.
// Each PileupColumn carries the locus, ref_base, depth(), allele_counts(), strand_counts().
for column in PileupEngine::new(source, reference, contig, region, PileupParams::default()) {
    let col = column?;
    // your bounded per-locus metric here
}

A complete, runnable example: examples/custom_pileup_analytics.rs (cargo run --example custom_pileup_analytics).

Legacy / non-bounded. The GenomicPlugin trait (src/plugin/), the framework/ evaluator, and the Python run_rna_seq_plugin demo still work but do not inherit the memory contract. Prefer the PileupColumn substrate above for bounded work.

Reproducibility

Every receipt is canonical JSON (sorted keys, no timestamps) with BLAKE3 content hashes of the index, the alignments, and the output VCF, plus the realized peak_rss_bytes / max_working_set_bytes and the contract params (memory_budget_mb, contract_verdict, enforced, max_depth, max_read_len). Identical inputs produce a byte-identical VCF and a byte-identical manifest — and rosalind verify proves it.