Skip to content

Commit 3c4fc1c

Browse files
authored
Merge pull request #37 from logannye/rosalind/act2-columnkit
feat(columnkit): implement one trait, inherit the bounded contract (SDK)
2 parents 7a60446 + 8be8491 commit 3c4fc1c

7 files changed

Lines changed: 419 additions & 22 deletions

File tree

README.md

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -168,6 +168,27 @@ Two properties no other pileup gives you together: it is **bounded** (the whole-
168168

169169
A dependency-light Python boundary ([`python/rosalind.py`](python/rosalind.py), stdlib + numpy) loads the table directly, and [`examples/reproducible_features_demo.py`](examples/reproducible_features_demo.py) trains a small model on it and *proves* the inputs are bit-reproducible (two independent extractions → matching receipt hashes → bit-identical trained weights; see [`docs/findings/2026-06-02-reproducible-features-demo.md`](docs/findings/2026-06-02-reproducible-features-demo.md)). *(TSV today; an Arrow/Parquet egress and a zero-copy `pyarrow` in-process binding are the next step.)*
170170

171+
## ColumnKit: implement one trait, inherit the contract
172+
173+
Want your *own* per-locus metric — methylation, a coverage/QC track, custom ML features, a star-allele genotyper? Implement one trait and run it through the driver; you inherit the **same** bounded whole-genome walk, the **same** working-set bound that `plan`/`--enforce` admit, and the **same** verifiable receipt the shipped subcommands enjoy — without re-deriving any of it.
174+
175+
```rust
176+
use rosalind::{ColumnAnalyzer, run_bounded_whole_genome, PileupColumn};
177+
use std::io::{self, Write};
178+
179+
struct CoverageTrack;
180+
impl ColumnAnalyzer for CoverageTrack {
181+
fn header(&self) -> Option<String> { Some("#contig\tpos\tdepth\n".into()) }
182+
fn on_column(&mut self, col: &PileupColumn, contig: &str, out: &mut dyn Write) -> io::Result<()> {
183+
writeln!(out, "{contig}\t{}\t{}", col.locus.pos.0 + 1, col.depth())
184+
}
185+
}
186+
// run_bounded_whole_genome(&mut CoverageTrack, source, &ref_view, contigs, params, &mut out)
187+
// → returns the bounded WorkingSet `plan`/`--enforce` reason about.
188+
```
189+
190+
The trait is *welded* to the bounded kernel — `run_bounded_whole_genome` drives the exact same column stream as the shipped `features` egress (which is itself just the first `ColumnAnalyzer`), so the estimator that admits a run provably upper-bounds the realized working set of *your* analyzer too. No other genomics library can offer "implement this trait, inherit a machine-checkable memory budget + a hash-verifiable receipt," because no other library has a memory contract to inherit. See [`examples/columnkit_coverage.rs`](examples/columnkit_coverage.rs).
191+
171192
## Roadmap
172193

173194
The core primitive is a streaming, CIGAR-aware pileup column stream; variant calling and custom plugins consume it. Performance work deliberately *follows* the unique capability — the target user needs "it fits and is predictable" before "it's fastest."
@@ -177,6 +198,7 @@ The core primitive is a streaming, CIGAR-aware pileup column stream; variant cal
177198
- **Phase C (done):** memory as a *verifiable contract*`rosalind plan` (a checkable envelope before you commit), `--enforce` (honor-or-refuse: refuse up front / fail loud, never a silent OOM-kill), and `rosalind verify`. See [CONTRACT.md](CONTRACT.md).
178199
- **Hardening & reach (done):** unbiased depth-cap downsampling (no silent variant drops) and a CI-enforced memory gate; **measured** germline detection accuracy ([Accuracy](#accuracy)); the **`rosalind features`** reproducible ML feature substrate; and a one-command adoption on-ramp — prebuilt binaries (`install.sh`, with checksum verification) plus the **Rosalind budget GitHub Action** (`action.yml`, used as `logannye/rosalind@v0.1.0`) that enforces the contract in *your* CI.
179200
- **Fleet scheduling (done):** [prediction → placement](#pack-a-fleet-prediction--placement)`rosalind pack` proves a co-location of N calling jobs fits a node before launching a byte (predicted peaks are additive and read from the index header); `plan --index --json` for a scheduler to read.
201+
- **ColumnKit SDK (done):** [implement one trait, inherit the contract](#columnkit-implement-one-trait-inherit-the-contract) — a `ColumnAnalyzer` trait + `run_bounded_whole_genome` driver so a builder's own per-locus analyzer inherits bounded memory, determinism, and a verifiable receipt. The shipped `features` egress is the first impl.
180202
- **Phase D (research):** sublinear-space index construction — the `~√t` space/time knob across the full curve — extending the contract to the index *build* step (today's build is O(reference)). The headline space-complexity bet; see [`docs/OPEN_PROBLEMS.md`](docs/OPEN_PROBLEMS.md).
181203
- **Later:** the aligner over the persisted multi-contig index (`align --index`, whole-genome alignment); germline indels and richer read QC; deterministic multithreading; a Python/tensor binding over the pileup stream.
182204

examples/columnkit_coverage.rs

Lines changed: 94 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,94 @@
1+
//! ColumnKit cookbook: a custom per-locus analyzer that inherits the bounded
2+
//! memory contract for free. The only domain logic is `on_column` (~3 lines) —
3+
//! by running it through `run_bounded_whole_genome` it gets the SAME bounded
4+
//! per-contig whole-genome walk and the SAME working-set bound that `plan` /
5+
//! `variants --index --enforce` admit, with no contract code re-derived.
6+
//!
7+
//! Run with: `cargo run --example columnkit_coverage`
8+
9+
use std::collections::BTreeMap;
10+
use std::io::{self, Write};
11+
use std::sync::Arc;
12+
13+
use rosalind::call::{run_bounded_whole_genome, ColumnAnalyzer};
14+
use rosalind::core::{AlignedRead, CigarOp, CigarOpKind, Position, SamFlags};
15+
use rosalind::genomics::{GenomeIndex, IndexReader, IndexWriter};
16+
use rosalind::{PileupColumn, PileupParams, SliceSource};
17+
18+
/// Emit a per-locus coverage track (contig, 1-based pos, depth). This is the
19+
/// entire analyzer a builder writes — bounded memory, determinism, and a
20+
/// verifiable receipt come from the driver, not from here.
21+
struct CoverageTrack;
22+
23+
impl ColumnAnalyzer for CoverageTrack {
24+
fn header(&self) -> Option<String> {
25+
Some("#contig\tpos\tdepth\n".to_string())
26+
}
27+
28+
fn params(&self) -> BTreeMap<String, String> {
29+
BTreeMap::from([("analyzer".to_string(), "coverage".to_string())])
30+
}
31+
32+
fn on_column(
33+
&mut self,
34+
col: &PileupColumn,
35+
contig: &str,
36+
out: &mut dyn Write,
37+
) -> io::Result<()> {
38+
writeln!(out, "{contig}\t{}\t{}", col.locus.pos.0 + 1, col.depth())
39+
}
40+
}
41+
42+
fn read(pos: u32, seq: &[u8]) -> AlignedRead {
43+
AlignedRead {
44+
contig: 0,
45+
pos: Position(pos),
46+
mapq: 60,
47+
flags: SamFlags(0),
48+
cigar: vec![CigarOp::new(CigarOpKind::Match, seq.len() as u32)],
49+
seq: Arc::from(seq.to_vec().into_boxed_slice()),
50+
qual: Arc::from(vec![40u8; seq.len()].into_boxed_slice()),
51+
}
52+
}
53+
54+
fn main() -> Result<(), Box<dyn std::error::Error>> {
55+
// A tiny in-memory genome + a few reads. A real run would pass a persisted
56+
// index and a coordinate-sorted BAM (`StreamingBamSource`) — the driver and
57+
// the contract are identical either way.
58+
let dir = std::env::temp_dir().join("rosalind-columnkit-example");
59+
std::fs::create_dir_all(&dir)?;
60+
let idx_path = dir.join("ref.idx");
61+
let index =
62+
GenomeIndex::from_named_sequences(&[("chr1".to_string(), b"ACGTACGTACGTACGT".to_vec())])?;
63+
IndexWriter::create(&idx_path)?.write_genome_index(&index)?;
64+
let loaded = IndexReader::open(&idx_path)?;
65+
let ref_view = loaded.reference_view()?;
66+
let contigs = loaded.contigs();
67+
68+
let reads = vec![
69+
read(0, b"ACGTACGT"),
70+
read(0, b"ACGTACGT"),
71+
read(4, b"ACGTACGT"),
72+
];
73+
74+
let mut analyzer = CoverageTrack;
75+
let mut out = io::stdout().lock();
76+
let (ws, _skips) = run_bounded_whole_genome(
77+
&mut analyzer,
78+
SliceSource::new(reads),
79+
&ref_view,
80+
contigs,
81+
PileupParams::default(),
82+
&mut out,
83+
)?;
84+
out.flush()?;
85+
86+
eprintln!(
87+
"\n# inherited a bounded working set of {} bytes — coverage-bounded, \
88+
independent of input size; this is the value `plan`/`--enforce` admit.",
89+
ws.bytes
90+
);
91+
92+
std::fs::remove_dir_all(&dir).ok();
93+
Ok(())
94+
}

0 commit comments

Comments
 (0)