Skip to content

Commit 45f4564

Browse files
logannyeclaude
andcommitted
docs(readme): the reproducible feature substrate (rosalind features)
bounded + byte-identical per-locus features -> bit-reproducible ML training inputs with a verifiable receipt; one-line pandas read; Arrow/pyarrow on the roadmap. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
1 parent 19daaee commit 45f4564

1 file changed

Lines changed: 17 additions & 0 deletions

File tree

README.md

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -121,6 +121,23 @@ Drop the `rosalind-budget` Action into any pipeline to make a declared memory bu
121121
122122
It runs `plan` (predicts the peak), then `variants --index --enforce` (honors the budget), and uploads the BLAKE3 receipt as a build artifact. This is the one thing a `--max-mem` flag on another caller can't give you: a portable, declarative, **verifiable** memory budget that fails a stranger's build loudly — the contract, enforced where your pipeline already lives. (Available once a release is published; see Quickstart.)
123123

124+
## A reproducible feature substrate for ML
125+
126+
The same bounded streaming engine that calls variants can emit **per-locus features** instead — one tabular row per callable position, ready for a model:
127+
128+
```sh
129+
rosalind features --index ref.idx --alignments sorted.bam -o features.tsv
130+
# columns: contig, pos, ref, depth, raw_depth, A/C/G/T counts,
131+
# per-allele fwd/rev strand counts, mean base-qual, mean mapq
132+
```
133+
134+
```python
135+
import pandas as pd
136+
df = pd.read_csv("features.tsv", sep="\t") # one line; ready for sklearn/PyTorch/JAX
137+
```
138+
139+
Two properties no other pileup gives you together: it is **bounded** (the whole-genome table streams to disk; peak memory tracks coverage, not genome size — a 1 Mbp toy genome's ~983k-row table is produced in ~6 MiB), and it is **byte-identical run-to-run**, with a BLAKE3 receipt over the output. That means **bit-reproducible training inputs**: hash your feature file, and you can prove this quarter's model saw exactly the same data as last quarter's. `features` honors the same `plan`/`--enforce`/`verify` memory contract as `variants`. *(TSV today; an Arrow/Parquet egress and a zero-copy `pyarrow` Python binding are on the roadmap.)*
140+
124141
## Roadmap
125142

126143
The core primitive is a streaming, CIGAR-aware pileup column stream; variant calling and custom plugins consume it. Performance work deliberately *follows* the unique capability — the target user needs "it fits and is predictable" before "it's fastest."

0 commit comments

Comments
 (0)