Skip to content

Commit bdce9c5

Browse files
logannyeclaude
andcommitted
test(cli): B3c determinism + self-contained gates; docs(readme): build-once → query
CLI gates: two rosalind index builds are byte-identical, and rosalind locate serves queries from the .idx alone after the source FASTA is deleted (load never rebuilds). README documents the build-once -> query workflow and the record-only --memory-budget-mb plan line. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
1 parent 9d8fed4 commit bdce9c5

2 files changed

Lines changed: 104 additions & 3 deletions

File tree

README.md

Lines changed: 30 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -21,7 +21,7 @@ Rosalind streams variant calling over coordinate-sorted alignments with a workin
2121

2222
At the **command line**, Rosalind currently operates on a **single reference contig per run**, reads plain or **gzip/bgzf-compressed** FASTQ/FASTA (auto-detected, including from stdin), and runs **single-threaded**. Variant calling is **single-sample** (germline) or a **tumor/normal pair** (somatic); calling is SNV-focused, with simple indels in the somatic path. Alignment uses exact-match seeding. The FM-index is built in memory at the start of each run (memory proportional to the reference); the bounded-memory property applies to the streaming pileup and variant-calling stages. These boundaries define what the engine targets well today — small-to-moderate references, targeted regions, and per-sample streaming workloads.
2323

24-
The library also provides a multi-contig FM-index over the concatenated genome (`genomics::GenomeIndex`) that resolves matches to `(contig, position)`; it is not yet used by the CLI. See the roadmap below.
24+
The library also provides a multi-contig FM-index over the concatenated genome (`genomics::GenomeIndex`) that resolves matches to `(contig, position)`; it is exposed via `rosalind index` / `rosalind locate` (wiring multi-contig through `align`/`variants` is a later phase). See the roadmap below.
2525

2626
## Who it's for
2727

@@ -36,7 +36,7 @@ The library also provides a multi-contig FM-index over the concatenated genome (
3636
The core primitive is a streaming, CIGAR-aware pileup column stream; variant calling and custom plugins consume it.
3737

3838
- **Phase A (done):** the streaming pileup engine; calibrated, abstention-aware germline SNV calling; tumor/normal somatic SNV calling; spec-valid VCF output; a BLAKE3 reproducibility receipt per run.
39-
- **Phase B (in progress):** streaming gzip/bgzf input and a multi-contig FM-index over the concatenated genome (`genomics::GenomeIndex`, with `(contig, position)` resolution and boundary-aware exact-match lookup) have landed. Next: wiring multi-contig through the CLI (whole-genome alignment and calling), a build-once memory-mapped index (`rosalind index`), and pipe-native composition across subcommands.
39+
- **Phase B (in progress):** streaming gzip/bgzf input, a multi-contig FM-index over the concatenated genome (`genomics::GenomeIndex`, with `(contig, position)` resolution and boundary-aware exact-match lookup), and a build-once, memory-mapped index (`rosalind index` to build, `rosalind locate` to query — never rebuilds, byte-identically reproducible) have landed. Next: wiring multi-contig through the `align`/`variants` CLI (whole-genome alignment and calling), and pipe-native composition across subcommands.
4040
- **Later:** germline indel calling and richer read QC; deterministic multithreading with an enforced memory budget; a Python binding exposing the pileup stream.
4141

4242
Target architecture and per-phase plans: [`docs/superpowers/specs/`](docs/superpowers/specs/), [`docs/superpowers/plans/`](docs/superpowers/plans/).
@@ -104,6 +104,34 @@ Other subcommands (run `rosalind <subcommand> --help` for exact flags):
104104

105105
`align` indexes the first FASTA record; additional records are ignored with a warning (single-contig scope). `variants` reads coordinate-sorted SAM/BAM alignments. Inputs may be plain or gzip/bgzf-compressed (auto-detected); pass `-` to read FASTQ from stdin, e.g. `gzip -dc reads.fastq.gz | rosalind align --reads - --reference ref.fa --format sam`.
106106

107+
## Build once, query many: the persisted index
108+
109+
Build a portable, memory-mappable index from a (multi-contig) reference once:
110+
111+
```bash
112+
rosalind index --reference genome.fa --output genome.idx
113+
# index: genome.idx
114+
# contigs: 3 (90 bp total)
115+
# chr1 30
116+
# ...
117+
# reference_blake3: <hex>
118+
# index_bytes: <n>
119+
```
120+
121+
Then query it in milliseconds — it is memory-mapped, never rebuilt:
122+
123+
```bash
124+
rosalind locate --index genome.idx --pattern GATTACA
125+
# chr3 0
126+
# chr3 11
127+
```
128+
129+
`rosalind index` is deterministic (the `.idx` is byte-identical across builds of
130+
the same reference). `--memory-budget-mb M` prints a record-only build plan line
131+
(`[OK]`/`[OVER]`) — it does not yet enforce the budget (that is a later phase).
132+
`locate` is exact-match only; seed/chain/extend alignment against the persisted
133+
index lands in a later phase.
134+
107135
### Rust API
108136

109137
```rust

tests/index_cli.rs

Lines changed: 74 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -143,7 +143,12 @@ fn locate_matches_in_ram_ground_truth() {
143143
])
144144
.unwrap();
145145

146-
for pat in ["GATTACA", "ACGT", "GGGGG", "NNNN", "TTTTGGGG", "ZZZZ"] {
146+
// Battery: multi-hit, lowercase (case-insensitive), cross-contig boundary
147+
// straddle ("CTTTTTT" spans chr1's tail into chr2 -> rejected, 0 hits),
148+
// N-bearing, and an invalid base ("ZZZZ" -> 0 hits).
149+
for pat in [
150+
"GATTACA", "gattaca", "ACGT", "GGGGG", "NNNN", "TTTTGGGG", "CTTTTTT", "ZZZZ",
151+
] {
147152
let out = Command::new(bin())
148153
.args(["locate", "--index", idx.to_str().unwrap(), "--pattern", pat])
149154
.output()
@@ -170,3 +175,71 @@ fn locate_matches_in_ram_ground_truth() {
170175
}
171176
std::fs::remove_dir_all(&dir).ok();
172177
}
178+
179+
#[test]
180+
fn index_build_is_deterministic_via_cli() {
181+
let dir = tmpdir();
182+
let fa = write_fasta(&dir);
183+
let idx1 = dir.join("a.idx");
184+
let idx2 = dir.join("b.idx");
185+
for out in [&idx1, &idx2] {
186+
let r = Command::new(bin())
187+
.args([
188+
"index",
189+
"--reference",
190+
fa.to_str().unwrap(),
191+
"--output",
192+
out.to_str().unwrap(),
193+
])
194+
.output()
195+
.expect("index");
196+
assert!(r.status.success());
197+
}
198+
assert_eq!(
199+
std::fs::read(&idx1).unwrap(),
200+
std::fs::read(&idx2).unwrap(),
201+
"two CLI builds of the same reference must be byte-identical"
202+
);
203+
std::fs::remove_dir_all(&dir).ok();
204+
}
205+
206+
#[test]
207+
fn locate_works_from_the_artifact_alone() {
208+
// Build, delete the source FASTA, then locate from the index file alone —
209+
// proving the load path is self-contained and never rebuilds.
210+
let dir = tmpdir();
211+
let fa = write_fasta(&dir);
212+
let idx = dir.join("ref.idx");
213+
assert!(Command::new(bin())
214+
.args([
215+
"index",
216+
"--reference",
217+
fa.to_str().unwrap(),
218+
"--output",
219+
idx.to_str().unwrap(),
220+
])
221+
.output()
222+
.expect("index")
223+
.status
224+
.success());
225+
std::fs::remove_file(&fa).unwrap(); // the only source of the sequence is now gone
226+
227+
let out = Command::new(bin())
228+
.args([
229+
"locate",
230+
"--index",
231+
idx.to_str().unwrap(),
232+
"--pattern",
233+
"GATTACA",
234+
])
235+
.output()
236+
.expect("locate");
237+
assert!(out.status.success());
238+
let stdout = String::from_utf8_lossy(&out.stdout);
239+
assert_eq!(
240+
stdout.lines().count(),
241+
2,
242+
"GATTACA: 2 hits in chr3, served from the artifact alone"
243+
);
244+
std::fs::remove_dir_all(&dir).ok();
245+
}

0 commit comments

Comments
 (0)