Feature-substrate Python boundary + reproducibility demo - #29
Merged
Conversation
…o spec (numpy-only) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…y-only) python/rosalind.py: a dependency-light boundary (stdlib + numpy) that runs "rosalind features" and loads the per-locus table into numpy — meets Python ML pipelines without a pyO3/pyarrow build. examples/reproducible_features_demo.py: builds a toy index+BAM, extracts features TWICE, proves byte-identical TSV + matching BLAKE3 receipts, trains a pure-numpy logistic regression (is-purine-from-counts, a genuine learnable task), and shows the trained weights + predictions are BIT-IDENTICAL across the two extractions. Verified locally: 983,011 loci, receipts match, held-out acc 0.999, weights/preds bit-identical. The novel claim, demonstrated end-to-end: byte-identical features -> bit- reproducible ML training inputs. (pyO3/pyarrow zero-copy binding + Arrow egress are the next step — unverifiable in this env: no maturin/pyarrow.) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…substrate Records the demo's actual output (983k loci, matching receipt BLAKE3, held-out accuracy 0.999, bit-identical weights/predictions across two extractions) + points the README feature-substrate section at python/rosalind.py and the demo. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Follow-up to the feature substrate (PR #28): prove the pull — train a model on
rosalind featuresoutput and show the inputs (and the trained model) are bit-reproducible, hash-verifiable.python/rosalind.py— a dependency-light boundary (stdlib + numpy):features(index, alignments, ...)runsrosalind featuresand loads the per-locus table into numpy. Meets Python ML pipelines without a pyO3/pyarrow build.examples/reproducible_features_demo.py(numpy-only) — builds a toy index+BAM, extracts features twice, proves byte-identical TSV + matching BLAKE3 receipts, trains a pure-numpy logistic regression on a genuine learnable task (is-purine from pileup counts), and shows the trained weights + predictions are bit-identical across the two extractions.Verified locally (output in the findings doc): 983,011 loci,
receipts match: True, held-out accuracy0.9990,weights bit-identical: True,predictions bit-identical: True. The novel claim, demonstrated end-to-end: byte-identical features → bit-reproducible ML training inputs.Spec:
docs/superpowers/specs/2026-06-02-feature-substrate-demo-design.md. Findings:docs/findings/2026-06-02-reproducible-features-demo.md.Honest notes
pyarrowin-process binding (replacing the dead-endpython_bindingsstub) + an Arrow/Parquet egress — built and verified where maturin + pyarrow exist. This demo proves the pull before that investment.Test plan
python -c ast.parseon both modules;cargo fmt --checkclean;cargo testgreen; 0 warnings🤖 Generated with Claude Code