Skip to content

escapepod-signal should own the signal-level k-mer encoding (leech has the only copy, and it cannot be linked) #271

Description

@jayhesselberth

escapepod-signal owns mapping, which produces a base-to-signal map. The
primitive that consumes one to build a signal-level k-mer encoding lives
downstream in leech, as a Python extension module Rust cannot link. That is
backwards, and it means any Rust consumer of a leech model has to transcribe the
rule.

The primitive

leech/rust/src/encoding.rs::encode_signal_kmer_inner:

fn encode_signal_kmer_inner(
    seq_ints: &[i8],      // sequence as 0..3, negative = skip
    sig_map: &[i64],      // base -> signal index, len = n_bases + 1
    signal_len: usize,
    kmer_before: usize,
    kmer_after: usize,
) -> Vec<f32>             // row-major (4 * kmer_len, signal_len)

It scatters the one-hot k-mer context along the signal axis: for each k-mer
position and each base, fill the signal span of that base with 1.0 in the row
for (kmer_position, base).

It qualifies on every rule this repo already applies:

  • No model vocabulary. No assay, no barcode, no tRNA — sequence ints, a map,
    a length, a context width. It is a signal primitive by construction.
  • No dependencies. Pure std plus slices; the pyo3/numpy code is only in the
    wrapper around it.
  • The input already lives here. escapepod_signal::mapping (feat(signal): move-table and CIGAR coordinate mapping primitives #262) produces
    exactly the seq_to_sig_map this takes.

Why it matters now

leech just gained ONNX export (rnabioco/leech#220). A leech classifier using
--seq-encoding signal_kmer takes a 36-channel sequence input that is this
function's output, computed in the dataset — so it is not in the exported
graph
, and a runtime here must produce it before it can call the model.

The obvious advice, "call leech-core", does not work: leech_core is
crate-type = ["cdylib"], a Python extension module. So today the only options
are to transcribe the rule into this repo or not to run those models at all.

Transcribing is what this stack has been repeatedly bitten by. From this repo's
own notes: KmerTable::extract_levels was written twice with different centring
conventions, silently shifting every predicted level and moving 25 of 100
features; and escapepod-classify reproduced a superseded feature definition for
two months, its counted golden missing it because all 19 fixture reads took the
other branch — found only by comparing against a real corpus, in 4 reads out of
842.

Suggested shape

Add it to escapepod-signal beside mapping — it is the natural pair — and have
leech's leech_core call it, the way leech already delegates the refinement
preset, the POD5 reader cache, span_stats and the mapping primitives upstream
rather than holding local copies.

Failing that, or in the interim, a shared golden pinning leech's implementation
against any Rust transcription, as the CRF decode and the charging features
already have. Note that a golden is the weaker option here for the usual reason:
it pins the paths it exercises, and this one has an easy-to-miss branch (a base
whose signal span is empty, sig_start == sig_end, contributes nothing).

Contract details worth capturing either way

  • Output is row-major (4 * kmer_len, signal_len), kmer_len = before + 1 + after.
  • seq_len is sig_map.len() - 1, not the sequence length.
  • Bases are indexed seq_pos + kmer_pos; out-of-range and negative bases are
    skipped, not padded.
  • Spans are clamped to signal_len on both ends.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions