feat(crf): CTC-CRF sequence models, and the manifest seam

[jayhesselberth]

Brings the CTC-CRF stack into src/leech/crf/, and defines the manifest that
lets the rest of the chain follow it out of escapepod-models. The trainer, the
ONNX export and the corpus builder are not in this PR.

What lands

A second task alongside the chunk classifiers. Where a classifier maps a signal
window to a label, a CRF maps one to a sequence: over n_base ** state_len
states whose Viterbi traceback emits one base per move.

• CrfEncoder — 3 convs → 5 alternating LSTMs → linear → tanh/scale → per-state
  blank splice
• CtcCrfLoss — with an analytic forward-backward that replaces a 300-step
  autograd replay with one elementwise multiply
• _triton.py — optional CUDA lattice kernels, gated, falling back to the
  PyTorch reference which stays the correctness oracle
• decode.py — the two-pass Viterbi
• manifest.py — the corpus contract (below)
• configs/crf_ctc.toml — the shipped geometry, as package data

The CTC-CRF formulation is Oxford Nanopore's, introduced in bonito; the seven
architecture hyperparameters are SeqTagger's published values (Genome Res
35:956), cited in the config next to each. It moved here because it had grown
two consumers and no dependency on escapepod-models' model vocabulary, and it is
signal ML — which is what leech is for.

The move is pinned, not assumed

Both copies were run in one process on identical seeded inputs:

surface	result
encoder forward	bit-identical
loss forward	bit-identical
loss backward	bit-identical
forward-backward posteriors	bit-identical
decode_batch strings	bit-identical
geometry from each repo's TOML	identical

On CPU (PyTorch reference paths) and on GPU (Triton kernels engaged). The
gradient is checked explicitly because a wrong one does not crash — it just
trains slightly worse. The script reads the pre-move copy out of
escapepod-models@main, so it stays reproducible once that repo deletes it.

The manifest seam

leech.crf.manifest is one table, one row per read — read_id, pod5, anchor_end, target, plus optional label, group, batch, quality_score,
quality_margin, split. Nothing about where those facts came from.

This is the interface that lets the corpus builder, the metrics and the generic
half of eval leave escapepod-models. Its extractor already declared itself
vocabulary-free — "this script knows about signal windows and nothing about
where a family keeps its labels" — and was, except for one thread: --panel,
used to do exactly two things, canonicalise a class name and map it to
oligo[-target:]. Resolving the target into the manifest cuts that thread.

Two rules encoded here rather than left to callers, because both fail silently:

• Label quality is numbers, never a keep boolean, so the gate stays
  sweepable at training time. Gating one panel's labels moved accuracy from
  0.875 to 0.97; a boolean decided at extraction would mean re-cutting an 8 GB
  corpus per threshold. quality_coverage() reports partial scoring, because an
  unscored read cannot pass a gate and is dropped without a word — this once cut
  a corpus to a non-random 13.5%.
• anchor_end and target are coupled. check_geometry raises rather than
  warns: a short window trains, converges, and quietly discriminates on fewer
  bases than designed. Its message names what the target would emit, because
  an error that only says "too short" invites widening the window — the one fix
  that recovers nothing, since emission is target - state_len at any width.

Import weight is a contract

import leech.crf pulls only torch and numpy, and only on demand — config
path eager, everything else lazy (PEP 562, as in leech.models). escapepod-models
installs leech --no-deps into a conda-forge pixi env so the solver never
reconciles leech's POD5/BAM stack against conda's pytorch, and ldxlib.CRF_CONFIG
is a module constant imported by two dozen scripts that want only edit distances.
polars is imported inside load_manifest for the same reason. Both pinned by
tests — including the sharper one that resolves the lazy attribute, which a
top-level import would break while the blanket check kept passing.

Changes from a straight copy

• test_crf_encoder.py split in two. Its bit-exactness cases compare against
  a built model's reference_io.npz and skipped unconditionally here, since leech
  ships no weights. leech gets a seeded-weights fixture that runs in CI (82 KB;
  weights are filled from a generator rather than stored, so it pins parameter
  names and shapes too). It carries a tolerance — a committed fixture crosses
  machines and torch releases, and CPU LSTM kernels promise no bit-reproducibility
  across either; still five orders of magnitude tighter than any architecture
  error. The real-weights claim stays in escapepod-models.
• pytest.importorskip("torch") dropped. torch is a hard leech dependency;
  here it was a skip that could silently hide the whole CRF suite.
• Renames: load_bonito_state_dict → load_crf_state_dict, _bonito_key_map
  → _legacy_key_map; the config's [model] package key dropped (nothing read
  it, and it named a trainer we no longer use).
• Docstrings describe what the modules are, not what they do not depend on.
  Attribution is stated plainly where owed.

Testing

• 74 CRF tests, all passing on a GPU node; Triton and CUDA cases skip on CPU
• Full suite: 1382 passed, 44 skipped
• ruff format --check, ruff check, ty check, zensical build clean
• Wheel verified to contain leech/crf/configs/crf_ctc.toml

Follow-ups

• escapepod-models rewiring — prepared and handed off; blocked on a leech release
• Corpus builder, metrics and the generic half of eval, over the manifest
• CrfTrainer + leech model train-crf; then the ONNX export
• A cross-language golden with escapepod-demux::crf — which now also covers
  refchain.rs, whose logZ_full/logZ_target duplicate this loss's forward
  terms with no test between them (docs(crf): the CTC-CRF loss refchain mirrors now lives in leech escapepod-rs#269)

🤖 Generated with Claude Code