Skip to content

Commit 388f7fd

Browse files
Your Nameclaude
andcommitted
feat(embedding): D -- optional local-ONNX embedding backend (tract)
Implements D from docs/superskills/plans/2026-07-30-calm-dfb-levers-design.md §3: an opt-in, off-by-default high-accuracy embedding backend alongside the default model2vec-rs path, selected via a `model_id` of the form `onnx:<dir>` where `<dir>` contains `model.onnx` (an encoder exported without a pooling head) + `tokenizer.json`. `Embedder` becomes a thin wrapper over a private `Backend` enum (`Static(Box<StaticModel>)` / `Onnx(Box<OnnxEmbedder>)`) so the existing public API (`load`/`dim`/`embed_one`/`embed_batch`) and every existing caller is unchanged. Runtime choice (`tract-onnx` 0.23.4) was verified, not assumed, against the audit-design Risk Assessment's HIGH-severity finding that musl-general evidence for a crate doesn't prove it can execute a specific model's ONNX graph: a standalone spike loaded a real BERT-family transformer export (all-MiniLM-L6-v2, same op family as code-embedding models like CodeRankEmbed), ran tokenize -> forward-pass -> masked-mean-pool -> L2-normalize, and confirmed a real semantic signal (paraphrased sentences cosine-scored 0.96 vs. 0.06 for an unrelated one) -- identically on native glibc and a static x86_64-unknown-linux-musl build. Two real, load-bearing findings surfaced by that verification, not assumed away: - `tokenizers`' default features pull in `onig` (C Oniguruma bindings) and `esaxx_fast` (`esaxx-rs/cpp`) -- exactly the class of problem `sqlite-vec` caused before. Fixed via `default-features = false` + `features = ["fancy-regex"]` (pure Rust). - `esaxx-rs` is still a required (non-optional) tokenizers dependency regardless of `esaxx_fast`, and its build.rs runs an UNCONDITIONAL C++ compiler-family probe that hard-fails looking for a literal `x86_64-linux-musl-g++` binary. Documented as a real musl build precondition (not eliminated): `CXX_x86_64_unknown_linux_musl` must point at any C++ compiler (native is fine -- the actual `cpp` path is never invoked with the feature off, so this only has to satisfy the probe, not genuinely cross-compile). Verified end-to-end on the real codebase, not just the spike: full workspace test suite (275+ tests) green on both default and `onnx-embeddings` feature sets; `cargo clippy -D warnings` clean on both (fixed 6 real lints along the way -- large_enum_variant on both Backend variants, 4 doc_lazy_continuation, 1 needless_borrows_for_generic_args); and the actual `calm-cli` release binary cross-compiled for musl with `onnx-embeddings` enabled, confirmed `static-pie linked` / `ldd: statically linked`, and runs correctly (`--version`) -- done twice (before and after the clippy fixes) to catch any regression the lint fixes might have introduced. Also fixes B3's measurement gap found by the same audit (Finding A2): a before/after backend comparison needs `kind="semantic"` scored on its own (pre-RRF-fusion -- `kind="hybrid"` dilutes a new backend's contribution with the FTS leg) and an explicit check that no query silently degraded to FTS-only (a "no improvement" result from a degraded run would mean the backend was never actually exercised, not that it doesn't help). Verified live: all 12 queries active, `all_queries_active: true`, `ci(semantic)` mean 0.276 now visible as its own number for the first time. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
1 parent 555126a commit 388f7fd

5 files changed

Lines changed: 914 additions & 16 deletions

File tree

0 commit comments

Comments
 (0)