Commit 388f7fd
feat(embedding): D -- optional local-ONNX embedding backend (tract)
Implements D from docs/superskills/plans/2026-07-30-calm-dfb-levers-design.md
§3: an opt-in, off-by-default high-accuracy embedding backend alongside the
default model2vec-rs path, selected via a `model_id` of the form `onnx:<dir>`
where `<dir>` contains `model.onnx` (an encoder exported without a pooling
head) + `tokenizer.json`. `Embedder` becomes a thin wrapper over a private
`Backend` enum (`Static(Box<StaticModel>)` / `Onnx(Box<OnnxEmbedder>)`) so the
existing public API (`load`/`dim`/`embed_one`/`embed_batch`) and every
existing caller is unchanged.
Runtime choice (`tract-onnx` 0.23.4) was verified, not assumed, against the
audit-design Risk Assessment's HIGH-severity finding that musl-general
evidence for a crate doesn't prove it can execute a specific model's ONNX
graph: a standalone spike loaded a real BERT-family transformer export
(all-MiniLM-L6-v2, same op family as code-embedding models like
CodeRankEmbed), ran tokenize -> forward-pass -> masked-mean-pool ->
L2-normalize, and confirmed a real semantic signal (paraphrased sentences
cosine-scored 0.96 vs. 0.06 for an unrelated one) -- identically on native
glibc and a static x86_64-unknown-linux-musl build.
Two real, load-bearing findings surfaced by that verification, not assumed
away:
- `tokenizers`' default features pull in `onig` (C Oniguruma bindings) and
`esaxx_fast` (`esaxx-rs/cpp`) -- exactly the class of problem
`sqlite-vec` caused before. Fixed via `default-features = false` +
`features = ["fancy-regex"]` (pure Rust).
- `esaxx-rs` is still a required (non-optional) tokenizers dependency
regardless of `esaxx_fast`, and its build.rs runs an UNCONDITIONAL C++
compiler-family probe that hard-fails looking for a literal
`x86_64-linux-musl-g++` binary. Documented as a real musl build
precondition (not eliminated): `CXX_x86_64_unknown_linux_musl` must
point at any C++ compiler (native is fine -- the actual `cpp` path is
never invoked with the feature off, so this only has to satisfy the
probe, not genuinely cross-compile).
Verified end-to-end on the real codebase, not just the spike: full workspace
test suite (275+ tests) green on both default and `onnx-embeddings` feature
sets; `cargo clippy -D warnings` clean on both (fixed 6 real lints along the
way -- large_enum_variant on both Backend variants, 4 doc_lazy_continuation,
1 needless_borrows_for_generic_args); and the actual `calm-cli` release
binary cross-compiled for musl with `onnx-embeddings` enabled, confirmed
`static-pie linked` / `ldd: statically linked`, and runs correctly
(`--version`) -- done twice (before and after the clippy fixes) to catch any
regression the lint fixes might have introduced.
Also fixes B3's measurement gap found by the same audit (Finding A2): a
before/after backend comparison needs `kind="semantic"` scored on its own
(pre-RRF-fusion -- `kind="hybrid"` dilutes a new backend's contribution with
the FTS leg) and an explicit check that no query silently degraded to
FTS-only (a "no improvement" result from a degraded run would mean the
backend was never actually exercised, not that it doesn't help). Verified
live: all 12 queries active, `all_queries_active: true`,
`ci(semantic)` mean 0.276 now visible as its own number for the first time.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>1 parent 555126a commit 388f7fd
5 files changed
Lines changed: 914 additions & 16 deletions
File tree
- benchmarks/b3_search_quality
- crates
- calm-cli
- calm-core
- src
0 commit comments