feat(recall): nomic embedder + cross-encoder rerank#5
Merged
Conversation
…coder rerank
Replace the 2021-era all-MiniLM-L6-v2 embedder with nomic-embed-text-v1.5
(asymmetric query/document prompts, Matryoshka 768→128, 1024-token seq cap)
and add a cross-encoder rerank stage (ms-marco-MiniLM-L-12-v2) between RRF
fusion and the boost step. Three-stage retrieval — recall (BM25 +
embeddings) → fusion (RRF) → precision (cross-encoder) — gives the priming
snippet meaningful precision lift on a 169-entry library.
Daemon eagerly loads both models + runs a full-pipeline warmup before the
priming RPC socket opens, so the first hook request hits hot MPS kernels
instead of a 6.8s graph-compile cliff. Steady-state latency ~300-500ms,
within the bumped 2s hook budget (500ms accommodates slower hardware).
Calibrations for the new embedder distribution:
- EMBEDDING_NOISE_FLOOR raised 0.25 → 0.50 (nomic's cosine sits higher)
- _RERANK_SCORE_FLOOR at -3.0 catches weak-but-real matches; the
embedding floor handles outright gibberish at the source.
Other adjustments:
- Cross-encoder load failures now propagate (no silent degradation)
- Hook budget 200ms → 2000ms; server-side request timeout 1.0s → 1.8s
- Logging setup pins httpcore/httpx/transformers/sentence_transformers
et al. at WARNING so -v doesn't drown the daemon log in HF wire traffic
- einops added as runtime dep (required by nomic's custom RoPE module)
README updated with HF model download expectations, ~25s boot time, ~2.5GB
resident footprint, and a corrected Tier-1 description.
Tests: 681 passing (439 daemon + 174 search + 68 hooks). Three existing
tests updated to reflect the new "quality > quantity" rerank contract.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
nickroci
added a commit
that referenced
this pull request
May 27, 2026
- Quick start: replaced "Phase 4 work" (internal jargon) with "auto-supervision not yet implemented" — readers landing here shouldn't need to know our phase numbering. - "Where your memories live" table: added the ``cost.json`` row (was already in the deeper Storage-on-disk tree but missing from the user-facing quickstart summary). - Layout section: "177 pytest tests" → pointer to the Status section's current count, so this stops drifting every time we add tests. - Cost section: ambient priming overhead was listed as "~500 chars / ~150 tokens" with "BM25 + embeddings" pipeline. Updated to "~1500 chars / ~400 tokens" (matches the actual Tier-1 budget) and added the cross-encoder rerank to the pipeline description (matches the Tier-1 table and what PR #5 actually shipped). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
all-MiniLM-L6-v2(2021) withnomic-ai/nomic-embed-text-v1.5— asymmetric query/document prompts, Matryoshka 768→128 dims, 8192-token context capped at 1024 for MPS stability.cross-encoder/ms-marco-MiniLM-L-12-v2) between RRF fusion and the reinforcement/scope boost. Three-stage retrieval: BM25 + embeddings → RRF → cross-encoder.EMBEDDING_NOISE_FLOOR0.25 → 0.50; new_RERANK_SCORE_FLOOR = -3.0filters outright non-matches.httpcore/httpx/transformers/sentence_transformerset al. at WARNING so-vdoesn't drown the daemon log in HF wire traffic.einopsadded as runtime dep (nomic's custom RoPE module).Test plan
cd tools/search && uv run pytest— 174/174 pass (includes newtest_reranker.py)cd daemon && uv run pytest— 439/439 pass (3 priming tests updated to reflect the new "quality > quantity" rerank contract)cd src && uv run pytest hooks/test_priming_client.py hooks/test_user_prompt_submit*.py— 68/68 passforgetting-ltd-decay-design).🤖 Generated with Claude Code