fix(rerank): preserve BM25 fallback order in no-op paths (wrong dict key)#62
Open
vinsocci wants to merge 1 commit into
Open
fix(rerank): preserve BM25 fallback order in no-op paths (wrong dict key)#62vinsocci wants to merge 1 commit into
vinsocci wants to merge 1 commit into
Conversation
…key)
rerank()'s four fallback branches (ollama unreachable, model not pulled,
query-embed failure, per-candidate embed failure) set:
c["rerank_score"] = float(c.get("score", 0.0))
But the in-process caller retrieve.py builds each candidate with the key
`bm25_score` (scripts/retrieve.py:147 `"bm25_score": h["score"]`), not
`score`. So in every no-op path c.get("score", 0.0) misses and returns
0.0 for ALL candidates — the BM25 fallback ordering these branches intend
to preserve when ollama is down is silently flattened to a 0.0 tie.
Fix: read bm25_score first, fall back to score (the standalone-CLI
candidate contract documented in the module docstring), then 0.0:
c["rerank_score"] = float(c.get("bm25_score", c.get("score", 0.0)))
This keeps BOTH callers correct — the retrieve.py module path and the
`rerank.py --candidates` CLI path.
Verified on macOS by forcing the no-op path (OLLAMA_URL at a dead
localhost port) with three candidates (bm25_score 12.5/3.1/7.8):
- before: all three rerank_score=0.0
- after: rerank_score = 12.5/3.1/7.8 (BM25 order preserved)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
rerank()'s four fallback branches read the wrong dict key, zeroing every candidate'srerank_scorewhenever the reranker can't run (ollama down, model not pulled, embed failure). The BM25 fallback ordering these branches intend to preserve is silently flattened to a 0.0 tie.The branches set:
But the in-process caller
retrieve.pybuilds each candidate with the keybm25_score(scripts/retrieve.py:147→"bm25_score": h["score"]), notscore. Soc.get("score", 0.0)misses and returns0.0for every candidate.Affected branches in
rerank()(scripts/rerank.py):noop-no-ollama)noop-no-model)noop-embed-error)embed-error)Fix
Read
bm25_scorefirst, fall back toscore(the standalone-CLI candidate contract documented in the module docstring, line 32), then0.0:This keeps both callers correct — the
retrieve.pymodule path (bm25_score) and thererank.py --candidatesCLI path (score).Test plan
Forced the no-op path on macOS by pointing
OLLAMA_URLat a dead localhost port, with three candidates keyedbm25_score(12.5 / 3.1 / 7.8):main)rerank_score=0.0rerank_score=12.5rerank_score=0.0rerank_score=3.1rerank_score=0.0rerank_score=7.8Before: all candidates tie at
0.0(fallback order lost). After: BM25 order preserved.rerank_sourceunchanged.🤖 Generated with Claude Code