Skip to content

fix(rerank): preserve BM25 fallback order in no-op paths (wrong dict key)#62

Open
vinsocci wants to merge 1 commit into
AgriciDaniel:mainfrom
vinsocci:fix/rerank-noop-bm25-fallback
Open

fix(rerank): preserve BM25 fallback order in no-op paths (wrong dict key)#62
vinsocci wants to merge 1 commit into
AgriciDaniel:mainfrom
vinsocci:fix/rerank-noop-bm25-fallback

Conversation

@vinsocci

Copy link
Copy Markdown

Summary

rerank()'s four fallback branches read the wrong dict key, zeroing every candidate's rerank_score whenever the reranker can't run (ollama down, model not pulled, embed failure). The BM25 fallback ordering these branches intend to preserve is silently flattened to a 0.0 tie.

The branches set:

c["rerank_score"] = float(c.get("score", 0.0))

But the in-process caller retrieve.py builds each candidate with the key bm25_score (scripts/retrieve.py:147"bm25_score": h["score"]), not score. So c.get("score", 0.0) misses and returns 0.0 for every candidate.

Affected branches in rerank() (scripts/rerank.py):

  • ollama unreachable (noop-no-ollama)
  • model not pulled (noop-no-model)
  • query-embed failure (noop-embed-error)
  • per-candidate embed failure (embed-error)

Fix

Read bm25_score first, fall back to score (the standalone-CLI candidate contract documented in the module docstring, line 32), then 0.0:

c["rerank_score"] = float(c.get("bm25_score", c.get("score", 0.0)))

This keeps both callers correct — the retrieve.py module path (bm25_score) and the rerank.py --candidates CLI path (score).

Test plan

Forced the no-op path on macOS by pointing OLLAMA_URL at a dead localhost port, with three candidates keyed bm25_score (12.5 / 3.1 / 7.8):

candidate before (main) after
a (bm25 12.5) rerank_score=0.0 rerank_score=12.5
b (bm25 3.1) rerank_score=0.0 rerank_score=3.1
c (bm25 7.8) rerank_score=0.0 rerank_score=7.8

Before: all candidates tie at 0.0 (fallback order lost). After: BM25 order preserved. rerank_source unchanged.

🤖 Generated with Claude Code

…key)

rerank()'s four fallback branches (ollama unreachable, model not pulled,
query-embed failure, per-candidate embed failure) set:

    c["rerank_score"] = float(c.get("score", 0.0))

But the in-process caller retrieve.py builds each candidate with the key
`bm25_score` (scripts/retrieve.py:147 `"bm25_score": h["score"]`), not
`score`. So in every no-op path c.get("score", 0.0) misses and returns
0.0 for ALL candidates — the BM25 fallback ordering these branches intend
to preserve when ollama is down is silently flattened to a 0.0 tie.

Fix: read bm25_score first, fall back to score (the standalone-CLI
candidate contract documented in the module docstring), then 0.0:

    c["rerank_score"] = float(c.get("bm25_score", c.get("score", 0.0)))

This keeps BOTH callers correct — the retrieve.py module path and the
`rerank.py --candidates` CLI path.

Verified on macOS by forcing the no-op path (OLLAMA_URL at a dead
localhost port) with three candidates (bm25_score 12.5/3.1/7.8):
- before: all three rerank_score=0.0
- after:  rerank_score = 12.5/3.1/7.8 (BM25 order preserved)

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@vinsocci vinsocci requested a review from AgriciDaniel as a code owner May 29, 2026 01:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant