Skip to content

Commit d616dd5

Browse files
kfaracikclaude
andcommitted
feat(rag): add hybrid retrieval + language layer on top of the vector baseline
Stacked on the vanilla vector RAG. Re-introduces the retrieval-quality layer: keyword (FTS5/BM25) indexing, reciprocal rank fusion, MMR diversification, adaptive-k, term-coverage boosting, and multilingual (Polish/English) no-answer detection, with the HybridRetriever app↔lib boundary and full test coverage (keywordIndex, rankFusion, hybridRetrieval, pipeline integration). Hardening carried over from review: - Keyword index backfills once from the existing vectors when it is empty, so documents imported on the vector-only build become keyword-searchable after upgrade instead of staying invisible forever. - Keyword-only hits must clear a low semantic-similarity bar when a query embedding is available, so a loose prefix match (false friend) no longer qualifies unconditionally; keyword-without-embeddings behaviour is preserved. - Semantic gate recalibrated to 0.40 with a top-keep floor (0.25) for the single best candidate, matching the on-device eval. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
1 parent f0e39ae commit d616dd5

16 files changed

Lines changed: 1594 additions & 45 deletions

__tests__/VectorStoreContext.test.tsx

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -32,6 +32,10 @@ jest.mock('../utils/embeddingModelMigration', () => ({
3232
migrateEmbeddingModelIfNeeded: jest.fn().mockResolvedValue(undefined),
3333
}));
3434

35+
jest.mock('../database/keywordIndex', () => ({
36+
ensureKeywordIndex: jest.fn().mockResolvedValue(undefined),
37+
}));
38+
3539
jest.mock('../utils/embeddingModel', () => ({
3640
isEmbeddingModelDownloaded: jest.fn().mockResolvedValue(true),
3741
}));

0 commit comments

Comments
 (0)