Commit d616dd5
feat(rag): add hybrid retrieval + language layer on top of the vector baseline
Stacked on the vanilla vector RAG. Re-introduces the retrieval-quality layer:
keyword (FTS5/BM25) indexing, reciprocal rank fusion, MMR diversification,
adaptive-k, term-coverage boosting, and multilingual (Polish/English)
no-answer detection, with the HybridRetriever app↔lib boundary and full test
coverage (keywordIndex, rankFusion, hybridRetrieval, pipeline integration).
Hardening carried over from review:
- Keyword index backfills once from the existing vectors when it is empty, so
documents imported on the vector-only build become keyword-searchable after
upgrade instead of staying invisible forever.
- Keyword-only hits must clear a low semantic-similarity bar when a query
embedding is available, so a loose prefix match (false friend) no longer
qualifies unconditionally; keyword-without-embeddings behaviour is preserved.
- Semantic gate recalibrated to 0.40 with a top-keep floor (0.25) for the single
best candidate, matching the on-device eval.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>1 parent f0e39ae commit d616dd5
16 files changed
Lines changed: 1594 additions & 45 deletions
File tree
- __tests__
- constants
- context
- database
- store
- utils
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
32 | 32 | | |
33 | 33 | | |
34 | 34 | | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
35 | 39 | | |
36 | 40 | | |
37 | 41 | | |
| |||
0 commit comments