-
Notifications
You must be signed in to change notification settings - Fork 94
Description
Hi @bharagha @14pankaj @jgespino, I have a proposal for following enhancement, please assign this issue to me
1. Description of the Enhancement
This enhancement proposes upgrading the chat-question-and-answer pipeline from a dense-only retrieval (PGVector MMR) to a Hybrid Retrieval strategy. By integrating a sparse BM25 retriever with the existing vector search, the application will achieve significantly higher accuracy for keyword-specific queries.
Technical Overview:
- Integrate LangChain's
EnsembleRetrieverto fuse Dense and Sparse search results. - Add
rank-bm25as a dependency inpyproject.toml. - Configure weighted scoring via environment variables to balance semantic vs. lexical relevance.
2. Use Cases and Benefits
- Technical Accuracy: BM25 provides precise matches for specific keywords, product codes, and rare technical terms (e.g., "OpenVINO-2026") where dense embeddings might be imprecise.
- Robust Retrieval: Provides a "safety net" for short queries or terms not well-represented in the embedding model's vocabulary.
- Industry Standard: Demonstrates RAG best practices using state-of-the-art hybrid search techniques.
3. Additional Context or References
- LangChain Ensemble Retriever: Documentation
- Target Component:
app/chain.py - This upgrade ensures the Intel Edge GenerativeAI Suite's sample applications remain benchmark-competitive and robust for technical documentation use cases.
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
No labels