Multimodal Agentic RAG

A production-grade RAG system: hybrid retrieval (vector + BM25 + RRF), local cross-encoder reranking, streaming answers with inline citations, and a FastAPI + React frontend.

Quick start (Docker — recommended)

# 1. Clone
git clone <your-repo-url> rag
cd rag

# 2. Add your API keys
cp .env.example .env
# then edit .env and paste your GROQ_API_KEY (free tier: console.groq.com)

# 3. Boot everything (first time: ~5 min while bge-m3 downloads)
docker compose up --build

# 4. Open the frontend
# http://localhost:5173

What you get

• http://localhost:5173 — React frontend
• http://localhost:8000/api/health — backend liveness check
• http://localhost:8000/api/info — model configuration
• http://localhost:11434 — Ollama serving bge-m3 embeddings

Architecture

Frontend (nginx :5173)  ──/api──►  Backend (FastAPI :8000)  ──►  Ollama (:11434)
                                          │
                                          ├──►  Chroma (vector DB, on disk)
                                          ├──►  BM25 (pickle, on disk)
                                          ├──►  Cross-encoder (in RAM)
                                          └──►  Groq (external API)

Screenshots

Local dev (without Docker)

If you want to iterate faster than rebuild-and-run:

# 1. Ollama (host machine)
ollama pull bge-m3
ollama serve &

# 2. Python backend
python -m venv .venv
source .venv/bin/activate      # Windows: .venv\Scripts\activate
pip install -r requirements.txt
uvicorn api:app --reload --port 8000

# 3. Frontend (separate terminal)
cd web
npm install
npm run dev                    # opens http://localhost:5173

# OR: Streamlit UI instead of React
streamlit run app.py

Ingest documents from the command line

# Single file
python scripts/ingest.py path/to/document.pdf

# All PDFs in data/raw_pdfs/
python scripts/ingest.py

Run the eval harness

See tests/eval/README.md for how to fill in the golden dataset. Then:

python tests/eval/run_eval.py

Observability

Logs are structured JSON with per-request IDs. To trace one query end-to-end:

# Follow live
docker compose logs -f backend

# Grep a single request across all its stages
docker compose logs backend | grep <request_id>

# Pretty output for local dev
LOG_FORMAT=console docker compose up

Configuration

Every knob is in config.py and overridable via .env or environment variables. Key settings:

Setting	Default	What it does
embed_model	bge-m3	Ollama embedding model
groq_model	llama-3.3-70b-versatile	Generator LLM
reranker_model	cross-encoder/ms-marco-MiniLM-L-6-v2	Local cross-encoder
top_k_final	3	Chunks sent to the LLM
chunk_size_tokens	512	Target chunk size

Directory structure

├── api.py                 # FastAPI app (lifespan warmup, middleware, routes)
├── app.py                 # Streamlit UI (alternative to React)
├── config.py              # Settings, .env-driven
├── docker-compose.yml
├── docker/
│   ├── Dockerfile.backend
│   ├── Dockerfile.frontend
│   └── nginx.conf
├── frontend/              # Streamlit components
├── scripts/
│   └── ingest.py          # CLI ingestion
├── src/
│   ├── chunk.py           # Header-aware + semantic chunking
│   ├── embed.py           # Ollama embeddings
│   ├── generate.py        # Groq streaming with citations
│   ├── logging_config.py  # structlog JSON + request IDs
│   ├── parse.py           # LlamaParse (multimodal)
│   ├── parse_local.py     # markitdown (free/local)
│   ├── pipeline.py        # Orchestration
│   ├── rerank.py          # Cross-encoder (warm-started at boot)
│   ├── router.py          # Follow-up rewriting + intent classification
│   ├── schema.py          # Pydantic models
│   └── store.py           # Chroma + BM25 + RRF
├── tests/
│   └── eval/              # RAGAS harness
└── web/                   # React (Vite) frontend