rag-params-finder

Find your optimal RAG configuration — before you build your RAG application.

RAG parameter sweep experimentation tool — systematically evaluate embedding models, chunking strategies, and retrieval methods using MongoDB Atlas Vector Search. Supports Voyage AI (hosted) and local sentence-transformers (no API key needed).

Most RAG projects start with a guess: pick an embedding model, pick a chunking method, a retrieval method (or a re-ranker), realise it's wrong, refactor. That loop is slow and expensive.

rag-params-finder inverts it.

Give it your data and your questions. It runs every combination — embedding model × chunking method × retrieval method — stores the retrieval scores and shows you exactly which configuration performs best. Before you write a single line of your RAG application.

Why this matters

What you avoid	What you get instead
No LLM calls	Embedding only — 10–100× cheaper
No eval framework setup	One YAML config, one CLI command
No deployed RAG app needed	Just your data, your questions, your credentials
No guessing	Actual retrieval scores across every config, side by side
No throwaway experiments	Results persist — compare runs across sessions

What it sweeps

• Embedding models: voyage-3.5-lite, voyage-3.5, voyage-context-3
• Chunking methods: Fixed · Recursive · Token · Sentence · Semantic
• Retrieval methods: Dense · Sparse · Hybrid
• Questions: Persona-organised — realistic, not synthetic

One YAML. N experiments. Evidence-based decision. Ship the right config first.

---

📸 Screenshots

Screen	Description
	Experiments list — all submitted sweeps with status badges and run counts
	Experiment detail — metric cards, live phase indicator dots, runs table
	Search Explorer — best-parameters card, ranked configs with score bars

---

🚀 Quick Start

Prerequisites: Python 3.12+, Node.js 22+, MongoDB Atlas account (free tier)

# Clone and install
git clone https://github.com/neomatrix369/rag-params-finder.git
cd rag-params-finder
uv venv && source .venv/bin/activate
uv pip install -e .
cd frontend && npm install && cd ..

# Configure
cp .env.example .env
# Edit .env: add MONGODB_URI (required) and VOYAGE_API_KEY (optional)

# Start
uvicorn server.main:app --reload --port 8001   # Terminal 1
cd frontend && npm run dev                      # Terminal 2

# Run an experiment (no API key needed)
rag-params-finder run --config configs/example-local.yaml

Open http://localhost:5173 to watch live progress and explore results.

---

🗺️ Choose Your Path

I want to…	Start here
Run my first experiment	Getting Started
Understand all config options	Configuration Reference
Learn all CLI commands	CLI Reference
Understand the dashboard	Dashboard Guide
Fix an error	Troubleshooting
Understand the system design	Architecture
Add a new model, chunker, or endpoint	Extending the System
Set up a development environment	Development Guide
Why these design choices?	ADR-001 · ADR-002 · ADR-003

---

⚡ Key Features

• 5 chunking methods: Fixed, Recursive, Token, Sentence, Semantic
• 3 retrieval methods: Dense (vector search), Sparse (BM25), Hybrid (70/30 weighted)
• Voyage AI models: voyage-3.5-lite, voyage-3.5, voyage-context-3 + rerank-2.5-lite
• Local models (no API key): all-MiniLM-L6-v2 + cross-encoder/ms-marco-MiniLM-L-6-v2
• Multi-format data loading: PDF, TXT, Markdown, CSV — files or directories
• Cartesian sweep: one YAML config → N models × M methods × P sizes × Q overlaps runs
• Live phase tracking: QUEUED → PARSING → CHUNKING → EMBEDDING → STORING → QUERYING → RERANKING → COMPLETE

---

🧱 Built With

Backend: FastAPI · Python 3.12 · Pydantic · PyMongo · LangChain text splitters · pypdf · Typer · Rich · sentence-transformers · NLTK · tiktoken

Frontend: React 19 · TypeScript 5.8 · Vite 6 · Tailwind CSS

AI/ML: Voyage AI · sentence-transformers · MongoDB Atlas Vector Search

Dev tools: uv · ruff · mypy · pytest · GitHub Actions

---

🤝 Contributing

Contributions welcome — please open an issue first to discuss the change.

Priority areas: additional chunkers (sentence, token, semantic are stubbed), sparse/hybrid retrieval wiring, test suite with mock MongoDB fixtures, SSE live updates, Docker Compose.

---

📄 License

MIT — see LICENSE

---

🙏 Credits

Inspired by pre-rag-explorer-dashboard.