A fully-local, privacy-focused RAG (Retrieval-Augmented Generation) chatbot system. All data processing, storage, and inference happens locally using Ollama for LLM capabilities.
Project under construction!
- 100% Local: All data stays on your machine — no external APIs, no data leaving your device
- RAG-Powered: Retrieval-Augmented Generation for accurate, context-aware responses with source attribution
- Multi-Format Support: PDF, DOCX, TXT, MD, CSV, XLSX document ingestion
- Project Isolation: Organize documents and conversations into separate projects with isolated vector stores
- Real-time Streaming: WebSocket-based token streaming for responsive chat experience
- Chat Settings: Fine-tune chat behaviour and generation parameters per session (e.g. top-k, temperature, max tokens)
- Model Management: Browse, download, and configure language models and embedding models directly from the UI via Ollama integration
- Pull new LLM models (e.g. Llama 3.2, Mistral, Mixtral)
- Pull and switch embedding models (e.g. nomic-embed-text, mxbai-embed-large)
- View model details such as size, family, and quantization level
- Embedding Analytics: Visualize and analyze document embeddings with interactive dashboards
-
2D/3D scatter plots with dimensionality reduction (PCA, t-SNE, UMAP)
-
Similarity heatmaps at document and chunk level
-
Embedding statistics and searchable data tables with CSV export
-
Retrieval quality testing to evaluate RAG performance
-
- Framework: FastAPI (Python 3.11+)
- Database: SQLAlchemy + SQLite
- Vector Store: ChromaDB
- LLM: Ollama (Llama 3.2)
- Embeddings: nomic-embed-text or mxbai-embed-large
- Analytics: scikit-learn (PCA, t-SNE), umap-learn (UMAP)
- Framework: Next.js 14 with App Router
- Language: TypeScript
- Styling: Tailwind CSS + shadcn/ui
- State Management: Zustand + React Query
- Real-time: Socket.io
- Visualization: Plotly.js, Recharts
Before you begin, ensure you have the following installed:
- Python 3.11 or higher
- Node.js 18+ and npm
- Git
- Ollama (for local LLM inference)
# Install Ollama from https://ollama.com
# Pull required models
ollama pull llama3.2
ollama pull nomic-embed-text# Clone the repository
git clone <your-repo-url>
cd daa-chatbot/backend
# Run automated setup script
./scripts/setup.sh
# Start the backend server
source venv/bin/activate
uvicorn api.main:socket_app --reload# Clone the repository
git clone <your-repo-url>
cd daa-chatbot
# Set up backend
cd backend
python -m venv venv
source venv/bin/activate # On Windows: venv\Scripts\activate
pip install -r requirements.txt
# Create .env file
cp .env.example .env # Edit with your settings
# **IMPORTANT: Run database migrations**
alembic upgrade head
# Initialize database with default settings
python scripts/init_db.py
# Verify setup (optional but recommended)
python scripts/check_setup.py
# Run backend server (with WebSocket support)
uvicorn api.main:socket_app --reloadBackend will be available at http://localhost:8000
# In a new terminal
cd frontend
npm install
# Create .env.local file
echo "NEXT_PUBLIC_API_URL=http://localhost:8000
NEXT_PUBLIC_WS_URL=ws://localhost:8000" > .env.local
# Run frontend dev server
npm run devFrontend will be available at http://localhost:3000
cd backend
source venv/bin/activate
# Install development dependencies
pip install -r requirements.txt
# Run database migrations (after model changes)
alembic revision --autogenerate -m "description"
alembic upgrade head
# Verify setup before starting
python scripts/check_setup.py
# Run with hot reload (with WebSocket support)
uvicorn api.main:socket_app --reload --host 0.0.0.0 --port 8000
# Format code
black backend/
# Lint code
pylint backend/
# Run tests
pytestcd frontend
# Install dependencies
npm install
# Run dev server
npm run dev
# Lint and format
npm run lint
npm run format
# Build for production
npm run build
npm startThis project includes VSCode workspace settings for optimal development experience:
- Auto-formatting on save (Black for Python, Prettier for TypeScript)
- ESLint and Pylint integration
- Recommended extensions (see
.vscode/extensions.json)
Install recommended extensions when prompted by VSCode.
Git hooks are configured to run code quality checks before commits:
# Hooks are automatically set up in .git/hooks/pre-commit
# They will check:
# - Python: Black formatting, Pylint linting
# - TypeScript: ESLint, Prettier formatting
# - General: trailing whitespace, file size, merge conflictsdaa-chatbot/
├── backend/ # FastAPI backend
│ ├── api/
│ │ ├── routes/ # API endpoints (chat, documents, projects, llm, analytics)
│ │ ├── websocket/ # WebSocket handlers for real-time chat
│ │ └── main.py # FastAPI app entry point with Socket.IO
│ ├── core/ # Core functionality
│ │ ├── rag_pipeline.py # RAG orchestration
│ │ ├── llm.py # Ollama client wrapper
│ │ ├── vectorstore.py # ChromaDB operations
│ │ ├── embeddings.py # Embedding generation
│ │ └── chunking.py # Text splitting strategies
│ ├── models/ # SQLAlchemy database models
│ ├── services/ # Business logic
│ │ ├── chat_service.py # Conversation management
│ │ ├── document_processor.py # Document text extraction
│ │ ├── project_service.py # Project CRUD operations
│ │ ├── analytics_service.py # Analytics computations (dim reduction, similarity)
│ │ └── file_storage.py # File management
│ ├── crud/ # Database CRUD operations
│ └── storage/ # Data storage (SQLite, ChromaDB, uploads)
├── frontend/ # Next.js frontend
│ └── src/
│ ├── app/ # Next.js app router
│ │ ├── page.tsx # Home page
│ │ ├── layout.tsx # Root layout
│ │ ├── chat/ # Chat pages
│ │ ├── projects/ # Project management
│ │ ├── documents/ # Document management
│ │ └── analytics/ # Embedding analytics dashboard
│ ├── components/ # React components
│ │ ├── ui/ # shadcn/ui components
│ │ ├── chat/ # Chat interface components
│ │ ├── documents/ # Document upload components
│ │ ├── projects/ # Project components
│ │ └── analytics/ # Analytics visualizations
│ │ ├── EmbeddingVisualization.tsx # 2D/3D scatter plots
│ │ ├── SimilarityHeatmap.tsx # Similarity matrix heatmap
│ │ ├── EmbeddingTable.tsx # Data table with search
│ │ └── RetrievalTester.tsx # Query testing interface
│ ├── lib/ # Utilities and API clients
│ │ └── analytics-api.ts # Analytics API client
│ ├── stores/ # Zustand state management
│ └── types/ # TypeScript type definitions
│ └── analytics.ts # Analytics type definitions
└── docker-compose.yml # Docker configuration
OLLAMA_HOST=http://localhost:11434
OLLAMA_MODEL=llama3.2
EMBEDDING_MODEL=nomic-embed-text
DATABASE_URL=sqlite:///./storage/sqlite/app.db
CHROMA_PERSIST_DIR=./storage/chroma
UPLOAD_DIR=./storage/documents
MAX_FILE_SIZE=10485760 # 10MBNEXT_PUBLIC_API_URL=http://localhost:8000
NEXT_PUBLIC_WS_URL=ws://localhost:8000Once the backend is running, visit:
- Swagger UI: http://localhost:8000/docs
- ReDoc: http://localhost:8000/redoc
cd backend
pytest # Run all tests
pytest -v # Verbose output
pytest --cov=backend # With coveragecd frontend
npm test # Run tests
npm run test:watch # Watch mode# Build and run with Docker Compose
docker-compose up --build
# Run in background
docker-compose up -d
# Stop services
docker-compose downNote: Ollama must be running on the host machine. The docker-compose.yml is configured to connect to Ollama via host.docker.internal.
Symptom: API returns 400 error when creating projects, or server fails to start with "Missing tables" error.
Solution:
cd backend
source venv/bin/activate
alembic upgrade headThe server now includes startup validation that will prevent it from starting if migrations haven't been run.
- Ensure Ollama is running:
ollama serve - Check if models are pulled:
ollama list
- Backend (8000): Change port in uvicorn command
- Frontend (3000): Set PORT=3001 in .env.local
- Backend: Ensure virtual environment is activated
- Frontend: Run
npm install
Run the pre-flight check script to verify your backend setup:
cd backend
source venv/bin/activate
python scripts/check_setup.pyThis will check:
- Database tables exist
- Storage directories are created
- Configuration files are present
- Ollama connection (optional)
- Fork the repository
- Create a feature branch
- Make your changes
- Run tests and linters
- Submit a pull request
This project is built on the shoulders of many incredible open source projects. We are grateful to all the maintainers and contributors.
- Ollama - Local LLM inference engine
- FastAPI - Modern, high-performance Python web framework
- Next.js - The React framework for production
- TypeScript - Typed superset of JavaScript
- Uvicorn - Lightning-fast ASGI server
- SQLAlchemy - Python SQL toolkit and ORM
- Alembic - Database migration tool
- ChromaDB - AI-native open-source vector database
- LangChain - Building applications with LLMs
- Pydantic - Data validation using Python type annotations
- python-socketio - Real-time bidirectional communication
- PyPDF - PDF file processing
- python-docx - Microsoft Word document handling
- openpyxl - Excel file processing
- unstructured - Document preprocessing and ETL
- scikit-learn - Machine learning library (PCA, t-SNE)
- UMAP - Dimension reduction and visualization
- tiktoken - Fast BPE tokenizer
- Tailwind CSS - Utility-first CSS framework
- shadcn/ui - Re-usable component collection
- Radix UI - Unstyled, accessible UI primitives
- Lucide React - Beautiful & consistent icon toolkit
- Zustand - Small, fast state management
- TanStack Query - Powerful asynchronous state management
- React Hook Form - Performant, flexible forms
- Zod - TypeScript-first schema validation
- Black - The uncompromising Python code formatter
- Pylint - Python static code analysis
- ESLint - JavaScript and TypeScript linting
- Prettier - Opinionated code formatter
- pytest - Python testing framework
- React Markdown - Markdown component for React
- React Syntax Highlighter - Syntax highlighting component
- React Dropzone - File upload with drag-and-drop
- Framer Motion - Production-ready animation library
- date-fns - Modern JavaScript date utility library
- clsx & tailwind-merge - Utility for constructing className strings