A full-stack, production-ready Retrieval-Augmented Generation (RAG) application that lets you upload PDF documents and ask questions about them using Google Gemini AI. Built with enterprise-grade security features including PII detection, injection prevention, and real-time security monitoring.
- Overview
- Features
- Architecture
- Tech Stack
- Getting Started
- Configuration
- API Reference
- Security
- Testing
- Project Structure
- Troubleshooting
LLM Document Agent enables intelligent question-answering over your PDF documents. Upload any PDF, ask natural language questions, and get contextual answers grounded in your document content — with full source attribution showing which page the answer came from.
The system uses a RAG pipeline:
- PDFs are parsed and split into overlapping text chunks
- Chunks are embedded into a vector database (Chroma) using Google Gemini embeddings
- At query time, semantically similar chunks are retrieved and passed to the LLM as context
- Gemini 2.0 Flash generates a grounded, accurate response
Conversation history is maintained so you can ask follow-up questions naturally.
- PDF Upload & Management — Upload, list, and delete documents via a clean web UI
- Semantic Search — Vector similarity search over document chunks using Chroma
- RAG-Powered Q&A — Contextual answers from Google Gemini, grounded in your documents
- Source Attribution — Every answer links back to the source document and page number
- Conversation Memory — Multi-turn conversations with maintained chat history
- Embedding Fallback — Automatically falls back to HuggingFace embeddings if Gemini quota is exceeded
- PII Detection — Identifies emails, phone numbers, SSNs, and credit card numbers using Microsoft Presidio (with regex fallback)
- Injection Prevention — Detects and blocks SQL injection, prompt injection, and XSS attacks in user input
- Output Content Filtering — Redacts API keys, passwords, and tokens from LLM responses before they reach the user
- Security Audit Logging — All security incidents are logged with timestamps for review
- Security Dashboard — Real-time monitoring UI showing incident counts, PII detections, and recent events
- Auto-generated API Docs — Swagger UI at
/docsand ReDoc at/redoc - Docker Support — One-command deployment with Docker Compose
- Makefile Shortcuts — Common tasks available as
makecommands - Comprehensive Test Suite — Pytest with coverage reporting
┌─────────────────────────────────────────────────────────┐
│ Frontend (React + Vite) │
│ ┌──────────────┐ ┌──────────────┐ ┌───────────────┐ │
│ │ DocumentUpload│ │ChatInterface │ │SecurityDashbrd│ │
│ └──────┬───────┘ └──────┬───────┘ └───────┬───────┘ │
└─────────┼─────────────────┼──────────────────┼──────────┘
│ │ │ HTTP/REST
▼ ▼ ▼
┌─────────────────────────────────────────────────────────┐
│ Backend (FastAPI) │
│ │
│ ┌────────────────────────────────────────────────────┐ │
│ │ Security Layer │ │
│ │ PII Detector │ Input Validator │ Content Filter │ │
│ └────────────────────────────────────────────────────┘ │
│ │ │
│ ┌───────────────────────┼───────────────────────────┐ │
│ │ RAG Agent (LangChain) │ │
│ │ │ │ │
│ │ ┌────────────────────┴─────────────────────┐ │ │
│ │ │ Document Processor │ │ │
│ │ │ PDF Parsing → Chunking → Embedding │ │ │
│ │ └────────────────────┬─────────────────────┘ │ │
│ │ │ │ │
│ │ ┌────────────────────┴─────────────────────┐ │ │
│ │ │ Vector Store (Chroma) │ │ │
│ │ │ Persist & Retrieve Semantic Embeddings │ │ │
│ │ └──────────────────────────────────────────┘ │ │
│ └───────────────────────────────────────────────────┘ │
│ │ │
│ Google Gemini API │
│ (Embeddings + Text Generation) │
└─────────────────────────────────────────────────────────┘
- Upload: User uploads a PDF → FastAPI saves it →
DocumentProcessorparses and chunks the text → chunks embedded via Gemini → stored in Chroma - Query: User sends a question → Security layer validates input → RAG agent retrieves top-k relevant chunks → Gemini generates answer with source citations → Content filter scans output → Response returned to UI
| Layer | Technology |
|---|---|
| Frontend | React 18, Vite, Axios, Lucide React |
| Backend | Python 3.9+, FastAPI, Uvicorn |
| LLM | Google Gemini 2.0 Flash (langchain-google-genai) |
| Embeddings | Gemini Embeddings (HuggingFace fallback) |
| Vector Store | ChromaDB 0.4 |
| LLM Orchestration | LangChain 0.1 |
| PDF Parsing | PyPDF 4.0 |
| PII Detection | Microsoft Presidio (spaCy en_core_web_lg) |
| Data Validation | Pydantic |
| Testing | pytest, pytest-cov |
| Containerization | Docker, Docker Compose |
- Python 3.9 or higher
- Node.js 18 or higher
- Google Gemini API Key — Get one free at aistudio.google.com
1. Clone the repository
git clone <repository-url>
cd llm-document-agent2. Run the setup script
This creates a Python virtual environment, installs all backend and frontend dependencies, and downloads the required spaCy NLP model.
bash setup.sh3. Configure your API key
cp .env.example backend/.env
# Open backend/.env and set your key:
# GEMINI_API_KEY=your_key_here4. Start the application
bash start.shThe application will be available at:
| Service | URL |
|---|---|
| Frontend | http://localhost:5173 |
| Backend API | http://localhost:8000 |
| Swagger Docs | http://localhost:8000/docs |
| ReDoc | http://localhost:8000/redoc |
1. Configure environment
cp .env.example .env
# Set GEMINI_API_KEY in .env2. Start with Docker Compose
bash start-docker.sh
# or
make docker-start3. Stop containers
make docker-stopmake setup # Run initial setup
make start # Start locally
make stop # Stop local services
make test # Run test suite with coverage
make clean # Remove venvs and caches
make install # Install dependencies only
make docker-start # Start with Docker
make docker-stop # Stop Docker containersAll configuration is managed via environment variables. Copy .env.example to backend/.env and adjust as needed.
| Variable | Default | Description |
|---|---|---|
GEMINI_API_KEY |
(required) | Your Google Gemini API key |
GEMINI_MODEL |
models/gemini-2.5-flash |
LLM model to use for generation |
CHROMA_PERSIST_DIRECTORY |
./data/vectorstore |
Path to persist the Chroma vector DB |
DOCUMENTS_DIRECTORY |
./data/documents |
Path to store uploaded PDF files |
LOGS_DIRECTORY |
./data/logs |
Path to store security event logs |
MAX_FILE_SIZE_MB |
10 |
Maximum allowed upload file size |
ALLOWED_EXTENSIONS |
pdf |
Comma-separated list of allowed file extensions |
CHROMA_TELEMETRY |
false |
Enable/disable Chroma anonymized telemetry |
DEBUG |
true |
Enable debug mode |
| Method | Endpoint | Description |
|---|---|---|
POST |
/api/upload |
Upload a PDF document |
GET |
/api/documents |
List all uploaded documents |
DELETE |
/api/documents/{filename} |
Delete a document and its embeddings |
| Method | Endpoint | Description |
|---|---|---|
POST |
/api/query |
Submit a question (with security checks) |
GET |
/api/stats |
Get system statistics (docs, queries, incidents) |
GET |
/api/health |
Health check endpoint |
| Method | Endpoint | Description |
|---|---|---|
GET |
/api/security/logs |
Retrieve security incident logs (last 50 by default) |
POST |
/api/reset-embeddings |
Reset to Gemini embeddings after quota recovery |
{
"question": "What is the main topic of the document?",
"document_name": "optional-specific-document.pdf"
}{
"answer": "The document covers...",
"sources": [
{ "document": "report.pdf", "page": 3 }
],
"security_flags": [],
"conversation_id": "abc123"
}Full interactive API documentation is available at /docs when the backend is running.
This application implements a multi-layer security architecture to protect both users and the system.
All user queries pass through InputValidator before reaching the LLM:
- SQL Injection — Detects
DROP TABLE,SELECT *,UNION SELECT, and similar patterns - Prompt Injection — Detects attempts to override system instructions (e.g., "ignore previous instructions")
- XSS — Detects
<script>,javascript:, and similar vectors
Flagged queries are blocked and logged before any LLM processing occurs.
The PIIDetector scans both user inputs and LLM outputs for:
- Email addresses
- Phone numbers
- Social Security Numbers
- Credit card numbers
Detection uses Microsoft Presidio (with spaCy NER) when available, with a regex-based fallback. Detections are flagged in the response and logged as security incidents.
ContentFilter scans LLM responses and redacts any accidental exposure of:
- API keys and tokens
- Passwords and credentials
- Private keys
The frontend Security Dashboard provides:
- Total incident count and PII detection count
- Breakdown of incident types
- Timestamped log of recent security events
All security incidents are written to data/logs/security_events.jsonl with the following structure:
{
"timestamp": "2024-01-15T10:30:00Z",
"event_type": "pii_detected",
"severity": "medium",
"details": { "pii_types": ["EMAIL"] }
}Run the full test suite with coverage:
bash run-tests.sh
# or
make testThis runs all tests under backend/tests/ and generates an HTML coverage report at backend/htmlcov/index.html.
To run tests directly with pytest:
cd backend
source venv/bin/activate
pytest tests/ -v --cov=. --cov-report=htmlllm-document-agent/
├── backend/
│ ├── agents/
│ │ ├── rag_agent.py # Core RAG pipeline (retrieval + generation)
│ │ └── prompt_templates.py # System and user prompt templates
│ ├── services/
│ │ ├── document_processor.py # PDF parsing and text chunking
│ │ ├── vectorstore.py # Chroma vector DB wrapper
│ │ ├── embedding_fallback.py # Gemini/HuggingFace embedding with fallback
│ │ ├── llm_service.py # LLM service abstraction
│ │ └── telemetry_shim.py # Chroma telemetry compatibility shim
│ ├── security/
│ │ ├── pii_detector.py # PII detection (Presidio + regex)
│ │ ├── input_validator.py # Injection and XSS detection
│ │ └── content_filter.py # Output sensitive data redaction
│ ├── tests/ # pytest test suite
│ ├── app.py # FastAPI application and route definitions
│ ├── config.py # Pydantic settings (reads from .env)
│ └── requirements.txt
│
├── frontend/
│ ├── src/
│ │ ├── App.jsx # Root component with sidebar navigation
│ │ ├── components/
│ │ │ ├── ChatInterface.jsx # Q&A chat UI
│ │ │ ├── DocumentUpload.jsx # Upload and document list management
│ │ │ ├── MessageBubble.jsx # Individual chat message rendering
│ │ │ └── SecurityDashboard.jsx # Security metrics and incident log
│ │ └── services/
│ │ └── api.js # Axios API client (all backend calls)
│ ├── package.json
│ └── vite.config.js
│
├── data/
│ ├── documents/ # Uploaded PDFs (persisted)
│ ├── vectorstore/ # Chroma vector database (persisted)
│ └── logs/ # Security event logs (persisted)
│
├── docker-compose.yml
├── Makefile
├── setup.sh
├── start.sh
├── start-docker.sh
├── run-tests.sh
└── .env.example
Gemini API quota exceeded
The embedding service will automatically fall back to HuggingFace sentence-transformers. Once your quota resets, call POST /api/reset-embeddings to switch back to Gemini embeddings.
spaCy model not found
If PII detection fails with a model error, run:
python -m spacy download en_core_web_lgChroma version conflicts
This project pins chromadb==0.4.22. Do not upgrade without testing, as the Chroma API changed significantly in 0.5+.
Port already in use
Check for existing processes on ports 8000 (backend) and 5173 (frontend):
lsof -i :8000
lsof -i :5173Docker: environment variables not picked up
Ensure your .env file is in the project root (not backend/.env) when running Docker Compose, as docker-compose.yml reads from the project root.