RAG Document Q&A Assistant

An AI-powered question-answering assistant that lets users upload PDF documents and ask natural-language questions about their contents, using Retrieval-Augmented Generation (RAG). Answers are strictly grounded in the uploaded documents — if information isn't present, the assistant says so explicitly rather than guessing.

Architecture

flowchart LR
    A[User uploads PDF] --> B[PyMuPDF: extract text per page]
    B --> C[LangChain: RecursiveCharacterTextSplitter]
    C --> D[HuggingFace Embeddings: all-MiniLM-L6-v2]
    D --> E[(ChromaDB: persistent vector store)]

    F[User asks question] --> G{Follow-up?}
    G -->|Yes, has history| H[Groq LLM: rewrite as standalone question]
    G -->|No, first turn| I[Use question as-is]
    H --> J[Semantic search: top-k chunks]
    I --> J
    E -.retrieval.-> J
    J --> K[Groq LLM: answer strictly from retrieved chunks]
    K --> L[Return answer + deduplicated sources]

Loading

Flow in plain terms: PDFs are parsed page-by-page (preserving page numbers for citations), split into overlapping ~900-character chunks, embedded into vectors, and stored in ChromaDB. When a question comes in, recent conversation history is used to rewrite follow-up questions into standalone ones (so "what is his email?" becomes "what is John Smith's email?"), the rewritten question is used to retrieve the most relevant chunks, and an LLM (Groq's Llama 3.3 70B) generates an answer using only those retrieved chunks — never its own general knowledge.

Tech Stack

• Python 3.11+, FastAPI, Uvicorn
• LangChain (text splitting, embeddings, vector store, LLM integration)
• ChromaDB — persistent local vector storage
• sentence-transformers (all-MiniLM-L6-v2) — embeddings
• Groq (Llama 3.3 70B Versatile) — free-tier LLM for answer generation and query rewriting
• PyMuPDF — PDF text extraction with page-level metadata
• Streamlit — frontend UI
• Docker / Docker Compose — containerized deployment

Project Structure

QA_RAG/
├── app/
│   ├── main.py                  # FastAPI app, middleware, global error handler
│   ├── config.py                # Environment/settings management
│   ├── logging_config.py        # Structured logging setup
│   ├── models/
│   │   └── schemas.py           # Pydantic request/response models
│   ├── routers/
│   │   ├── documents.py         # upload, list, delete endpoints
│   │   └── chat.py               # chat endpoint
│   ├── services/
│   │   ├── pdf_extractor.py     # PyMuPDF extraction + LangChain Document conversion
│   │   ├── text_splitter.py     # LangChain RecursiveCharacterTextSplitter wrapper
│   │   ├── vector_store.py      # ChromaDB + embeddings wrapper
│   │   ├── document_registry.py # JSON-backed document metadata store
│   │   └── rag_chain.py         # Retrieval, query rewriting, grounded answer generation
│   └── utils/
│       └── validation.py        # Upload validation, filename sanitization
├── streamlit_app/
│   └── app.py                   # Streamlit frontend
├── tests/                       # pytest unit tests
├── data/                        # PDFs, ChromaDB storage, document registry (persistent)
├── Dockerfile.api
├── Dockerfile.streamlit
├── docker-compose.yml
└── requirements*.txt

Setup Instructions

Option A: Local (without Docker)

# 1. Clone and enter the project
cd QA_RAG

# 2. Create and activate a virtual environment (Python 3.11 or 3.12)
python3.11 -m venv .venv
source .venv/bin/activate

# 3. Install dependencies
pip install -r requirements.txt

# 4. Set up your API key
cp .env.example .env
# Edit .env and add your GROQ_API_KEY (get one free at https://console.groq.com)

# 5. Run the backend
uvicorn app.main:app --reload --port 8000

# 6. In a second terminal, run the frontend
source .venv/bin/activate
streamlit run streamlit_app/app.py

Visit http://localhost:8501 for the UI, or http://localhost:8000/docs for interactive API documentation.

Option B: Docker

cp .env.example .env
# Add your GROQ_API_KEY to .env

docker compose up --build

Visit http://localhost:8501 for the UI, or http://localhost:8000/docs for the API.

Note: Docker setup is provided and builds successfully, covering both the API and Streamlit containers with persistent volumes for uploaded data and vector storage. First build may take several minutes due to ML dependency downloads.

API Documentation

POST /documents/upload

Uploads and indexes a PDF document.

Request: multipart/form-data, field name file (PDF only, max 20MB)

Response (201):

{
  "document_id": "uuid-string",
  "filename": "policy.pdf",
  "page_count": 5,
  "chunk_count": 8,
  "message": "Document uploaded and indexed successfully."
}

Errors: 400 (non-PDF, empty file, oversized file), 422 (corrupted/unreadable PDF)

GET /documents

Lists all uploaded documents.

Response (200):

{
  "documents": [
    {"document_id": "uuid", "filename": "policy.pdf", "page_count": 5, "chunk_count": 8}
  ],
  "total": 1
}

DELETE /documents/{document_id}

Deletes a document: removes its chunks from ChromaDB, its PDF from disk, and its registry entry.

Response (200): {"document_id": "uuid", "message": "Document deleted successfully."} Errors: 404 if the document doesn't exist

POST /chat

Asks a question about the uploaded documents.

Request:

{
  "question": "What is the refund policy?",
  "session_id": "optional-uuid",
  "document_id": "optional-uuid, restricts search to one document"
}

Response (200):

{
  "answer": "The refund policy allows cancellation within 30 days.",
  "sources": [{"document": "policy.pdf", "page": 4}],
  "session_id": "uuid-string"
}

If session_id is omitted, one is generated and returned — reuse it in subsequent requests to maintain conversation history (last 5 exchanges, used to resolve follow-up questions like "what is his email?").

Errors: 400 (no documents uploaded), 404 (invalid document_id filter), 503 (LLM service temporarily unavailable)

Assumptions & Limitations

• No OCR: scanned/image-only PDFs with no text layer will be rejected with a 422 error, since PyMuPDF cannot extract text that isn't there.
• Conversation history is in-memory only — it is explicitly non-persistent and is lost on server restart, per design (documented, not a bug).
• Cross-document retrieval by default: questions search across all uploaded documents unless a document_id is explicitly provided to scope the search to one file.
• Single-file upload per request: multiple files require multiple upload calls (the Streamlit UI handles this automatically in a loop).
• Free-tier Groq rate limits apply — heavy concurrent testing may occasionally hit a temporary rate limit.

Testing

pytest tests/ -v

Covers: document registry CRUD behavior and persistence, and text chunking (splitting, overlap, blank-chunk prevention, metadata preservation, page-boundary respect).

Manual end-to-end test matrix (all verified during development): valid PDF upload, multiple PDF uploads, non-PDF rejection, empty file rejection, oversized file rejection, in-document question, out-of-document question ("cannot be found" response), pronoun follow-up resolution, document listing, document deletion, post-deletion query behavior, and server-restart persistence (both ChromaDB and the document registry).