Local RAG QA Assistant (LangChain + Ollama)

A 100% local Question Answering (QA) system built with LangChain RAG. This project loads a QA handbook PDF, creates embeddings locally, stores them in FAISS, and answers questions using a local LLM via Ollama.

✔ No OpenAI
✔ No API keys
✔ Works offline
✔ Optimized for Apple Silicon (M1–M4)

---

✨ Features

• PDF ingestion using PyPDFLoader
• Chunking with RecursiveCharacterTextSplitter
• Local embeddings via nomic-embed-text
• Vector search with FAISS
• Local LLM inference with llama3.1
• Optional FAISS index persistence
• Perfect for QA / SDET documentation assistants

---

🧱 Tech Stack

• Python: 3.10 / 3.11
• LangChain: 0.2.16
• LangChain Community: 0.2.16
• Ollama: Local LLM runtime
• FAISS: Vector database
• pypdf: PDF loader

---

📁 Project Structure

.
├─ docs/
│  └─ QA_Handbook.pdf
├─ rag.py
├─ requirements.txt
├─ .gitignore
└─ README.md

---

🧩 Prerequisites

1) Install Ollama (macOS)

brew install ollama

Start the Ollama service:

ollama serve

Pull required models:

ollama pull llama3.1
ollama pull nomic-embed-text

Verify:

ollama list

---

📦 Installation

Create virtual environment

python -m venv .venv
source .venv/bin/activate

Install dependencies

pip install -r requirements.txt

---

📄 requirements.txt (Stable)

langchain==0.2.16
langchain-community==0.2.16
faiss-cpu==1.9.0.post1
pypdf==4.3.1
ollama==0.3.3

---

▶️ Usage

1. Place your handbook PDF:

docs/QA_Handbook.pdf

2. Run the RAG QA app:

python rag.py

3. Ask questions interactively:

What are the release criteria?
What metrics should QA report?
What is the defect lifecycle?

---

🧠 Sample rag.py

from langchain_community.document_loaders import PyPDFLoader
from langchain.text_splitter import RecursiveCharacterTextSplitter
from langchain_community.vectorstores import FAISS
from langchain_community.embeddings import OllamaEmbeddings
from langchain_community.llms import Ollama
from langchain.chains import RetrievalQA

PDF_PATH = "docs/handbook.pdf"
INDEX_DIR = "faiss_index"

# Load document
docs = PyPDFLoader(PDF_PATH).load()

# Split into chunks
splitter = RecursiveCharacterTextSplitter(chunk_size=800, chunk_overlap=120)
chunks = splitter.split_documents(docs)

# Embeddings + Vector Store
embeddings = OllamaEmbeddings(model="nomic-embed-text")
vectorstore = FAISS.from_documents(chunks, embeddings)

# Optional: save index
vectorstore.save_local(INDEX_DIR)

retriever = vectorstore.as_retriever(search_kwargs={"k": 4})

# Local LLM
llm = Ollama(model="llama3.1", temperature=0)

# RAG chain
qa = RetrievalQA.from_chain_type(
    llm=llm,
    retriever=retriever,
    chain_type="stuff"
)

while True:
    question = input("\nAsk a question (or type 'exit'): ").strip()
    if question.lower() in {"exit", "quit"}:
        break
    answer = qa.invoke({"query": question})["result"]
    print("\nAnswer:\n", answer)

---

🧪 Example Questions

• What are QA roles and responsibilities?
• What should be included in a test plan?
• What metrics are used in QA reporting?
• What is the release readiness criteria?

---

🛠 Troubleshooting

ModuleNotFoundError: langchain_community

python -m pip install langchain-community

FAISS version error

Use:

faiss-cpu==1.9.0.post1

Verify imports

python - <<EOF
from langchain.text_splitter import RecursiveCharacterTextSplitter
from langchain_community.document_loaders import PyPDFLoader
print("Imports OK")
EOF

---

🚀 Future Enhancements

• Add citations (page numbers per answer)
• FastAPI backend (/ask)
• Streamlit / Gradio UI
• Multiple PDF support
• Test-case generation from answers

---

📜 License

MIT License