RAG System with Gemini API and FAISS Vector Database

This project implements a Retrieval-Augmented Generation (RAG) system using the Google Gemini API and a simple FAISS vector database. The system processes PDF documents, creates vector embeddings, and allows semantic search and question answering based on the document content.

Setup

1. Create a virtual environment:

   python -m venv venv

2. Activate the virtual environment:

   • Windows:

     venv\Scripts\activate

   • Linux/Mac:

     source venv/bin/activate

3. Install dependencies:

   pip install -r requirements.txt

4. Add your Google API key to the .env file:

   GOOGLE_API_KEY=your_google_api_key_here
   

Project Structure

• data/ - Directory containing PDF documents
• src/ - Source code
  • rag_gemini.py - Full RAG implementation with Gemini API
  • simple_vector_search.py - Simple vector database search
  • load_vector_db.py - Load a previously saved vector database
  • streamlit_app.py - Streamlit web interface for the RAG system
  • list_models.py - Utility to list available Google AI models

Usage

Streamlit Web Interface

The easiest way to use the RAG system is through the Streamlit web interface:

streamlit run src/streamlit_app.py

This will:

1. Load and process the PDF document
2. Create a vector database of document chunks
3. Provide a chat interface for asking questions about the document content
4. Display the source documents used to generate answers

Full RAG System

To use the full RAG system with question answering:

python src/rag_gemini.py

This will:

1. Load and process the PDF document
2. Create a vector database of document chunks
3. Set up a question-answering chain using Gemini API
4. Allow you to ask questions about the document content

Simple Vector Search

For a simpler approach that just uses vector similarity search:

python src/simple_vector_search.py

This will:

1. Load and process the PDF document
2. Create and save a vector database
3. Allow you to search for relevant document sections

Load Existing Vector Database

If you've already created a vector database:

python src/load_vector_db.py

This loads a previously saved vector database and allows you to perform searches.

How It Works

1. Document Processing: The PDF is loaded and split into smaller chunks.
2. Embedding Generation: Each chunk is converted into a vector embedding using Google's embedding model.
3. Vector Storage: Embeddings are stored in a FAISS vector database for efficient similarity search.
4. Retrieval: When a query is received, the system finds the most relevant document chunks.
5. Generation: For the full RAG system, the retrieved chunks are passed to the Gemini model to generate an answer.

Requirements

• Python 3.8+
• langchain
• langchain_community
• langchain-google-genai
• google-generativeai
• faiss-cpu
• python-dotenv
• pypdf
• sentence-transformers