ChatPDF Project

ChatPDF is a document-based AI assistant powered by large language models. It allows users to upload PDF files and interact with their content through a chatbot interface. The assistant is capable of retrieving relevant information from the uploaded documents and answering questions related to the content.

Features

• PDF Ingestion: Upload PDF documents to be ingested by the assistant for context-based questioning.
• Chat-based Interaction: Ask questions regarding the content of the uploaded PDFs, and the assistant provides concise answers.
• GPU-Optimized: Uses CUDA-enabled GPUs for fast processing (supports NVIDIA hardware).
• Streamlit Interface: Provides a user-friendly web interface for interacting with the assistant.
• Advanced Vector Search: Uses FAISS to create and query a vector store for fast retrieval of relevant information from documents.
• Custom Llama Model: Leverages the Llama model and embeddings for high-performance text embedding and question answering.

Docker Installation (Using Docker Compose)

To run the ChatPDF project in a Docker environment with GPU support, follow the steps below:

Prerequisites

• Docker and Docker Compose installed.
• An NVIDIA GPU with the NVIDIA Docker runtime enabled.
• NVIDIA Driver and nvidia-container-toolkit.

Important Note on GPU Support in Docker Desktop (Windows)

GPU support will not work under Docker Desktop for Windows because it relies on a virtualization layer, and Windows doesn't natively support CUDA-enabled containers. For GPU support, Docker Desktop for Windows users may need to use a Linux-based system or Windows Subsystem for Linux (WSL 2) with proper GPU pass-through configurations. Or just use the manual install method.

Steps

1. Clone the repository (if you haven't already):

   git clone https://github.com/habib-source/pdf-rag.git
   cd pdf-rag

2. Build and run the Docker container:

   docker-compose up --build

   This will:

   • Build the Docker image based on the Dockerfile.
   • Start the services defined in the docker-compose.yml file.
   • Expose the Streamlit application on port 80.

3. Access the web interface:

   Once the services are up, you can access the ChatPDF interface by navigating to http://localhost in your web browser.

4. Stop the container:

   To stop the container, use:

   docker-compose down

---

Manual Installation (Using Python Virtual Environment)

If you prefer to set up the project manually, follow these instructions:

Prerequisites

• Python 3.12
• A virtual environment manager (venv).
• CUDA 12 and necessary libraries for GPU support.

Steps

1. Clone the repository:

   git clone https://github.com/your-repository-url.git
   cd your-repository-directory

2. Create and activate a Python virtual environment:

   python3 -m venv env
   source env/bin/activate  # For Linux/macOS
   .\env\Scripts\activate  # For Windows

3. Install the dependencies:

   Make sure you have the required dependencies listed in requirements.txt:

   CMAKE_ARGS="-DGGML_CUDA=on" pip install -r requirements.txt

4. Download the models:

   You can manually download the necessary models using the following links:

   • Llama Model
   • Embed Model

   After downloading, you need to rename the models files to match the following names:

   • Rename the Llama model to model.gguf .
   • Rename the Embed model to embed.gguf .

   Place the renamed models (model.gguf and embed.gguf) in your project directory.

5. Run the Streamlit application:

   You can now run the application using:

   streamlit run ui.py

6. Access the web interface:

   Open your browser and go to http://localhost:8501 to interact with the assistant.

7. Stop the application:

   Press Ctrl+C in the terminal to stop the Streamlit app.

---

File Structure

.
├── Dockerfile                  # Dockerfile for setting up the container
├── .dockerignore               # Ignore this list of files when doing the Dockerfile COPY . . command
├── .gitignore                  # Ignore this list of files when doing the git add . command
├── docker-compose.yml          # Docker Compose file to manage services
├── requirements.txt            # Python dependencies
├── ui.py                       # Streamlit application
├── loed.py                  	# Core logic for PDF ingestion and Q&A
├── embed.gguf                  # Llama Embedding Model
├── model.gguf                  # Llama Model
└── README.md                   # Project documentation (this file)

---

Troubleshooting

• Error: nvidia-docker not found: Ensure that Docker is set up correctly with the NVIDIA runtime. You may need to install the NVIDIA Container Toolkit.

---

Contributing

We welcome contributions! Please fork the repository, create a feature branch, and submit a pull request with your changes. If you have any issues or feature requests, feel free to open an issue on GitHub.

---