Skip to content

Latest commit

 

History

History
54 lines (39 loc) · 1.92 KB

README.md

File metadata and controls

54 lines (39 loc) · 1.92 KB

RAG Pipeline for Website Question Answering

A simple Retrieval-Augmented Generation (RAG) pipeline for answering questions based on website content. This project combines retrieval of relevant website information with generative models to deliver contextually accurate answers.

Features

  • Website Content Parsing: Extracts and indexes content from a specified website URL.
  • Retrieval-Augmented Generation (RAG): Uses a hybrid approach for question answering.
  • Streamlit Interface: User-friendly web interface for inputting URLs and questions.

Tools

  • Orchestration: LangChain
  • Guardrails: NeMo-Guardrails
  • Monitoring: Langsmith
  • Retrieval: Hugging Face
  • Vector Database: Pinecone
  • Generation: Groq, Llama3
  • Deployment: Render

Workflow Architecture

  1. Data Loading: Accepts various data formats such as PDF, CSV, or URL links.
  2. Chunking: Uses text splitters to divide content into manageable chunks.
  3. Embedding: Generates embeddings from text using Hugging Face models.
  4. Vector Store: Stores embeddings in Pinecone for efficient retrieval.
  5. User Query (Input): Receives the user's question.
  6. Input Check: Uses Guardrails (NeMo-Guardrails) to ensure the question is within the scope of the dataset.
  7. Retriever: Retrieves relevant chunks from Pinecone based on the user’s query.
  8. Generator: Groq or Llama3 generates responses based on retrieved information.
  9. Monitoring: Langsmith monitors responses to ensure output quality.
  10. Output: The final answer is presented to the user.

Getting Started

Prerequisites

  • Python 3.9+

Run

  • Install dependencies: 
    pip install -r requirements.txt
  • Start the app 
    streamlit run app.py

Usage

  1. Enter a Website URL in the sidebar.
  2. Ask Questions based on the website content.
  3. The RAG pipeline retrieves relevant information and generates answers.