Skip to content

README.md

Latest commit

 

History

History
268 lines (199 loc) · 7.84 KB

README.md

File metadata and controls

268 lines (199 loc) · 7.84 KB
Raw

🚀 RAG Mastery Hub

The Complete Learning Path from Basic RAG to Production Multi-Agent Systems

Master Retrieval-Augmented Generation (RAG) with real-world implementations across all major frameworks: LangChain, AutoGen, CrewAI, LangGraph, and Amazon Bedrock.

Python 3.9+ License: MIT Code style: black

🎯 What You'll Learn

This repository provides a complete learning path for RAG systems, from basic concepts to production-grade multi-agent architectures:

📚 Level 1: Basic RAG (3 Implementations)

  • Simple RAG: Keyword-based retrieval + LLM generation
  • Vector RAG: Semantic search with ChromaDB/Pinecone
  • Production RAG: Circuit breakers, caching, metrics

🔬 Level 2: Advanced RAG Techniques

  • Query Rewriting: Expansion, HyDE, multi-query
  • Reranking: Cross-encoder, LLM-based, fusion (RRF)
  • Chunking: Semantic, recursive, sliding window
  • Knowledge Graphs: Entity extraction, graph reasoning
  • Hybrid Search: BM25 + semantic fusion
  • Multimodal RAG: Text + images + embeddings

🤖 Level 3: Multi-Agent Systems

  • LangChain Agents: Research, analysis, planning agents
  • AutoGen: Conversational agents with group chat
  • CrewAI: Role-based agent crews with task delegation
  • LangGraph: Graph-based workflows with state management
  • Amazon Bedrock: AWS-native agent orchestration

🏭 Level 4: Production Pipelines

  • Ingestion: Batch & streaming document processors
  • Evaluation: RAG metrics, benchmarks, A/B testing
  • Monitoring: Observability, logging, tracing

☁️ Level 5: Cloud Deployments

  • AWS: Lambda, SageMaker, Bedrock
  • GCP: Vertex AI, Cloud Run
  • Azure: OpenAI Service, Container Apps

🚀 Quick Start

Prerequisites

Python 3.9+
pip or conda

Installation

# Clone the repository
git clone https://github.com/KlementMultiverse/rag-mastery-hub.git
cd rag-mastery-hub

# Install dependencies
pip install -r requirements.txt

# Set up environment variables
cp .env.example .env
# Edit .env with your API keys

Run Your First RAG

# Simple RAG (no external dependencies)
python 01_basic_rag/level_1_simple/simple_rag.py

# Vector RAG with ChromaDB
python 01_basic_rag/level_2_vector_db/vector_rag.py

# Production RAG
python 01_basic_rag/level_3_production/production_rag.py

📦 Repository Structure

rag-mastery-hub/
├── 01_basic_rag/              # Basic RAG implementations
│   ├── level_1_simple/        # Keyword-based RAG
│   ├── level_2_vector_db/     # Vector database RAG
│   └── level_3_production/    # Production-ready RAG
├── 02_advanced_rag/           # Advanced techniques
│   ├── query_rewriting/       # Query expansion & HyDE
│   ├── reranking/             # Multiple reranking strategies
│   ├── chunking_strategies/   # Semantic chunking
│   ├── knowledge_graphs/      # Graph-based RAG
│   ├── hybrid_search/         # BM25 + semantic
│   └── multimodal/            # Text + image RAG
├── 03_multi_agent/            # Multi-agent systems
│   ├── langchain/             # LangChain agents
│   ├── autogen/               # AutoGen agents
│   ├── crewai/                # CrewAI agents
│   ├── langgraph/             # LangGraph workflows
│   └── amazon_bedrock/        # AWS Bedrock agents
├── 04_production_pipelines/   # Production systems
│   ├── ingestion/             # Document processing
│   ├── evaluation/            # RAG evaluation
│   └── monitoring/            # Observability
├── 05_cloud_deployments/      # Cloud platforms
│   ├── aws/                   # AWS deployment
│   ├── gcp/                   # GCP deployment
│   └── azure/                 # Azure deployment
├── 06_use_cases/              # Real-world examples
│   ├── customer_support/      # Support chatbot
│   ├── research_assistant/    # Research bot
│   ├── code_assistant/        # Code helper
│   └── legal_assistant/       # Legal document analyzer
└── shared/                    # Shared utilities
    ├── config.py              # Configuration management
    ├── clients/               # API clients (Grok, embeddings)
    └── prompts/               # Prompt templates

🔑 Environment Setup

Create a .env file with your API keys:

# Primary LLM (Grok)
GROK_API_KEY=your_grok_api_key
GROK_BASE_URL=https://api.x.ai/v1

# Vector Databases
PINECONE_API_KEY=your_pinecone_key
CHROMA_PATH=./chroma_db

# Embeddings
OPENAI_API_KEY=your_openai_key
COHERE_API_KEY=your_cohere_key

# Cloud Providers
AWS_ACCESS_KEY_ID=your_aws_key
AWS_SECRET_ACCESS_KEY=your_aws_secret

💡 Key Features

Production-Ready Code

✅ Error Handling: Comprehensive exception management
✅ Type Hints: 100% type hint coverage
✅ SOLID Principles: Clean, maintainable architecture
✅ Logging: Structured logging throughout
✅ Testing: Unit and integration tests
✅ Documentation: Detailed docstrings & examples

Multi-Framework Support

🔧 LangChain: Agent workflows, chains, tools
🔧 AutoGen: Multi-agent conversations
🔧 CrewAI: Role-based agent coordination
🔧 LangGraph: Graph-based state management
🔧 Bedrock: AWS-native agents

Cloud-Native

☁️ AWS: Lambda, SageMaker, Bedrock
☁️ GCP: Vertex AI, Cloud Run
☁️ Azure: OpenAI Service, Container Apps

📚 Documentation

🎓 Learning Path

For Beginners

  1. Start with 01_basic_rag/level_1_simple
  2. Progress to level_2_vector_db
  3. Review level_3_production

For Intermediate

  1. Explore 02_advanced_rag/ techniques
  2. Try multi-agent examples in 03_multi_agent/
  3. Study production pipelines

For Advanced

  1. Deploy to cloud (05_cloud_deployments/)
  2. Build custom use cases (06_use_cases/)
  3. Contribute improvements

🛠️ Tech Stack

Component Technologies
LLMs Grok (xAI), OpenAI, Claude
Embeddings OpenAI, Cohere, HuggingFace
Vector DBs Pinecone, ChromaDB, Weaviate, Qdrant
Frameworks LangChain, AutoGen, CrewAI, LangGraph
Cloud AWS, GCP, Azure
Monitoring LangSmith, OpenTelemetry

🎯 Real-World Use Cases

Customer Support Bot

Answers customer queries using company knowledge base

Research Assistant

Finds and summarizes academic papers

Code Assistant

Helps with code generation and debugging

Legal Document Analyzer

Analyzes legal documents and contracts

🤝 Contributing

Contributions welcome! Please see CONTRIBUTING.md

📄 License

MIT License - see LICENSE for details

🌟 Show Your Support

If this repository helps you learn RAG systems, please ⭐ star it!

👨‍💻 Author

Klement Gunndu

📈 Roadmap

  • Add more cloud deployment examples
  • Add video tutorials
  • Add Jupyter notebooks
  • Add performance benchmarks
  • Add more use cases

Built with ❤️ for the AI community