Welcome to the comprehensive RAG (Retrieval-Augmented Generation) system tutorial! This tutorial will take you from basic concepts to building production-ready RAG systems.
欢迎来到全面的RAG（检索增强生成）系统教程！本教程将带您从基础概念到构建生产就绪的RAG系统。
- Goal / 目标: Understand what RAG is and build a simple implementation / 理解RAG是什么并构建简单实现
- Topics / 主题: Basic concepts, document loading, keyword-based retrieval, LLM integration / 基础概念、文档加载、关键词检索、LLM集成
- Time / 时间: 2-3 hours / 2-3小时
- Prerequisites / 前置要求: Basic Python knowledge / Python基础知识
- Search Method / 搜索方法: Simple keyword matching / 简单关键词匹配
- Goal / 目标: Upgrade from keyword to semantic search using embeddings / 从关键词搜索升级到使用嵌入的语义搜索
- Topics / 主题: Vector embeddings, semantic search, vector databases (ChromaDB), search comparison / 向量嵌入、语义搜索、向量数据库(ChromaDB)、搜索对比
- Time / 时间: 4-5 hours / 4-5小时
- Prerequisites / 前置要求: Complete Level 1 / 完成Level 1
- Search Method / 搜索方法: Embedding-based semantic search / 基于嵌入的语义搜索
- Goal / 目标: Implement advanced retrieval techniques and evaluation / 实现高级检索技术和评估
- Topics / 主题: Hybrid search, reranking, evaluation metrics, query optimization / 混合搜索、重排序、评估指标、查询优化
- Time / 时间: 6-8 hours / 6-8小时
- Prerequisites / 前置要求: Complete Level 2 / 完成Level 2
- Search Method / 搜索方法: Hybrid (BM25 + semantic) with reranking / 混合搜索（BM25 + 语义）加重排序
- Goal / 目标: Build production-ready, scalable RAG systems / 构建生产就绪的可扩展RAG系统
- Topics / 主题: Scalability, monitoring, deployment, security / 可扩展性、监控、部署、安全性
- Time / 时间: 8-10 hours / 8-10小时
- Prerequisites / 前置要求: Complete Level 3 / 完成Level 3
RAG-Tutorial/
├── 01-beginner/ # Level 1: Basic RAG implementation / 基础RAG实现
│ └── rag_01_env/ # Level 1虚拟环境 / Level 1 virtual environment
├── 02-intermediate/ # Level 2: Enhanced with vector DB / 增强向量数据库
│ └── rag_02_env/ # Level 2虚拟环境 / Level 2 virtual environment
├── 03-advanced/ # Level 3: Advanced techniques / 高级技术
│ ├── src/ # Advanced RAG implementations / 高级RAG实现
│ │ ├── hybrid_search.py # Hybrid search (BM25 + semantic) / 混合搜索
│ │ ├── reranker.py # Cross-encoder reranking / 交叉编码器重排序
│ │ ├── evaluator.py # Evaluation metrics / 评估指标
│ │ ├── query_optimizer.py # Query optimization / 查询优化
│ │ └── advanced_rag.py # Complete advanced system / 完整高级系统
│ ├── examples/ # Usage examples and demos / 使用示例和演示
│ │ ├── basic_usage.py # Step-by-step tutorials / 逐步教程
│ │ └── evaluation_demo.py # Evaluation demonstrations / 评估演示
│ ├── theory.md # Advanced RAG theory / 高级RAG理论
│ ├── start_level3.py # Interactive learning guide / 交互式学习指南
│ └── rag_03_env/ # Level 3虚拟环境 / Level 3 virtual environment
├── 04-industrial/ # Level 4: Production-ready system / 生产就绪系统
│ └── rag_04_env/ # Level 4虚拟环境 / Level 4 virtual environment
├── docs/ # Documentation and theory / 文档和理论
├── shared/ # Shared modular RAG components / 共享模块化RAG组件
│ ├── __init__.py # Package initialization / 包初始化
│ ├── base_rag.py # Abstract base class (185 lines) / 抽象基类
│ ├── config.py # Configuration management / 配置管理
│ ├── document_loader.py # Document loading & chunking / 文档加载和分块
│ ├── response_generator.py # LLM response generation / LLM响应生成
│ └── data/ # Sample documents / 示例文档
└── README.md # This file / 本文件
注意：虚拟环境文件夹被git忽略 / Note: Virtual environment folders are git-ignored
为了避免依赖冲突和保持教学清晰，每个教程级别都使用独立的虚拟环境： To avoid dependency conflicts and maintain teaching clarity, each tutorial level uses its own virtual environment:
每个教程级别都有自己的虚拟环境，确保依赖管理清晰和避免版本冲突： Each tutorial level has its own virtual environment to ensure clear dependency management and avoid version conflicts:
# Level 1: 基础RAG实现 / Basic RAG Implementation
cd 01-beginner
python -m venv rag_01_env
source rag_01_env/bin/activate # macOS/Linux
# rag_01_env\Scripts\activate # Windows
pip install -r requirements.txt
# Level 2: 向量数据库增强 / Vector Database Enhancement
cd ../02-intermediate
python -m venv rag_02_env
source rag_02_env/bin/activate # macOS/Linux
# rag_02_env\Scripts\activate # Windows
pip install -r requirements.txt
# Level 3: 高级技术 / Advanced Techniques
cd ../03-advanced
python -m venv rag_03_env
source rag_03_env/bin/activate # macOS/Linux
# rag_03_env\Scripts\activate # Windows
pip install -r requirements.txt
# Level 4: 工业级实现 / Industrial Implementation
cd ../04-industrial
python -m venv rag_04_env
source rag_04_env/bin/activate # macOS/Linux
# rag_04_env\Scripts\activate # Windows
pip install -r requirements.txt💡 教学优势 / Teaching Benefits:
- 每个级别的依赖都很明确 / Clear dependencies for each level
- 避免版本冲突 / Avoid version conflicts
- 学生可以独立管理每个项目 / Students can manage each project independently
- 便于理解项目依赖关系 / Easy to understand project dependencies
Quick Setup / 快速设置:
- Read the Setup Guide for detailed instructions / 阅读设置指南获取详细说明
- For local setup without API keys, see Local Setup / 本地设置无需API密钥，请参见本地设置
- 必须为每个教程创建独立的虚拟环境 / Must create separate virtual environments for each tutorial
学习顺序 / Learning Order:
- 进入
01-beginner/目录，创建虚拟环境并开始学习 / Enter01-beginner/directory, create virtual environment and start learning - 每个级别都建立在前一个级别的基础上，建议按顺序完成 / Each level builds upon the previous one, complete them in order
- 每个新级别都需要激活对应的虚拟环境 / Each new level requires activating the corresponding virtual environment
Start with Level 1 in the 01-beginner/ directory. Each level builds upon the previous one, so it's recommended to complete them in order.
从01-beginner/目录中的Level 1开始。每个级别都建立在前一个级别的基础上，建议按顺序完成。
Important / 重要提醒: This repository does NOT contain any API keys or sensitive information. / 本仓库不包含任何API密钥或敏感信息。
- All API keys are stored in
.envfiles (which are gitignored) / 所有API密钥存储在.env文件中（已被gitignore忽略） - Use the provided
.env.exampleas a template / 使用提供的.env.example作为模板 - Never commit real API keys to version control / 永远不要将真实的API密钥提交到版本控制
The shared components have been refactored into a clean, modular architecture: / 共享组件已重构为清晰的模块化架构：
config.py: Centralized configuration management / 集中配置管理document_loader.py: Document loading and text chunking / 文档加载和文本分块response_generator.py: LLM interactions and embedding generation / LLM交互和嵌入生成base_rag.py: Clean abstract interface using composition / 使用组合的清晰抽象接口
- Single Responsibility: Each module has one clear purpose / 单一职责：每个模块都有明确的目的
- Maintainability: Easier to understand and modify / 可维护性：更容易理解和修改
- Testability: Components can be tested independently / 可测试性：组件可以独立测试
- Reusability: Modules can be used across different RAG implementations / 可重用性：模块可在不同RAG实现中使用
By the end of this tutorial, you'll be able to: / 完成本教程后，您将能够：
- Build RAG systems from scratch / 从零构建RAG系统
- Understand modular architecture design / 理解模块化架构设计
- Choose appropriate vector databases and embedding models / 选择合适的向量数据库和嵌入模型
- Implement advanced retrieval strategies / 实现高级检索策略
- Evaluate and optimize RAG performance / 评估和优化RAG性能
- Deploy RAG systems to production / 将RAG系统部署到生产环境
- Monitor and maintain RAG systems at scale / 大规模监控和维护RAG系统
Let's begin your RAG journey! 🎉 / 让我们开始您的RAG之旅！🎉