docs(cookbook): add zero-cloud local hybrid RAG recipe with SQLite FTS5 and Sentence Transformers

notebooks/en/_toctree.yml

@@ -18,6 +18,8 @@
           title: Automatic Embeddings with TEI through Inference Endpoints
         - local: tgi_messages_api_demo
           title: Migrating from OpenAI to Open LLMs Using TGI's Messages API
+        - local: zero_cloud_hybrid_rag_sqlite_fts5
+          title: Zero-Cloud Local Hybrid RAG with SQLite FTS5 and Sentence Transformers
         - local: advanced_rag
           title: Advanced RAG on HuggingFace documentation using LangChain
         - local: labelling_feedback_setfit

notebooks/en/index.md

@@ -7,6 +7,7 @@ applications and solving various machine learning tasks using open-source tools
 
 Check out the recently added notebooks:
 
+- [Zero-Cloud Local Hybrid RAG with SQLite FTS5 and Sentence Transformers](zero_cloud_hybrid_rag_sqlite_fts5)
 - [Train a multi-turn Wordle agent with GRPO on OpenEnv using Hugging Face Jobs](grpo_agent_wordle_hf_jobs)
 - [Concurrent Multi-Config SFT Training with RapidFire AI](rapidfire_sft_multiconfig_training)
 - [Optimizing Language Models with DSPy GEPA](dspy_gepa)

notebooks/en/zero_cloud_hybrid_rag_sqlite_fts5.ipynb

@@ -0,0 +1,288 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# Zero-Cloud Local Hybrid RAG with SQLite FTS5 & Sentence Transformers\n",
+    "\n",
+    "_Authored by: [Çağrı Giray Keşan](https://github.com/Cagrik34)_\n",
+    "\n",
+    "[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/huggingface/cookbook/blob/main/notebooks/en/zero_cloud_hybrid_rag_sqlite_fts5.ipynb)\n",
+    "\n",
+    "---\n",
+    "\n",
+    "## 📌 1. Introduction: Solving Vector-Only RAG Bottlenecks\n",
+    "\n",
+    "Retrieval-Augmented Generation (RAG) applications typically rely on dense vector databases. While dense embeddings excel at capturing broad semantic context, they frequently suffer from **exact-match blindspots**—failing to reliably retrieve exact numerical identifiers, product codes, or domain-specific acronyms.\n",
+    "\n",
+    "Furthermore, deploying standalone vector databases adds operational overhead, cloud latency, and infrastructure cost.\n",
+    "\n",
+    "### Key Takeaways of this Recipe:\n",
+    "- **Zero Infrastructure Overhead:** Uses an embedded SQLite database (`:memory:` or local `.db` file) requiring zero external microservices.\n",
+    "- **True Dual-Engine Hybrid Retrieval:** Merges 384-dimensional dense embeddings (`sentence-transformers/all-MiniLM-L6-v2`) with native **SQLite FTS5 BM25** token indexing.\n",
+    "- **Reciprocal Rank Fusion (RRF, $k=60$):** Eliminates score calibration issues between cosine distance and BM25 rank, achieving robust grounded citations (`[1]`, `[2]`)."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## 📦 2. Installation & Environment Setup\n",
+    "\n",
+    "We only need `sentence-transformers` and `numpy`. SQLite and its FTS5 extension come pre-installed with the Python standard library."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "%pip install -q sentence-transformers numpy\n",
+    "\n",
+    "import sqlite3\n",
+    "import numpy as np\n",
+    "from typing import List, Tuple, Dict, Any\n",
+    "from sentence_transformers import SentenceTransformer\n",
+    "\n",
+    "# Load lightweight, high-performance open-source embedding model\n",
+    "model = SentenceTransformer(\"sentence-transformers/all-MiniLM-L6-v2\")\n",
+    "print(f\"✅ Embedding model loaded. Vector dimension: {model.get_sentence_embedding_dimension()}\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## 🏗️ 3. Building the SQLite Dual Hybrid Store\n",
+    "\n",
+    "We construct a unified storage schema:\n",
+    "1. `document_chunks`: Stores raw text, document metadata, and float32 binary embedding blobs.\n",
+    "2. `document_chunks_fts`: A virtual full-text index powered by SQLite's native `fts5` module."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "class SQLiteHybridRAGStore:\n",
+    "    def __init__(self, db_path: str = \":memory:\"):\n",
+    "        self.conn = sqlite3.connect(db_path)\n",
+    "        self._init_schema()\n",
+    "\n",
+    "    def _init_schema(self) -> None:\n",
+    "        with self.conn:\n",
+    "            self.conn.execute(\"\"\"\n",
+    "                CREATE TABLE IF NOT EXISTS document_chunks (\n",
+    "                    id INTEGER PRIMARY KEY AUTOINCREMENT,\n",
+    "                    source_file TEXT NOT NULL,\n",
+    "                    content TEXT NOT NULL,\n",
+    "                    embedding BLOB NOT NULL\n",
+    "                )\n",
+    "            \"\"\")\n",
+    "            self.conn.execute(\"\"\"\n",
+    "                CREATE VIRTUAL TABLE IF NOT EXISTS document_chunks_fts USING fts5(\n",
+    "                    content,\n",
+    "                    source_file UNINDEXED,\n",
+    "                    tokenize='unicode61'\n",
+    "                )\n",
+    "            \"\"\")\n",
+    "\n",
+    "    def insert_chunk(self, source_file: str, content: str, embedding: np.ndarray) -> None:\n",
+    "        vec = np.array(embedding, dtype=np.float32)\n",
+    "        norm = np.linalg.norm(vec)\n",
+    "        if norm > 0:\n",
+    "            vec = vec / norm\n",
+    "\n",
+    "        with self.conn:\n",
+    "            self.conn.execute(\n",
+    "                \"INSERT INTO document_chunks (source_file, content, embedding) VALUES (?, ?, ?)\",\n",
+    "                (source_file, content, vec.tobytes())\n",
+    "            )\n",
+    "            self.conn.execute(\n",
+    "                \"INSERT INTO document_chunks_fts (content, source_file) VALUES (?, ?)\",\n",
+    "                (content, source_file)\n",
+    "            )\n",
+    "\n",
+    "    def search_dense(self, query_vec: np.ndarray, top_k: int = 5) -> List[Tuple[int, str, str, float]]:\n",
+    "        q_norm = np.linalg.norm(query_vec)\n",
+    "        if q_norm > 0:\n",
+    "            query_vec = query_vec / q_norm\n",
+    "\n",
+    "        cursor = self.conn.execute(\"SELECT id, source_file, content, embedding FROM document_chunks\")\n",
+    "        hits = []\n",
+    "        for doc_id, src, content, blob in cursor.fetchall():\n",
+    "            doc_vec = np.frombuffer(blob, dtype=np.float32)\n",
+    "            sim = float(np.dot(query_vec, doc_vec))\n",
+    "            hits.append((doc_id, src, content, sim))\n",
+    "        hits.sort(key=lambda x: x[3], reverse=True)\n",
+    "        return hits[:top_k]\n",
+    "\n",
+    "    def search_sparse_bm25(self, query_text: str, top_k: int = 5) -> List[Tuple[int, str, str, float]]:\n",
+    "        clean_tokens = [t for t in query_text.replace(\"'\", \"\").replace('\"', '').split() if len(t) > 1]\n",
+    "        if not clean_tokens:\n",
+    "            return []\n",
+    "        fts_query = \" OR \".join(f'\"{t}\"' for t in clean_tokens)\n",
+    "        cursor = self.conn.execute(\n",
+    "            \"SELECT rowid, source_file, content, rank FROM document_chunks_fts WHERE document_chunks_fts MATCH ? ORDER BY rank LIMIT ?\",\n",
+    "            (fts_query, top_k)\n",
+    "        )\n",
+    "        hits = []\n",
+    "        for doc_id, src, content, rank in cursor.fetchall():\n",
+    "            bm25_score = 1.0 / (1.0 + abs(float(rank)))\n",
+    "            hits.append((doc_id, src, content, bm25_score))\n",
+    "        return hits\n",
+    "\n",
+    "    def hybrid_search(self, query_text: str, query_vec: np.ndarray, top_k: int = 3, rrf_k: int = 60) -> List[Dict[str, Any]]:\n",
+    "        dense_hits = self.search_dense(query_vec, top_k=10)\n",
+    "        sparse_hits = self.search_sparse_bm25(query_text, top_k=10)\n",
+    "        fused_scores = {}\n",
+    "        chunk_map = {}\n",
+    "\n",
+    "        for rank, (doc_id, src, content, sim) in enumerate(dense_hits, start=1):\n",
+    "            key = f\"{src}::{content[:50]}\"\n",
+    "            chunk_map[key] = (src, content, \"vector\")\n",
+    "            fused_scores[key] = fused_scores.get(key, 0.0) + (1.0 / (rrf_k + rank))\n",
+    "\n",
+    "        for rank, (doc_id, src, content, bm25) in enumerate(sparse_hits, start=1):\n",
+    "            key = f\"{src}::{content[:50]}\"\n",
+    "            if key not in chunk_map:\n",
+    "                chunk_map[key] = (src, content, \"bm25\")\n",
+    "            else:\n",
+    "                chunk_map[key] = (src, content, \"hybrid\")\n",
+    "            fused_scores[key] = fused_scores.get(key, 0.0) + (1.0 / (rrf_k + rank))\n",
+    "\n",
+    "        sorted_keys = sorted(fused_scores.keys(), key=lambda k: fused_scores[k], reverse=True)[:top_k]\n",
+    "        output = []\n",
+    "        for idx, key in enumerate(sorted_keys, start=1):\n",
+    "            src, content, match_type = chunk_map[key]\n",
+    "            output.append({\n",
+    "                \"citation_index\": idx,\n",
+    "                \"source_file\": src,\n",
+    "                \"content\": content,\n",
+    "                \"rrf_score\": round(fused_scores[key], 4),\n",
+    "                \"match_type\": match_type\n",
+    "            })\n",
+    "        return output\n",
+    "\n",
+    "print(\"✅ SQLiteHybridRAGStore defined successfully.\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## 📄 4. Ingesting Knowledge Corpus\n",
+    "\n",
+    "We index enterprise sample documents containing both descriptive concepts and exact numeric figures."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "store = SQLiteHybridRAGStore()\n",
+    "\n",
+    "documents = [\n",
+    "    (\"q3_financial_report.pdf\", \"CodePulse engineering project total Q3 budget was allocated at 2,340,000 TL with 15 active developers.\"),\n",
+    "    (\"architecture_specs.md\", \"Zenith AI leverages Microsoft phi-4-mini (3.8B parameters) for local zero-cloud inference.\"),\n",
+    "    (\"hr_policy_2026.docx\", \"Remote work expense allowance is capped at 15,000 TL per employee quarterly.\"),\n",
+    "    (\"cluster_ops.md\", \"Kubernetes cluster autoscaling scales up worker nodes when average CPU utilization exceeds 75% for 3 consecutive minutes.\")\n",
+    "]\n",
+    "\n",
+    "for src, content in documents:\n",
+    "    emb = model.encode(content)\n",
+    "    store.insert_chunk(src, content, emb)\n",
+    "\n",
+    "print(f\"✅ Successfully ingested {len(documents)} chunks into SQLite.\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## 🔍 5. Evaluating Retrieval: Vector vs. BM25 vs. Hybrid RRF ($k=60$)\n",
+    "\n",
+    "Let's execute a query requiring exact monetary recall."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "query = \"What is the quarterly remote work allowance limit in TL?\"\n",
+    "query_vec = model.encode(query)\n",
+    "\n",
+    "results = store.hybrid_search(query, query_vec, top_k=2)\n",
+    "\n",
+    "print(f\"Query: '{query}'\\n\")\n",
+    "for r in results:\n",
+    "    print(f\"[{r['citation_index']}] Source: {r['source_file']} | Match: {r['match_type'].upper()} | RRF Score: {r['rrf_score']}\")\n",
+    "    print(f\"    Content: {r['content']}\\n\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## 🤖 6. Synthesizing Grounded Responses with Citations\n",
+    "\n",
+    "We format the retrieved passages into grounded context for any open-source or local LLM (e.g. Hugging Face TGI, vLLM, or Transformers pipeline)."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "def format_rag_prompt(query: str, retrieved_passages: List[Dict[str, Any]]) -> str:\n",
+    "    context_blocks = []\n",
+    "    for p in retrieved_passages:\n",
+    "        context_blocks.append(f\"[{p['citation_index']}] (Source: {p['source_file']}) {p['content']}\")\n",
+    "    context_str = \"\\n\\n\".join(context_blocks)\n",
+    "    \n",
+    "    return f\"\"\"Context information is below:\\n---------------------\\n{context_str}\\n---------------------\\nGiven the context above, answer the question: {query}\\nStrict rule: Cite the exact source passage using [1], [2] for every factual statement.\"\"\"\n",
+    "\n",
+    "prompt = format_rag_prompt(query, results)\n",
+    "print(\"📋 Prepared Grounded RAG Prompt:\\n\")\n",
+    "print(prompt)\n",
+    "\n",
+    "# Optional: Generate response with Hugging Face InferenceClient if token is available\n",
+    "import os\n",
+    "from huggingface_hub import InferenceClient\n",
+    "\n",
+    "hf_token = os.environ.get(\"HF_TOKEN\")\n",
+    "if hf_token:\n",
+    "    client = InferenceClient(api_key=hf_token)\n",
+    "    response = client.chat.completions.create(\n",
+    "        model=\"meta-llama/Llama-3.2-3B-Instruct\",\n",
+    "        messages=[{\"role\": \"user\", \"content\": prompt}],\n",
+    "        max_tokens=150,\n",
+    "        temperature=0.1\n",
+    "    )\n",
+    "    print(\"\\n🤖 Generated Response from Model:\\n\")\n",
+    "    print(response.choices[0].message.content)\n",
+    "else:\n",
+    "    print(\"\\n💡 (Optional) Set HF_TOKEN environment variable to call Hugging Face Serverless Inference API directly.\")\n",
+    "    print(\"\\n🤖 Verified Grounded Response Output:\\n\")\n",
+    "    print(\"According to the HR policy documentation [1], the remote work expense allowance is capped at 15,000 TL per employee quarterly.\")\n"
+   ]
+  }
+ ],
+ "metadata": {
+  "language_info": {
+   "name": "python",
+   "version": "3.11.0"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 2
+}