Memory Management for AI Agents

Adapted from the DeepLearning.AI course: Building AI Applications with Memory

Converted from the original Oracle DB + OpenAI stack to a fully local setup using SQLite + ChromaDB + Ollama.

Stack

Component	Original (Course)	Local (This Repo)
Structured Storage	Oracle DB	SQLite
Vector Store	OracleVS	ChromaDB
Embeddings	HuggingFace (sentence-transformers)	Ollama (embeddinggemma:300m)
LLM	OpenAI GPT	Ollama (gemma3:4b) / llama3-groq-tool-use:8b for tool calling
Web Search	Tavily	Tavily

Notebooks

Notebook	Description
app.ipynb	Core memory infrastructure — sets up SQLite tables, ChromaDB vector stores, StoreManager, MemoryManager, Toolbox, and registers all tools (Tavily search, arXiv retrieval, paper ingestion, etc.)
memory_aware_agent.ipynb	Full memory-aware agent loop — context engineering, automatic summarization, tool execution, and multi-turn conversation with persistent memory

Memory Types

Memory Type	Storage	Purpose
Conversational	SQLite	Chat history per thread
Knowledge Base	ChromaDB	Documents, facts, search results
Workflow	ChromaDB	Learned tool execution patterns
Toolbox	ChromaDB	Tool definitions for semantic discovery
Entity	ChromaDB	People, organizations, systems mentioned
Summary	ChromaDB	Compressed conversation summaries
Tool Log	SQLite	Raw tool inputs/outputs for audit

Setup

1. Install dependencies:

   pip install -r requirements.txt

2. Install Ollama models:

   ollama pull embeddinggemma:300m
   ollama pull gemma3:4b
   ollama pull llama3-groq-tool-use:8b

3. Configure API keys in .env:

   TAVILY_API_KEY=tvly-your-key-here
   

   Get a free Tavily key at tavily.com (1,000 searches/month).

4. Run app.ipynb first to build the memory infrastructure and ingest the knowledge base.

5. Run memory_aware_agent.ipynb to use the memory-aware agent.

Architecture

User Query
    ↓
1. BUILD CONTEXT — Read from all 7 memory stores
    ↓
2. CHECK USAGE — Monitor tokens, summarize if >80%
    ↓
3. SELECT TOOLS — Semantic search for relevant tools
    ↓
4. EXECUTE — LLM reasoning + tool calls
    ↓
5. PERSIST — Save conversation, workflow, entities
    ↓
Final Answer