MCP Server for Agentic Memory

[Atharva-Kanherkar]

Context

The agentic memory system has a complete API surface (20+ endpoints), working forgetting cycle, contradiction detection, and procedure ranking. The next step is an MCP server so Claude Code (or any MCP host) can use the memory system as a tool. The system was designed with three LLM-as-judge decision points that currently use deterministic fallbacks — the MCP server is where the host LLM takes over those decisions.

Architecture Decision

Direct Python, not HTTP proxy. The MCP server imports MemoryAPIService and the serialization helpers from api/app.py directly. No HTTP calls. Same ChromaDB instance, zero overhead.

New Files

File	Purpose
mcp_server/__init__.py	Empty package marker
mcp_server/server.py	FastMCP server with all 12 tool definitions
mcp_server/__main__.py	Entry point: python -m mcp_server
tests/test_mcp_server.py	Offline tests using HashingEmbedder

Modified Files

File	Change
requirements.txt	Add mcp>=1.6.0

No changes to any existing Python source files. All stores, models, services, and the HTTP API remain untouched.

MCP Tools (12 total)

Storage (3 tools)

memory_store_semantic — content, importance=0.5, category="general", domain=None, confidence=1.0, supersedes=None, related_ids=None

• Returns record + potential_contradictions list
• LLM Judge guidance in description: "When potential_contradictions is non-empty, evaluate each candidate. If the existing memory genuinely contradicts the new one, call memory_resolve_supersession with keep_id=new and supersede_id=old. If merely related but not contradictory, no action needed."
• Implementation: construct SemanticMemory, call service.semantic_store.store(), run _safe_contradiction_lookup(), serialize

memory_store_episodic — text, session_id, turn_number=None, summary=None, importance=0.5

• Returns record
• No judge guidance
• Note: file-based episodic upload omitted (MCP tools are JSON-only)

memory_store_procedural — content, steps, preconditions=None, importance=0.5

• Returns record
• Description reminds LLM to report outcomes after use

Retrieval (4 tools)

memory_query — query, top_k=5, memory_types=None

• Returns ranked results with raw_similarity, recency_score, importance_score, final_score

memory_best_procedures — task, top_k=3

• Returns procedural matches with similarity, wilson_score, combined_score
• LLM Judge guidance: "Procedures with zero outcomes are ranked by similarity alone. If top result has zero outcomes but high similarity, consider trying it and reporting the outcome. This helps the system learn."

memory_recent_episodes — n=5

• Returns records newest-first

memory_session_episodes — session_id

• Returns records for a session

Feedback (1 tool)

memory_record_outcome — record_id, success

• Returns updated record with new success/failure counts and wilson_score

Forgetting (3 tools)

memory_forgetting_preview — no params

• Returns full ForgettingReport (dry_run=True)
• LLM Judge guidance: "Review before running. Check 'prune' decisions with reason 'likely_duplicate' — the system uses a deterministic tiebreaker. If the wrong duplicate would be pruned, resolve it manually first with memory_resolve_supersession. Only call memory_forgetting_run after you are satisfied."

memory_forgetting_run — no params

• Returns ForgettingReport (executed)
• Description: "Call memory_forgetting_preview first. This is irreversible."

memory_resolve_supersession — keep_id, supersede_id

• Returns {superseded_id, kept_id, status: "resolved"}
• This is the action endpoint for all three judge decision points

System (1 tool)

memory_overview — no params

• Returns counts by type, recent sessions, latest events

LLM-as-Judge: How It Works in MCP

The MCP server can't call the LLM — the LLM calls the tools. So "judge" guidance is embedded in tool descriptions. The host LLM reads the description, receives structured data in the response, makes a judgment, then calls another tool to act on it.

Three judge flows:

1. Contradiction resolution: memory_store_semantic returns candidates → LLM evaluates content → calls memory_resolve_supersession if real contradiction
2. Duplicate review: memory_forgetting_preview shows duplicate prune decisions → LLM reviews which record should survive → optionally calls memory_resolve_supersession to override → then calls memory_forgetting_run
3. Procedure exploration: memory_best_procedures shows untested procedures → LLM decides whether to try them → reports outcome via memory_record_outcome

Server Structure (mcp_server/server.py)

- sys.path setup (same pattern as demo/cli.py and tests)
- Import MemoryAPIService, serialization helpers, models from api/app.py
- AppContext dataclass holding MemoryAPIService
- app_lifespan async context manager (instantiates service once)
- FastMCP("agentic-memory", lifespan=app_lifespan)
- _svc(ctx) helper to extract service from context
- 12 @mcp.tool() functions, each:
  1. Gets service via _svc(ctx)
  2. Constructs model object
  3. Calls store/service method
  4. Serializes with existing _serialise_* helpers
  5. Returns dict (error cases return {"error": str})

Testing (tests/test_mcp_server.py)

Tests call tool functions directly with a MemoryAPIService constructed with HashingEmbedder. Pattern matches tests/test_api.py:

1. Store + query round-trip (all 3 types)
2. Contradiction detection → resolve flow
3. Procedural outcome tracking → ranking change
4. Forgetting preview (dry run) → run → verify prune
5. Session episode retrieval
6. Overview counts
7. Error cases (missing fields, nonexistent IDs)

Acceptance Criteria

• python -m pytest tests/test_mcp_server.py -v — all tests pass
• python -m mcp_server — server starts on stdio
• Configure in Claude Desktop / Claude Code settings and test manually
• Full suite still passes: python -m pytest tests/ --ignore=tests/benchmark_memory_system.py -q
• Tool descriptions contain LLM-as-judge guidance for contradiction, duplicate, and exploration flows
• No changes to existing Python source files

[Atharva-Kanherkar]

@claude hi