Skip to content

Commit e738f08

Browse files
authored
Merge pull request #10 from Kcodess2807/graph-memory
Agent-written graph memory: remember, replace, forget
2 parents 0033372 + 73389a9 commit e738f08

9 files changed

Lines changed: 444 additions & 19 deletions

File tree

‎.github/workflows/ci.yml‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -46,6 +46,7 @@ jobs:
4646
tests/test_github_graph.py
4747
tests/test_eval_grading.py
4848
tests/test_db_integration.py
49+
tests/test_memory.py
4950
tests/test_ingest_github_wiring.py
5051
working-directory: backend
5152

‎README.md‎

Lines changed: 30 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -106,6 +106,33 @@ Matching is lexical, so a reworded question with different words won't match. Re
106106
appear as nodes in the viewer. `backend/scripts/episodic_agent.py` is a working LangGraph agent
107107
that uses this with the graph engine, and `--no-memory` gives you the baseline.
108108

109+
### Graph memory
110+
111+
An agent can write facts into the graph, correct them, and delete them. They sit beside the facts
112+
read from GitHub without ever overwriting them.
113+
114+
```python
115+
from tracerag.memory import AgentMemory
116+
117+
memory = AgentMemory(db, router.embed_query)
118+
memory.remember("checkout-service", "owned by", "payments team", source="run-42")
119+
memory.remember("checkout-service", "owned by", "billing team", source="run-57", replace=True)
120+
memory.forget("checkout-service", "owned by", "billing team")
121+
```
122+
123+
- **Labelled.** Remembered facts reach the model as
124+
`checkout-service <-> billing team: OWNED_BY (remembered, source: run-57)`, and they weigh less
125+
than any fact read from GitHub.
126+
- **Replaced, not lost.** `replace=True` ends the older fact. It stops being retrieved but stays
127+
in the store with the time it ended.
128+
- **Forget is scoped.** `forget()` deletes only agent facts, history included, and removes nodes
129+
it created that are left with no edges. Facts read from GitHub can't be forgotten this way.
130+
- **Over MCP** as `remember` and `forget`, which exist only when `TRACERAG_MCP_WRITE_TOKEN` is set
131+
and check that token on every call.
132+
133+
Known limit: after a replace, the old object's name can still surface as a node with no live
134+
facts, because its ended fact is kept as history.
135+
109136
## What this actually is
110137

111138
Two halves, deliberately separable.
@@ -208,7 +235,7 @@ flowchart TB
208235
| **Embeddings** | `sentence-transformers` · `all-MiniLM-L6-v2` | 384-dim vectors (cosine) |
209236
| **Entity extraction** | **GLiNER**, with **spaCy** as fallback | Zero-shot NER. spaCy runs when GLiNER can't load |
210237
| **Storage** | **LadybugDB** (embedded, Kùzu-lineage) | Single-file hybrid vector + graph store, native HNSW |
211-
| **LLM, router intent** | **Groq** · `llama-3.1-8b-instant` | Query intent classification; falls back to OpenRouter when unset |
238+
| **LLM, router intent** | **Groq** · `qwen/qwen3.8-27b` | Query intent classification; falls back to OpenRouter when unset |
212239
| **LLM, generation** | **OpenRouter** · `anthropic/claude-haiku-5.5` | Grounded answers + node summaries |
213240
| **API** | **FastAPI** + Uvicorn | trace · subgraph · answer · graphs · history |
214241
| **Concurrency** | read connection pool + bounded embed pool | parallel graph/vector reads; embeds off request threads |
@@ -367,7 +394,7 @@ null-byte queries all return normally. A Cypher injection attempt
367394
(`'; MATCH (n) DETACH DELETE n; //`) left all 1274 nodes intact. The parameterized queries
368395
hold.
369396

370-
**Test suite.** 66 tests. Scoring logic runs against a fake store (no driver, no model
397+
**Test suite.** 80 tests. Scoring logic runs against a fake store (no driver, no model
371398
download); `test_db_integration.py` runs the real DDL, migration and `MERGE ... ON MATCH`
372399
clauses through LadybugDB. CI installs `pytest ladybug numpy pandas requests tqdm` (no torch),
373400
so the whole job finishes in seconds.
@@ -455,7 +482,7 @@ Honest accounting of what is finished, what is partial, and what is not measured
455482
| Layer | Status |
456483
|---|---|
457484
| `graphsight` viewer + `graphsight-langgraph` tracer | **Real**, published at 0.3.1, install-verified from PyPI |
458-
| Retrieval engine (ingest → curate → route → trace) | **Real**, 66 tests, live-verified against `fastapi/fastapi` and `pallets/click` |
485+
| Retrieval engine (ingest → curate → route → trace) | **Real**, 80 tests, live-verified against `fastapi/fastapi` and `pallets/click` |
459486
| Structured GitHub ingest (typed, dated edges) | **Real**, exercised against the live API |
460487
| Studio UI (trace canvas, streamed answers, citations, sessions) | **Real**, wired to the live API with an offline sample fallback |
461488
| Multi-tenant pipeline | On the `saas-control-plane` branch, not `main`; see [Multi-tenant mode](#multi-tenant-mode) |

‎backend/mcp_server.py‎

Lines changed: 82 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,5 @@
1-
"""MCP server: semantic, read-only tools over the TraceRAG hybrid engine.
1+
"""MCP server: semantic tools over the TraceRAG hybrid engine. Read-only by default;
2+
remember and forget exist only when TRACERAG_MCP_WRITE_TOKEN is set.
23
34
This is the agent-facing surface of TraceRAG. It exposes the retrieval engine to
45
IDE agents (Cursor, Claude Desktop, etc.) over the Model Context Protocol as a
@@ -24,13 +25,15 @@
2425
from __future__ import annotations
2526

2627
import asyncio
28+
import hmac
2729
import logging
2830
from contextlib import asynccontextmanager
2931
from dataclasses import dataclass
3032
from typing import Callable, Optional
3133

3234
from tracerag import config
3335
from tracerag.db import TraceDB
36+
from tracerag.memory import AgentMemory
3437
from tracerag.router import TraceRouter
3538

3639
logger = logging.getLogger("tracerag.mcp")
@@ -253,6 +256,43 @@ def _search_context_impl(query: str, top_k: int) -> dict:
253256
"result_count": len(passages), "passages": passages}
254257

255258

259+
def _write_refusal(token: str) -> dict | None:
260+
"""None when the call may write. A missing or wrong token is refused."""
261+
expected = config.MCP_WRITE_TOKEN
262+
if not expected:
263+
return {"ok": False, "note": "Memory writes are disabled on this server."}
264+
# ponytail: the token rides in the tool call. Fine for one local agent; move it to
265+
# the transport's auth header before this server faces more than one client.
266+
if not hmac.compare_digest((token or "").encode(), expected.encode()):
267+
return {"ok": False, "note": "Wrong or missing write token."}
268+
return None
269+
270+
271+
def _remember_impl(token: str, subject: str, relation: str, obj: str, replace: bool) -> dict:
272+
refused = _write_refusal(token)
273+
if refused:
274+
return refused
275+
tenant = get_current_tenant()
276+
try:
277+
stored = AgentMemory(tenant.db, tenant.router.embed_query).remember(
278+
subject, relation, obj, source="mcp", replace=replace)
279+
except ValueError as exc:
280+
return {"ok": False, "note": str(exc)}
281+
return {"ok": True, **stored}
282+
283+
284+
def _forget_impl(token: str, subject: str, relation: str, obj: str) -> dict:
285+
refused = _write_refusal(token)
286+
if refused:
287+
return refused
288+
tenant = get_current_tenant()
289+
try:
290+
removed = AgentMemory(tenant.db, tenant.router.embed_query).forget(subject, relation, obj)
291+
except ValueError as exc:
292+
return {"ok": False, "note": str(exc)}
293+
return {"ok": True, "removed": removed}
294+
295+
256296
def _find_entity_impl(name: str, limit: int) -> dict:
257297
tenant = get_current_tenant()
258298
n = (name or "").strip()
@@ -360,6 +400,47 @@ async def find_entity(name: str, limit: int = 5) -> dict:
360400
"""
361401
return await asyncio.to_thread(_find_entity_impl, name, limit)
362402

403+
if config.MCP_WRITE_TOKEN:
404+
405+
@mcp.tool()
406+
async def remember(token: str, subject: str, relation: str, obj: str,
407+
replace: bool = False) -> dict:
408+
"""Store a fact you learned, as subject -RELATION-> object, so later
409+
questions can use it. Example: subject "checkout-service", relation
410+
"OWNED_BY", obj "payments team".
411+
412+
USE THIS WHEN the user tells you something about the codebase that the
413+
repository itself doesn't record: who owns what, a decision, a known
414+
workaround. Don't store what search_context already returns.
415+
416+
Set replace=True when the fact supersedes an older one with the same
417+
subject and relation (ownership moved, a decision changed). The old fact
418+
stops being returned but is kept as history.
419+
420+
Remembered facts come back labelled "(remembered, source: mcp)", and they
421+
never override facts read from GitHub.
422+
423+
Args:
424+
token: The write token this server was configured with.
425+
subject, obj: Entity names. An existing entity is matched by exact
426+
name; a new name becomes a new node.
427+
relation: A short name such as OWNED_BY or DECIDED.
428+
replace: End this subject's other facts with the same relation.
429+
430+
Returns ok, the stored ids, and how many facts were replaced.
431+
"""
432+
return await asyncio.to_thread(_remember_impl, token, subject, relation, obj, replace)
433+
434+
@mcp.tool()
435+
async def forget(token: str, subject: str, relation: str, obj: str) -> dict:
436+
"""Delete a fact that remember stored, history included. Use it when a
437+
remembered fact was wrong rather than out of date (for out of date, use
438+
remember with replace=True). Facts read from GitHub can't be forgotten.
439+
440+
Returns ok and how many facts were removed.
441+
"""
442+
return await asyncio.to_thread(_forget_impl, token, subject, relation, obj)
443+
363444
else: # pragma: no cover — SDK absent
364445
mcp = None
365446

‎backend/tests/test_db_integration.py‎

Lines changed: 6 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -52,7 +52,7 @@ def add(d: TraceDB, node_id, label, ntype, embedding=None, ts=0):
5252

5353
def test_schema_has_the_new_columns(db):
5454
assert "ts" in db.table_columns(config.NODE_TABLE)
55-
assert {"relation", "ts"} <= db.table_columns(config.REL_TABLE)
55+
assert {"relation", "ts", "origin", "ended", "source"} <= db.table_columns(config.REL_TABLE)
5656

5757

5858
def test_fresh_store_needs_no_migration(db):
@@ -89,12 +89,14 @@ def test_pre_03_store_migrates_in_place(tmp_path):
8989
conn.close()
9090
raw.close()
9191

92-
d = TraceDB(store, pool_size=1)
93-
added = d.migrate_schema()
94-
assert set(added) == {
92+
d = TraceDB(store, pool_size=1) # opening is enough: no init_schema call
93+
assert set(d.migrated) == {
9594
f"{config.NODE_TABLE}.ts",
9695
f"{config.REL_TABLE}.relation",
9796
f"{config.REL_TABLE}.ts",
97+
f"{config.REL_TABLE}.origin",
98+
f"{config.REL_TABLE}.ended",
99+
f"{config.REL_TABLE}.source",
98100
}
99101
# the pre-existing rows survived and read back with the column default
100102
rows = d._fetch(

‎backend/tests/test_memory.py‎

Lines changed: 153 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,153 @@
1+
"""Agent-written memory against a real .lbug store: write, replace, forget, and
2+
the line between an agent's facts and the source system's."""
3+
import shutil
4+
5+
import pytest
6+
7+
lb = pytest.importorskip("ladybug", reason="LadybugDB driver not installed")
8+
9+
from tracerag import config # noqa: E402
10+
from tracerag.db import TraceDB # noqa: E402
11+
from tracerag.memory import AgentMemory # noqa: E402
12+
from tracerag.router import TraceRouter # noqa: E402
13+
14+
TEAM_A = [1.0] + [0.0] * (config.EMBED_DIM - 1)
15+
16+
17+
def embed(text: str) -> list[float]:
18+
"""Deterministic stand-in for the real embedder: one hot slot per name."""
19+
v = [0.0] * config.EMBED_DIM
20+
v[sum(map(ord, text)) % config.EMBED_DIM] = 1.0
21+
return v
22+
23+
24+
@pytest.fixture
25+
def db(tmp_path):
26+
store = tmp_path / "m.lbug"
27+
d = TraceDB(store, pool_size=2)
28+
d.init_schema()
29+
d.upsert_node("service:checkout", "checkout-service", "Service", embed("checkout-service"))
30+
d.upsert_node("pr:a/b#7", "PR #7", "PR", embed("PR #7"))
31+
d.add_relationship("pr:a/b#7", "service:checkout", relation=config.RELATION_TOUCHES, ts=1)
32+
d.build_vector_index() # memory must work with the index live, as it is in serving
33+
yield d
34+
d.close()
35+
shutil.rmtree(store, ignore_errors=True)
36+
37+
38+
def router(db: TraceDB) -> TraceRouter:
39+
r = TraceRouter(db)
40+
r._encode = embed
41+
r._get_extractor = lambda: type("E", (), {"extract": lambda self, q: []})()
42+
r._classify_intent = lambda q: (0.15, 0.85, "relational") # no live model in tests
43+
return r
44+
45+
46+
def has_fact(found: list[str], a: str, b: str, relation: str, remembered: bool) -> bool:
47+
"""Facts are undirected, so either end can print first."""
48+
return any(a in f and b in f and f": {relation}" in f and ("remembered" in f) == remembered
49+
for f in found)
50+
51+
52+
def facts(db: TraceDB, question: str) -> list[str]:
53+
return router(db).route(question, top_k=10).facts
54+
55+
56+
def agent_edges(db: TraceDB) -> list[dict]:
57+
return db._fetch(
58+
f"MATCH (a)-[r:{config.REL_TABLE}]->(b) WHERE r.origin = 'agent' "
59+
f"RETURN a.label AS a, r.relation AS rel, b.label AS b, r.ended AS ended, "
60+
f"r.source AS source ORDER BY b.label;")
61+
62+
63+
def test_a_remembered_fact_is_retrieved_and_labelled(db):
64+
AgentMemory(db, embed).remember("checkout-service", "owned by", "payments team", source="run-1")
65+
found = facts(db, "checkout-service")
66+
assert any(f.endswith(": OWNED_BY (remembered, source: run-1)") for f in found)
67+
# the GitHub fact on the same node is still there, and unlabelled
68+
assert has_fact(found, "PR #7", "checkout-service", "TOUCHES", remembered=False)
69+
70+
71+
def test_a_new_name_becomes_a_searchable_node(db):
72+
AgentMemory(db, embed).remember("checkout-service", "owned by", "payments team", source="run-1")
73+
hits = [h["id"] for h in db.vector_search(embed("payments team"), k=3)]
74+
assert "mem:payments-team" in hits
75+
76+
77+
def test_replacing_ends_the_old_fact_but_keeps_its_history(db):
78+
memory = AgentMemory(db, embed)
79+
memory.remember("checkout-service", "owned by", "payments team", source="run-1")
80+
result = memory.remember("checkout-service", "owned by", "billing team",
81+
source="run-2", replace=True)
82+
assert result["replaced"] == 1
83+
found = " ".join(facts(db, "checkout-service"))
84+
assert "billing team" in found and "payments team" not in found
85+
history = {e["b"]: e["ended"] for e in agent_edges(db)}
86+
assert history["payments team"] > 0 and history["billing team"] == 0
87+
# walking from the node whose only fact ended must not crash or resurrect it
88+
assert not any("payments team" in f for f in facts(db, "payments team"))
89+
90+
91+
def test_without_replace_both_facts_stand(db):
92+
memory = AgentMemory(db, embed)
93+
memory.remember("checkout-service", "owned by", "payments team", source="run-1")
94+
memory.remember("checkout-service", "owned by", "billing team", source="run-2")
95+
assert all(e["ended"] == 0 for e in agent_edges(db))
96+
97+
98+
def test_forget_removes_the_fact_and_the_node_it_created(db):
99+
memory = AgentMemory(db, embed)
100+
memory.remember("checkout-service", "owned by", "payments team", source="run-1")
101+
assert memory.forget("checkout-service", "OWNED_BY", "payments team") == 1
102+
assert agent_edges(db) == []
103+
assert "payments team" not in " ".join(facts(db, "checkout-service"))
104+
assert "mem:payments-team" not in [h["id"] for h in db.vector_search(embed("payments team"), k=3)]
105+
106+
107+
def test_forget_cannot_touch_a_github_fact(db):
108+
assert AgentMemory(db, embed).forget("PR #7", "TOUCHES", "checkout-service") == 0
109+
assert has_fact(facts(db, "checkout-service"), "PR #7", "checkout-service", "TOUCHES", remembered=False)
110+
111+
112+
def test_ingest_never_rewrites_an_agent_fact_on_the_same_pair(db):
113+
AgentMemory(db, embed).remember("PR #7", "broke", "checkout-service", source="run-1")
114+
# a stronger API relation on the same pair must not "upgrade" the agent's edge
115+
db.add_relationship("pr:a/b#7", "service:checkout", relation=config.RELATION_AUTHORED, ts=2)
116+
assert [(e["rel"], e["source"]) for e in agent_edges(db)] == [("BROKE", "run-1")]
117+
118+
119+
def test_an_agent_fact_ranks_below_a_github_fact(db):
120+
AgentMemory(db, embed).remember("PR #7", "broke", "checkout-service", source="run-1")
121+
neighbours = db.expand_frontier(["pr:a/b#7"], k=5, max_degree=10)["pr:a/b#7"]
122+
assert [n["origin"] for n in neighbours] == ["api", "agent"]
123+
124+
125+
@pytest.mark.parametrize("subject,relation,obj", [
126+
("", "OWNED_BY", "x"),
127+
("x", "; --", "y"),
128+
("x", "9 lives", "y"),
129+
("x", "", "y"),
130+
("x" * 201, "OWNED_BY", "y"),
131+
])
132+
def test_bad_input_is_refused(db, subject, relation, obj):
133+
with pytest.raises(ValueError):
134+
AgentMemory(db, embed).remember(subject, relation, obj, source="t")
135+
136+
137+
def test_mcp_writes_need_the_configured_token(db, monkeypatch):
138+
import mcp_server
139+
140+
r = router(db)
141+
monkeypatch.setattr(mcp_server, "get_current_tenant", lambda: mcp_server.Tenant(router=r, db=db))
142+
monkeypatch.setattr(config, "MCP_WRITE_TOKEN", None)
143+
assert not mcp_server._remember_impl("anything", "a", "OWNS", "b", False)["ok"]
144+
145+
monkeypatch.setattr(config, "MCP_WRITE_TOKEN", "s3cret")
146+
assert not mcp_server._remember_impl("wrong", "a", "OWNS", "b", False)["ok"]
147+
assert not mcp_server._forget_impl("", "a", "OWNS", "b")["ok"]
148+
assert agent_edges(db) == []
149+
150+
assert mcp_server._remember_impl("s3cret", "checkout-service", "OWNED_BY", "payments team", False)["ok"]
151+
assert agent_edges(db)[0]["source"] == "mcp"
152+
assert mcp_server._forget_impl("s3cret", "checkout-service", "OWNED_BY", "payments team")["removed"] == 1
153+
assert mcp_server._remember_impl("s3cret", "", "OWNED_BY", "x", False)["ok"] is False

‎backend/tracerag/config.py‎

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -68,6 +68,8 @@
6868
# Floor for the semantic fallback when auto-resolving a seed name: below this
6969
# cosine similarity, treat the name as unresolved (tell the agent to use
7070
# find_entity) rather than tracing a spurious weak match.
71+
# write tools (remember, forget) only exist when this is set; every call must pass it
72+
MCP_WRITE_TOKEN = os.getenv("TRACERAG_MCP_WRITE_TOKEN") or None
7173
MCP_RESOLVE_MIN_SIM = float(os.getenv("TRACERAG_MCP_RESOLVE_MIN_SIM", "0.40"))
7274

7375
# The org every request maps to. Single tenant; the multi-tenant control plane
@@ -93,7 +95,8 @@
9395
# groq for the latency-critical router intent call; falls back to openrouter if unset
9496
GROQ_API_KEY = os.getenv("GROQ_API_KEY")
9597
GROQ_BASE_URL = os.getenv("GROQ_BASE_URL", "https://api.groq.com/openai/v1")
96-
GROQ_MODEL = os.getenv("GROQ_MODEL", "llama-3.1-8b-instant")
98+
# must answer in 4 tokens: reasoning models like gpt-oss spend them thinking and return nothing
99+
GROQ_MODEL = os.getenv("GROQ_MODEL", "qwen/qwen3.8-27b")
97100

98101

99102
# --- Clerk auth (backend token verification) ---
@@ -201,6 +204,8 @@
201204
RELATION_CO_OCCURS: 0.35, # proximity is a hint, not a fact
202205
}
203206
DEFAULT_RELATION_WEIGHT = float(os.getenv("TRACERAG_DEFAULT_REL_WEIGHT", "0.5"))
207+
# an agent's own fact: above a proximity guess, below anything read from the API
208+
AGENT_FACT_WEIGHT = float(os.getenv("TRACERAG_AGENT_FACT_WEIGHT", "0.6"))
204209

205210
# Recency: score *= 0.5 ** (age_days / half_life) — at half_life days an item
206211
# keeps half its score. Incidents go stale in weeks, service definitions in a

0 commit comments

Comments
 (0)