You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The reliability of any long-running system degrades over time: databases accumulate stale indices, software accrues technical debt, and human memory fades with age. Memory-enabled agents are no exception. Even with frozen weights, their system state continues to change as they accumulate context across sessions, and their ability to store, retrieve, and apply knowledge deteriorates in ways that standard snapshot evaluation cannot capture.
18
+
19
+
AgingBench is a longitudinal reliability benchmark suite organized around four aging mechanisms: compression, interference, revision, and maintenance, supporting various scenarios, models, memory policies and agent frameworks, ranging from a fully controlled runner to practical autonomous agents (e.g., Claude Code).
20
+
21
+
We are committed to actively maintaining this repository as a resource for evaluating the longitudinal reliability of AI agents, and we welcome contributions and suggestions from the community.
4
22
5
-
AI agents with persistent memory (Claude Code, ChatGPT, enterprise assistants) are deployed for weeks or months. Over that time they don't just forget — they lose specific values while appearing correct, cite stale facts after updates, confuse similar entities across domains, and silently regress after routine maintenance events. Standard evaluations miss this: an agent can pass every behavioral test while its factual precision has dropped by 50%.
23
+
## 📢 Updates
6
24
7
-
AgingBench measures this degradation across four mechanisms (compression, interference, revision, maintenance), diagnoses *where in the agent's memory infrastructure* each failure originates, and points to *which component* to fix.
25
+
***v0.3.0** — Initial public release (2026-05-13). Eight scenarios (S1–S8) across the four aging mechanisms, AgingCard schema v1.0.0, `agingbench` + `agingbench-lite` CLIs in a single `pip install`, telemetry-mode post-hoc trace analysis. Full log will be updated in [docs/CHANGELOG.md](docs/CHANGELOG.md).
8
26
9
27
---
10
28
@@ -29,15 +47,15 @@ Over time M_t loses facts → scores drop → AgingBench fits the aging curve.
29
47
-**Aging curve `m(t)`** — score vs. session. Half-life = sessions until 50% of capability is lost.
30
48
-**Memory is the independent variable** — same model with different policies produces different aging curves.
31
49
32
-
### The 8 scenarios
50
+
### Latest supporting scenarios
33
51
34
52
| ID | Name | Tier | Sessions | What it tests |
35
53
|----|------|:----:|:--------:|---------------|
36
54
|**S1**| Research Literature | T1 | 8–20 | Fact survival under compression |
Each adapter is a ~100-line Python file that consumes an `aging_card.json` and emits the target system's preferred format. Sample cards in [`prototype/examples/sample_cards/`](prototype/examples/sample_cards/) let you test against without running the benchmark.
161
-
162
-
---
163
-
164
169
## Four aging mechanisms
165
170
166
171
1.**Compression** — write-before-query barrier destroys facts at compaction time
@@ -185,6 +190,8 @@ AgingBench treats memory policy as the independent variable: same model, differe
185
190
186
191
## Citation
187
192
193
+
If you find this work useful, please cite:
194
+
188
195
```bibtex
189
196
@inproceedings{agingbench2026,
190
197
title = {Long-Lived AI Agents Age Too: They Quietly Decay After Deployment},
0 commit comments