-
Notifications
You must be signed in to change notification settings - Fork 2.5k
Open
Description
Hi STORM team,
Thank you for releasing STORM. The idea of agents that curate long-term knowledge bases is very powerful and important for real-world use.
I have been working on 16-mode failure maps for RAG and memory-centric agents and recently contributed a robustness-related entry to Harvard MIMS Lab’s ToolUniverse. In systems like STORM, I see recurring classes of problems:
- knowledge entries are duplicated, inconsistent or partially conflicting
- the agent’s behavior diverges as the curated memory grows
- important context from earlier interactions is lost, while irrelevant details are preserved
- evaluation focuses on short-term answers instead of long-term knowledge quality
I would like to propose a small, docs-only feature.
Proposed feature
Add a short markdown page, for example:
storm_knowledge_and_memory_failure_modes.md
The page could:
- List typical failure modes for knowledge curation and long-term memory:
- duplication and inconsistency
- drift as memory grows
- forgetting important constraints.
- For each, describe:
- symptoms
- likely causes (update policies, retrieval strategies, summarization)
- basic diagnostics or sanity checks.
- Provide a simple checklist for STORM users when they design new experiments or report issues.
Motivation
- STORM sits at the center of long-term memory and knowledge curation, where failures can be subtle.
- A small failure-mode guide would help practitioners reason about memory quality and debugging strategies.
- This is documentation only and should be straightforward to review and maintain.
If this seems in scope for the project, I would be glad to open a PR with a concise first draft.
Thank you for considering.
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
No labels