Skip to content

[Feature Request] Add a small "knowledge curation & memory failure modes" doc (docs only) #409

@onestardao

Description

@onestardao

Hi STORM team,

Thank you for releasing STORM. The idea of agents that curate long-term knowledge bases is very powerful and important for real-world use.

I have been working on 16-mode failure maps for RAG and memory-centric agents and recently contributed a robustness-related entry to Harvard MIMS Lab’s ToolUniverse. In systems like STORM, I see recurring classes of problems:

  • knowledge entries are duplicated, inconsistent or partially conflicting
  • the agent’s behavior diverges as the curated memory grows
  • important context from earlier interactions is lost, while irrelevant details are preserved
  • evaluation focuses on short-term answers instead of long-term knowledge quality

I would like to propose a small, docs-only feature.

Proposed feature

Add a short markdown page, for example:

storm_knowledge_and_memory_failure_modes.md

The page could:

  1. List typical failure modes for knowledge curation and long-term memory:
    • duplication and inconsistency
    • drift as memory grows
    • forgetting important constraints.
  2. For each, describe:
    • symptoms
    • likely causes (update policies, retrieval strategies, summarization)
    • basic diagnostics or sanity checks.
  3. Provide a simple checklist for STORM users when they design new experiments or report issues.

Motivation

  • STORM sits at the center of long-term memory and knowledge curation, where failures can be subtle.
  • A small failure-mode guide would help practitioners reason about memory quality and debugging strategies.
  • This is documentation only and should be straightforward to review and maintain.

If this seems in scope for the project, I would be glad to open a PR with a concise first draft.

Thank you for considering.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions