Skip to content

feat(memory): memory decay lifecycle events and cache boundary hints#58

Merged
Siddhant-K-code merged 2 commits into
mainfrom
feat/54-memory-cache-coevolution
May 2, 2026
Merged

feat(memory): memory decay lifecycle events and cache boundary hints#58
Siddhant-K-code merged 2 commits into
mainfrom
feat/54-memory-cache-coevolution

Conversation

@Siddhant-K-code
Copy link
Copy Markdown
Owner

Closes #54

What

Connects pkg/memory's hierarchical decay lifecycle to the cache boundary layer. When the DecayWorker compresses or evicts a memory entry, it now emits a typed event so subscribers (e.g. a CacheBoundaryManager) can retreat the cached prefix before the next request sends stale content. Recall also returns a CacheBoundaryHint derived from recall scores, giving the boundary manager early stability signal without waiting for the normal promotion cycle.

Changes

pkg/memory/cache_events.go (new)

  • MemoryEventType: EventStabilized, EventCompressed, EventEvicted
  • MemoryEvent: Type, EntryID, TokensBefore, TokensAfter, CompressionLevel, OccurredAt
  • MemoryEventHandler callback type (non-blocking contract)
  • CacheBoundaryHint: StableEntryIDs, ConfidenceScore

pkg/memory/store.go

  • RecallResult gains CacheHint *CacheBoundaryHint
  • Store interface gains OnLifecycleEvent(MemoryEventHandler)

pkg/memory/sqlite.go

  • SQLiteStore gains handlers []MemoryEventHandler
  • OnLifecycleEvent appends a handler; emit dispatches to all in order
  • Recall calls buildCacheBoundaryHint — entries with relevance ≥ 0.7 are listed as stable candidates; ConfidenceScore is the mean relevance of all returned entries

pkg/memory/decay.go

  • evictRows (new): queries entries at DecayKeywords level older than cutoff, deletes them, emits EventEvicted per entry
  • decayRows: emits EventCompressed after each UPDATE with TokensBefore, TokensAfter, and CompressionLevel
  • runOnce delegates eviction to evictRows instead of a bare DELETE

pkg/memory/memory_test.go

  • TestLifecycleEvents_Compression: EventCompressed fires with TokensBefore > TokensAfter and correct CompressionLevel
  • TestLifecycleEvents_Eviction: EventEvicted fires with TokensAfter == 0
  • TestRecall_CacheBoundaryHint: near-exact embedding match produces a hint with ≥ 1 stable entry ID and positive confidence score

Why

Without co-evolution, a document that is stable for 24 turns and then compressed from full text to summary causes a cache miss on the next request — the boundary still points to the full-text prefix, which no longer matches. EventCompressed lets the boundary manager retreat before that request is sent. Similarly, EventEvicted prevents the boundary from pointing to content that no longer exists.

The CacheBoundaryHint in RecallResult lets the boundary manager act on turn 1 for high-confidence recalled entries, rather than waiting MinStableTurns pushes to confirm stability independently.

Dependency

Builds on the CacheBoundaryManager introduced in PR #56. The events and hint are emitted regardless of whether a boundary manager is registered — zero overhead when no handlers are attached.

Siddhant-K-code and others added 2 commits May 2, 2026 13:06
…54)

Wire pkg/memory lifecycle transitions into the cache boundary layer via
events and a CacheBoundaryHint on RecallResult.

New file pkg/memory/cache_events.go:
- MemoryEventType: EventStabilized, EventCompressed, EventEvicted
- MemoryEvent: Type, EntryID, TokensBefore, TokensAfter,
  CompressionLevel, OccurredAt
- MemoryEventHandler callback type
- CacheBoundaryHint: StableEntryIDs, ConfidenceScore

pkg/memory/store.go:
- RecallResult gains CacheHint *CacheBoundaryHint
- Store interface gains OnLifecycleEvent(MemoryEventHandler)

pkg/memory/sqlite.go:
- SQLiteStore gains handlers []MemoryEventHandler
- OnLifecycleEvent registers a handler; emit dispatches to all handlers

pkg/memory/decay.go:
- evictRows (new): queries entries to evict, deletes them, emits
  EventEvicted per entry with TokensBefore set
- decayRows: emits EventCompressed after each UPDATE with TokensBefore,
  TokensAfter, and CompressionLevel set

pkg/memory/sqlite.go (Recall):
- buildCacheBoundaryHint: entries with relevance >= 0.7 are treated as
  stable; ConfidenceScore is the mean relevance of all returned entries
- RecallResult.CacheHint populated on every Recall call

Tests (memory_test.go):
- TestLifecycleEvents_Compression: verifies EventCompressed fires with
  correct token counts and compression level
- TestLifecycleEvents_Eviction: verifies EventEvicted fires with
  TokensAfter=0
- TestRecall_CacheBoundaryHint: near-exact embedding match produces a
  hint with at least one stable entry ID

Co-authored-by: Ona <no-reply@ona.com>
@Siddhant-K-code Siddhant-K-code force-pushed the feat/54-memory-cache-coevolution branch from d8fda17 to fe0f3bd Compare May 2, 2026 13:06
@Siddhant-K-code Siddhant-K-code merged commit c172c7b into main May 2, 2026
@Siddhant-K-code Siddhant-K-code deleted the feat/54-memory-cache-coevolution branch May 2, 2026 13:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature] Memory decay and cache boundary co-evolution — align cache_control advancement with memory promotion/demotion lifecycle

1 participant