Skip to content

feat(cache): add unified Cache#7082

Open
tomas-zijdemans wants to merge 8 commits into
denoland:mainfrom
tomas-zijdemans:cache
Open

feat(cache): add unified Cache#7082
tomas-zijdemans wants to merge 8 commits into
denoland:mainfrom
tomas-zijdemans:cache

Conversation

@tomas-zijdemans

@tomas-zijdemans tomas-zijdemans commented Apr 2, 2026

Copy link
Copy Markdown
Contributor

This PR replaces LruCache and TtlCache with a single Cache<K, V> that unifies LRU eviction, TTL expiration, stale-while-revalidate, and load-through (getOrLoad).

The existing caches extend Map, so inherited methods bypass LRU/TTL logic entirely. Both subclass Map, making it structurally impossible to compose LRU with TTL.

Cache<K, V> uses composition over inheritance: it owns a Map for storage and delegates to IndexedHeap for deadline-ordered expiration. Mode is determined by options, not class choice, and a discriminated union on CacheOptions makes illegal combinations compile-time errors.

Beyond fixing the structural issues, this introduces capabilities that didn't exist before:

  • Combined LRU + TTL in a single cache
  • Renames the base type to CacheLike, a more natural type name
  • getOrLoad(key, loader) with automatic in-flight deduplication
  • Stale-while-revalidate with background refresh, error handling, and configurable soft/hard TTL thresholds
  • Per-entry TTL overrides and sliding expiration
  • Single heap-driven timer replacing per-entry setTimeouts
  • Hit/miss/eviction stats, typed ejection reasons, and Symbol.dispose

Since @std/cache is still experimental (0.2.2), this is the right time to make a clean break before stabilization locks things in.

Bonus: The new Cache has significant performance improvements for real-life workflows (e.g. in my testing, write-heavy workloads (set, eviction) are 4-22x faster. hot-key reads are 55x faster thanks to a linked-list)

@codecov

codecov Bot commented Apr 2, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 99.79296% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 94.60%. Comparing base (f0c9f14) to head (7138d78).

Files with missing lines Patch % Lines
cache/cache.ts 99.79% 0 Missing and 1 partial ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #7082      +/-   ##
==========================================
+ Coverage   94.57%   94.60%   +0.02%     
==========================================
  Files         636      635       -1     
  Lines       52138    52398     +260     
  Branches     9399     9464      +65     
==========================================
+ Hits        49311    49572     +261     
  Misses       2249     2249              
+ Partials      578      577       -1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@bartlomieju

Copy link
Copy Markdown
Member

I'll hold on reviewing this one until #7076 lands.

@tomas-zijdemans tomas-zijdemans changed the title feat(cache): add unified Cache BREAKING(cache): add unified Cache Apr 13, 2026

@bartlomieju bartlomieju left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This needs a rebase — there are conflicts with main that need resolving.

Two inline comments below.

Comment thread cache/cache.ts
Comment thread cache/cache.ts
Introduces a single `Cache` class that subsumes both `LruCache` and
`TtlCache` into a configuration-driven API (maxSize, ttl, sliding
expiration, stale-while-revalidate refresh, and onRemove). The new
implementation delegates expiration tracking to `IndexedHeap` from
`@std/data-structures/unstable-indexed-heap` for O(log n) evictions.

- Add cache/cache.ts and cache/cache_test.ts
- Remove cache/lru_cache.ts, cache/ttl_cache.ts and their tests
- Update cache/memoize.ts doc to reference Cache
- Update cache/mod.ts and cache/deno.json exports

Made-with: Cursor
@tomas-zijdemans

tomas-zijdemans commented Apr 22, 2026

Copy link
Copy Markdown
Contributor Author

Thanks,

  • Extracted a new exported CacheOptionsShared<K, V> interface as suggested with maxSize and onRemove. CacheOptionsBase, CacheOptionsTtl, and CacheOptionsSwr now extend it.
  • Defensive guard added in #onTimer as suggested

@tomas-zijdemans

Copy link
Copy Markdown
Contributor Author

Renamed MemoizationCache to CacheLike and moved it to the base file (cache.ts)

@tomas-zijdemans

tomas-zijdemans commented May 26, 2026

Copy link
Copy Markdown
Contributor Author

Not sure if this should have "BREAKING" in the PR title, since the module is not stable 🤷 . Changing it based on the comment here: #7117 (comment)

Ready for review! 😁

@tomas-zijdemans tomas-zijdemans changed the title BREAKING(cache): add unified Cache feat(cache): add unified Cache May 26, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants