feat: add monitored cache wrapper and use by ferhatelmas · Pull Request #915 · supabase/storage

ferhatelmas · 2026-03-17T11:33:54Z

What kind of change does this PR introduce?

Feature

What is the current behavior?

Tenant JWK and tenant config caches are bare maps so they can grow unbounded. There is no metrics to observe how much memory used or how many tenants are in memory, etc.

What is the new behavior?

Add a cache wrapper that uses adapter pattern to wrap LRU or TTL caches and emits metrics.
Refactor existing pool TTL cache and JWT/S3 creds LRU caches to this wrapper. Convert barebone maps of JWK and tenant configs to wrapper as well to prevent unbounded growth.

Additional context

To keep purge overhead predictable, each cache registers a periodic stale cleanup and also forces cleanup before metric reporting for accuracy.

This also fixes JWT cache for secret rotation and tenant poisoning because cache is checked first and cache key is token only (excluding tenant secret/jwks).

Tenant pool cache doesn't enable monitoring since it caches live objects with their own monitoring.

One test file is renamed for repo conventions.

Copilot

Pull request overview

Adds a shared LRU-cache wrapper with built-in telemetry (hits/misses/stale, evictions, occupancy) and applies it to previously unbounded tenant/JWKS caches, while also migrating existing JWT and S3-credentials caches to the same abstraction. This fits into the codebase by bounding memory growth of multi-tenant in-memory caches and making cache behavior observable via OTel + Grafana.

Changes:

Introduce @internal/cache (LRU wrapper + cache name constants) and new OTel cache metrics (cache_requests_total, cache_evictions_total, cache_entries, cache_size_bytes).
Replace tenant config/JWKS/JWT/S3-credentials cache implementations with bounded LRU caches + periodic stale purging.
Add tests for eviction behavior and cache telemetry; extend Grafana dashboard with cache panels.

Reviewed changes

Copilot reviewed 14 out of 14 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
src/internal/cache/adapter.ts	Defines cache adapter interfaces and lookup options (incl. `recordMetrics`).
src/internal/cache/lru.ts	Implements LRU wrapper that records request/eviction metrics and emits occupancy gauges.
src/internal/cache/names.ts	Adds typed cache-name constants used as metric labels.
src/internal/cache/index.ts	Exposes cache modules via a barrel export for `@internal/cache`.
src/internal/monitoring/metrics.ts	Registers new cache counters and observable gauges.
src/internal/database/tenant.ts	Replaces unbounded tenant config `Map` with bounded LRU + metric-aware lookups.
src/internal/auth/jwt.ts	Migrates JWT verification cache to the shared LRU wrapper.
src/internal/auth/jwks/manager.ts	Replaces unbounded tenant JWKS config `Map` with bounded LRU + metric-aware lookups.
src/storage/protocols/s3/credentials/manager.ts	Migrates S3 credentials cache to shared LRU wrapper; avoids truthy checks.
src/test/cache-metrics.test.ts	Adds unit tests for request/eviction/occupancy telemetry and stale purging behavior.
src/test/tenant.test.ts	Adds eviction + “logical lookup” cache-request metric tests for tenant config cache.
src/test/tenant-jwks.test.ts	Adds eviction test for tenant JWKS cache.
src/test/tenant-s3-credentials.test.ts	Adds eviction test for oversized cached credentials.
monitoring/grafana/dashboards/storage-otel.json	Adds a Cache row with panels for lookup rate, hit rate, evictions, entries, and size.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot

Pull request overview

Copilot reviewed 19 out of 20 changed files in this pull request and generated 3 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot

Pull request overview

Copilot reviewed 20 out of 21 changed files in this pull request and generated 2 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot

Pull request overview

Copilot reviewed 21 out of 22 changed files in this pull request and generated 5 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot

Pull request overview

Copilot reviewed 21 out of 22 changed files in this pull request and generated 2 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot

Pull request overview

Copilot reviewed 22 out of 23 changed files in this pull request and generated 3 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot

Pull request overview

Copilot reviewed 21 out of 22 changed files in this pull request and generated no new comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot

Pull request overview

Copilot reviewed 22 out of 23 changed files in this pull request and generated 2 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot

Pull request overview

Copilot reviewed 23 out of 24 changed files in this pull request and generated no new comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot

Pull request overview

Copilot reviewed 23 out of 24 changed files in this pull request and generated 1 comment.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

coveralls · 2026-03-27T14:12:55Z

Pull Request Test Coverage Report for Build 23654229206

Details

822 of 849 (96.82%) changed or added relevant lines in 14 files are covered.
6 unchanged lines in 1 file lost coverage.
Overall coverage increased (+0.6%) to 80.199%

Changes Missing Coverage	Covered Lines	Changed/Added Lines	%
src/internal/auth/jwt.ts	54	56	96.43%
src/internal/cache/monitoring.ts	128	130	98.46%
src/test/utils/cache-mock.ts	20	25	80.0%
src/internal/database/pool.ts	31	39	79.49%
src/internal/cache/ttl.ts	172	182	94.51%

Files with Coverage Reduction	New Missed Lines	%
src/http/plugins/signals.ts	6	83.33%

Totals
Change from base Build 23653160443:	0.6%
Covered Lines:	29646
Relevant Lines:	36654

💛 - Coveralls

create an internal cache package with using adapter pattern to wrap LRU or TTL caches. use it for refactoring existing TTL pool cache and JWT/S3 creds LRU caches. use it also for tenant JWKs and configs as LRUs. add metrics and a few panels to show fix double destroy call in pool TTL cache fix caching jwt only by token for rotation Signed-off-by: ferhat elmas <elmas.ferhat@gmail.com>

ferhatelmas requested a review from a team as a code owner March 17, 2026 11:33

Copilot AI review requested due to automatic review settings March 23, 2026 11:39

ferhatelmas force-pushed the ferhat/cache branch from 0fa0a5e to b7021c7 Compare March 23, 2026 11:39

Copilot started reviewing on behalf of ferhatelmas March 23, 2026 11:39 View session

Copilot AI reviewed Mar 23, 2026

View reviewed changes

Comment thread src/test/tenant.test.ts

Comment thread src/internal/cache/lru.ts Outdated

ferhatelmas force-pushed the ferhat/cache branch from b7021c7 to 3cae924 Compare March 23, 2026 12:32

ferhatelmas requested a review from Copilot March 23, 2026 12:45

Copilot started reviewing on behalf of ferhatelmas March 23, 2026 12:46 View session

Copilot AI reviewed Mar 23, 2026

View reviewed changes

Comment thread src/internal/cache/monitoring.ts Outdated

Comment thread src/internal/monitoring/metrics.ts Outdated

Comment thread monitoring/grafana/dashboards/storage-otel.json Outdated

ferhatelmas force-pushed the ferhat/cache branch from 3cae924 to 0f61926 Compare March 23, 2026 13:32

ferhatelmas changed the title ~~feat: add lru cache for jwks and tenants~~ feat: add monitored cache wrapper and use Mar 23, 2026

ferhatelmas force-pushed the ferhat/cache branch 2 times, most recently from b6e8d5b to 7693693 Compare March 23, 2026 13:49

ferhatelmas requested a review from Copilot March 23, 2026 13:49

Copilot started reviewing on behalf of ferhatelmas March 23, 2026 13:49 View session

Copilot AI reviewed Mar 23, 2026

View reviewed changes

Comment thread src/test/jwt.test.ts Outdated

Comment thread src/internal/auth/jwt.ts Outdated

ferhatelmas force-pushed the ferhat/cache branch from 7693693 to fda035a Compare March 23, 2026 14:16

ferhatelmas requested a review from Copilot March 23, 2026 14:16

Copilot started reviewing on behalf of ferhatelmas March 23, 2026 14:17 View session

ferhatelmas force-pushed the ferhat/cache branch from fda035a to 817ad49 Compare March 23, 2026 14:19

Copilot AI reviewed Mar 23, 2026

View reviewed changes

Comment thread src/test/tenant.test.ts Outdated

Comment thread src/test/tenant-s3-credentials.test.ts Outdated

Comment thread src/test/pool-cache.test.ts Outdated

Comment thread src/test/pool-cache.test.ts

Comment thread src/test/jwt.test.ts

ferhatelmas force-pushed the ferhat/cache branch from 817ad49 to dec133b Compare March 23, 2026 14:28

ferhatelmas requested a review from Copilot March 23, 2026 14:28

Copilot started reviewing on behalf of ferhatelmas March 23, 2026 14:28 View session

Copilot AI reviewed Mar 23, 2026

View reviewed changes

Comment thread src/internal/cache/ttl.ts Outdated

Comment thread src/test/pool-cache.test.ts

ferhatelmas force-pushed the ferhat/cache branch from dec133b to 2316d24 Compare March 23, 2026 15:05

ferhatelmas requested a review from Copilot March 23, 2026 15:06

Copilot started reviewing on behalf of ferhatelmas March 23, 2026 15:06 View session

Copilot AI reviewed Mar 23, 2026

View reviewed changes

Comment thread src/internal/cache/lru.ts

Comment thread src/internal/cache/ttl.ts

Comment thread src/test/jwt.test.ts Outdated

ferhatelmas force-pushed the ferhat/cache branch from 2316d24 to 5e61cc0 Compare March 23, 2026 16:13

ferhatelmas force-pushed the ferhat/cache branch from 5e61cc0 to dff2051 Compare March 23, 2026 17:11

ferhatelmas requested a review from Copilot March 23, 2026 17:12

Copilot started reviewing on behalf of ferhatelmas March 23, 2026 17:13 View session

Copilot AI reviewed Mar 23, 2026

View reviewed changes

ferhatelmas force-pushed the ferhat/cache branch from dff2051 to 97c0918 Compare March 23, 2026 20:17

ferhatelmas requested a review from Copilot March 23, 2026 20:21

Copilot started reviewing on behalf of ferhatelmas March 23, 2026 20:21 View session

Copilot AI reviewed Mar 23, 2026

View reviewed changes

Comment thread src/internal/cache/lru.ts

Comment thread src/internal/auth/jwt.ts

ferhatelmas force-pushed the ferhat/cache branch from 97c0918 to be975c9 Compare March 24, 2026 08:21

ferhatelmas requested a review from Copilot March 24, 2026 08:21

Copilot started reviewing on behalf of ferhatelmas March 24, 2026 08:22 View session

Copilot AI reviewed Mar 24, 2026

View reviewed changes

ferhatelmas force-pushed the ferhat/cache branch from be975c9 to 2b2fd96 Compare March 27, 2026 13:45

ferhatelmas requested a review from Copilot March 27, 2026 13:45

Copilot started reviewing on behalf of ferhatelmas March 27, 2026 13:45 View session

Copilot AI reviewed Mar 27, 2026

View reviewed changes

Comment thread src/test/utils/cache-metrics.ts Outdated

ferhatelmas force-pushed the ferhat/cache branch from 2b2fd96 to 36b8ac1 Compare March 27, 2026 13:56

ferhatelmas requested a review from Copilot March 27, 2026 13:57

Copilot started reviewing on behalf of ferhatelmas March 27, 2026 13:57 View session

ferhatelmas force-pushed the ferhat/cache branch from 36b8ac1 to fee41c3 Compare March 27, 2026 15:10

This comment has been minimized.

Sign in to view

ferhatelmas force-pushed the ferhat/cache branch from fee41c3 to 08f7448 Compare March 27, 2026 15:33

fenos approved these changes Mar 27, 2026

View reviewed changes

ferhatelmas merged commit 6d4a5a0 into master Mar 27, 2026
5 checks passed

ferhatelmas deleted the ferhat/cache branch March 27, 2026 16:41

Uh oh!

Conversation

ferhatelmas commented Mar 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What kind of change does this PR introduce?

What is the current behavior?

What is the new behavior?

Additional context

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

coveralls commented Mar 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Pull Request Test Coverage Report for Build 23654229206

Details

💛 - Coveralls

Uh oh!

This comment has been minimized.

Uh oh!

Reviewers

Assignees

ferhatelmas commented Mar 17, 2026 •

edited

Loading

coveralls commented Mar 27, 2026 •

edited

Loading