Skip to content

fix: create the routing cache per router instance so apps can be garbage collected#4930

Merged
provinzkraut merged 2 commits into
litestar-org:mainfrom
uttam12331:fix/per-instance-routing-cache-4876
Jul 15, 2026
Merged

fix: create the routing cache per router instance so apps can be garbage collected#4930
provinzkraut merged 2 commits into
litestar-org:mainfrom
uttam12331:fix/per-instance-routing-cache-4876

Conversation

@uttam12331

@uttam12331 uttam12331 commented Jul 15, 2026

Copy link
Copy Markdown
Contributor

Description

ASGIRouter.handle_routing was decorated with a class-level @lru_cache(1024). A class-level cache is a garbage-collection root: every router instance — and, through router.app, every Litestar app with all of its state — stayed strongly referenced by the cache forever. Long-running processes that create many app instances (most notably test suites) accumulated every app ever created, as reported with a gc-based MCVE in the issue.

This PR binds the cache per instance in __init__:

self.handle_routing = lru_cache(1024)(self._handle_routing)

The cache then only participates in a self-referential cycle (router → cache → bound method → router) that the garbage collector reclaims once the app is no longer externally referenced. Details:

  • The call surface is unchanged: self.handle_routing(path=..., method=...) works as before, and cache_clear() / cache_info() remain available on the per-instance bound cache (the workaround from the issue keeps working).
  • handle_routing is added to __slots__; the uncached implementation lives on as _handle_routing.
  • Per-call overhead is the same one lru_cache wrapper as before.

Verified with the issue's MCVE: on main, one Litestar instance survives del app; gc.collect(); with this change, zero survive. Added test_routing_cache_does_not_prevent_garbage_collection, which fails on main and passes here; the full tests/unit/test_asgi/test_asgi_router.py suite passes (14 passed).

Disclosure per the contribution guidelines: this fix was developed with AI assistance (Claude Code), with the leak and fix verified locally as described above.

Closes

Closes #4876


📚 Documentation preview 📚: https://litestar-org.github.io/litestar-docs-preview/4930

@uttam12331 uttam12331 requested review from a team as code owners July 15, 2026 11:31
@uttam12331 uttam12331 force-pushed the fix/per-instance-routing-cache-4876 branch from 5715259 to 9efe83d Compare July 15, 2026 12:02
ASGIRouter.handle_routing was decorated with a class-level lru_cache, which
acts as a garbage collection root: every router instance (and, through it,
every Litestar app with all of its state) stayed strongly referenced by the
cache forever, so apps were never garbage collected. Long-running processes
that create many app instances (e.g. test suites) accumulated them all.

Bind the cache per instance in __init__ instead
(self.handle_routing = lru_cache(1024)(self._handle_routing)). The cache then
only participates in a self-referential cycle that the garbage collector can
reclaim once the app is no longer referenced. The call surface is unchanged,
including cache_clear()/cache_info() on the bound cache.

Fixes litestar-org#4876
@uttam12331 uttam12331 force-pushed the fix/per-instance-routing-cache-4876 branch from 9efe83d to 7e64206 Compare July 15, 2026 12:27
@codecov

codecov Bot commented Jul 15, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 50.00000% with 1 line in your changes missing coverage. Please review.
⚠️ Please upload report for BASE (main@6d07038). Learn more about missing BASE report.

Files with missing lines Patch % Lines
litestar/_asgi/asgi_router.py 50.00% 1 Missing ⚠️
Additional details and impacted files
@@           Coverage Diff           @@
##             main    #4930   +/-   ##
=======================================
  Coverage        ?   67.30%           
=======================================
  Files           ?      293           
  Lines           ?    15230           
  Branches        ?     1728           
=======================================
  Hits            ?    10251           
  Misses          ?     4832           
  Partials        ?      147           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@provinzkraut provinzkraut enabled auto-merge (squash) July 15, 2026 12:38
@provinzkraut provinzkraut merged commit 2453f45 into litestar-org:main Jul 15, 2026
22 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Bug: ASGIRouter.handle_routing lru_cache decorator keeps strong references to self, preventing app garbage collection

2 participants