fix(channels): make subscribe cancellation-safe to prevent subscriber leak#4872
Open
lesnik512 wants to merge 5 commits into
Open
fix(channels): make subscribe cancellation-safe to prevent subscriber leak#4872lesnik512 wants to merge 5 commits into
lesnik512 wants to merge 5 commits into
Conversation
… leak ChannelsPlugin.subscribe registered the new subscriber into self._channels synchronously, before awaiting backend.subscribe and put_subscriber_history. The history fetch performs real I/O (e.g. redis xrevrange), so a cancellation there raised CancelledError without returning the subscriber: it stayed registered in _channels, unreachable by the caller and never unsubscribed, and the backend stayed subscribed. Under bursty connect/disconnect (e.g. SSE clients that disconnect mid-subscribe) _channels grew without bound until restart. Fetch history before registering the subscriber so a cancellation discards an unregistered subscriber instead of leaking it. start_subscription is fixed by the same change, since subscribe now leaves nothing behind on cancel. Closes litestar-org#4871 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #4872 +/- ##
=======================================
Coverage 67.28% 67.28%
=======================================
Files 293 293
Lines 15226 15226
Branches 1727 1727
=======================================
Hits 10245 10245
Misses 4834 4834
Partials 147 147 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Author
|
@sobolevn Hi! Is it OK now? |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
ChannelsPlugin.subscriberegisters the new subscriber intoself._channelssynchronously, before itawaitsbackend.subscribeandput_subscriber_history. The history fetch performs real I/O (e.g. Redisxrevrange), so a cancellation there raisesCancelledErrorwithout returning the subscriber — it stays registered in_channels, unreachable by the caller (which never received it) and so neverunsubscribed, while the backend stays subscribed.It surfaces with SSE / long-lived subscriptions on ASGI servers that cancel the request task when the client disconnects. A client that connects and disconnects while
subscribeis still fetching history leaks a subscriber every time; under bursty connect/disconnect (reconnect storms)_channelsand the backend's subscribed-channel set grow without bound — rising memory and, for the Redis stream backend, rising CPU asXREADkeeps scanning orphaned channels. Cleared only by a restart.start_subscriptioninherited the flaw: it doessubscriber = await self.subscribe(...)and only enters itstry/finally(which unsubscribes) aftersubscribereturns, so a cancellation insidesubscribeskipped the cleanup.Fix
Fetch history before registering the subscriber.
put_subscriber_historyis the only real cancellation window here, so doing it first means a cancellation discards an unregistered subscriber instead of leaking it.start_subscriptionneeds no separate change —subscribenow leaves nothing behind on cancel, and there is noawaitbetweensubscribereturning and thetry, so the context manager is safe too.Behavior note
History is now enqueued strictly before any live event (previously live events could interleave ahead of history). The trade-off is a small window: events published between the history snapshot and
backend.subscribecompleting are captured by neither — comparable to the existing history-vs-live ordering races.Tests
Added
test_subscribe_cancellation_does_not_leakandtest_start_subscription_cancellation_does_not_leak, using aMemoryChannelsBackendsubclass whoseget_historysuspends (standing in for any backend whose history fetch awaits I/O). Each cancelssubscribe/start_subscriptionmid-history and asserts no subscriber is left in_channelsand the backend is not left subscribed. Both fail onmainand pass with the fix; the fulltest_plugin.pysuite remains green. No external services required.Closes
Closes #4871
📚 Documentation preview 📚: https://litestar-org.github.io/litestar-docs-preview/4872