Skip to content

Optimise message expiry#15846

Merged
kjnilsson merged 1 commit intomainfrom
qq-opt-expiry
Mar 27, 2026
Merged

Optimise message expiry#15846
kjnilsson merged 1 commit intomainfrom
qq-opt-expiry

Conversation

@kjnilsson
Copy link
Copy Markdown
Contributor

@kjnilsson kjnilsson commented Mar 26, 2026

Optimise message expiry
Problem
The expire function in rabbit_fifo processed expired messages one at a time in a recursive loop: it would take a single message, call discard_or_dead_letter for that one message, update the state, and then recurse via expire_msgs to check for the next expired message. This meant that if N messages were expired, discard_or_dead_letter was called N times (once per message), and the state record was rebuilt N times.

Similarly, expire_shallow (used by the expire_msgs timer callback) only expired a single returned message from the returns queue before moving on to the messages queue.

Changes
Replace the recursive one-at-a-time expire/expire_msgs loop with a batch-oriented expire_batch that collects all expired messages from both the returns queue and the messages priority queue in one pass, then processes them with a single discard_or_dead_letter call
Extract take_expired_returns/2 to drain all consecutive expired messages from the returns queue (previously only the head was checked)
Extract count_and_size/1 to compute the count and total byte size of expired messages in a single fold
Reuse expire_batch from expire_shallow instead of duplicating the expiry logic
Effect
Reduces the number of discard_or_dead_letter calls from N (one per expired message) to 1 per expiry pass
Reduces intermediate state record allocations from N to 1
Eliminates code duplication between expire_shallow and expire

@kjnilsson kjnilsson requested a review from mkuratczyk March 26, 2026 14:15
@kjnilsson kjnilsson requested a review from ansd March 27, 2026 16:59
@kjnilsson kjnilsson marked this pull request as ready for review March 27, 2026 19:46
@kjnilsson kjnilsson merged commit befb98e into main Mar 27, 2026
353 of 354 checks passed
@kjnilsson kjnilsson deleted the qq-opt-expiry branch March 27, 2026 19:46
ansd added a commit that referenced this pull request Mar 27, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants