rabbit_fifo: Add reject_publish_dlx overflow strategy#16673
rabbit_fifo: Add reject_publish_dlx overflow strategy#16673the-mikedavis wants to merge 3 commits into
reject_publish_dlx overflow strategy#16673Conversation
Implement the OSS infrastructure for ingress-dependent quorum queue leader
rebalancing. This adds:
- Per-node enqueue byte counters in the replicated state machine, updated
deterministically on every enqueue via apply_enqueue/6
- Exposure of cumulative totals via overview/1, propagated to all replicas
on every tick
- Aux state leaky integrators (via ra_li) to compute smoothed per-node
ingress rates, updated on handle_tick
- A get_ingress_rates aux query that returns bytes/second per node,
callable locally without cross-node coordination
The new field ingress_bytes_by_node defaults to #{} and is never read by
OSS, so pre-v9 members in mixed clusters are unaffected. The aux state
version is bumped to aux_v5 with an upgrade clause for rolling restarts.
Decay time is configurable via persistent_term.
Tests added to verify accumulation, snapshot persistence, v8→v9 conversion,
and rate decay behavior under controlled tick timing.
|
I couldn't quite dig up the history for why this wasn't implemented, historically. I think it was because it doesn't stem publishers off when the queue is full like the |
This operates just like the classic queue overflow strategy. If enqueueing a message would overflow the queue, the message is run through the dead lettering route and also the enqueue is rejected.
the-mikedavis
force-pushed
the
md/qq-reject-publish-dlx
branch
from
ab26182 to
c1f54b1
Compare
|
Is this PR related to #8495? |
|
Not quite, but the concepts overlap in a way where it's not at all obvious. #8495 is about how |
kjnilsson
force-pushed
the
qq-vNext
branch
4 times, most recently
from
84bb860 to
4409b16
Compare
3 participants
This operates just like the classic queue overflow strategy. If enqueueing a message would overflow the queue, the message is run through the dead lettering route and also the enqueue is rejected.
Prior to this change, QQs would fall back to drop-head when
reject-publish-dlxwas specified.This is based on the next QQ version PR: #16583.