Skip to content

epoch: Skip try_advance in collect() when the queue is empty#1299

Open
raphaelroshan wants to merge 1 commit into
crossbeam-rs:mainfrom
raphaelroshan:perf/epoch-skip-advance-empty-queue
Open

epoch: Skip try_advance in collect() when the queue is empty#1299
raphaelroshan wants to merge 1 commit into
crossbeam-rs:mainfrom
raphaelroshan:perf/epoch-skip-advance-empty-queue

Conversation

@raphaelroshan

Copy link
Copy Markdown

Addresses #1001 (and the overhead reported in #852).

Problem

Global::collect() runs on the pin() hot path (every PINNINGS_BETWEEN_COLLECT pins) and unconditionally calls try_advance() first. try_advance() is genuinely expensive: a SeqCst fence plus a full traversal of the intrusive linked list of every registered participant (cache-miss-heavy pointer chasing, as the TODO there notes).

Advancing the epoch is only useful to make queued SealedBags expirable. When the global garbage queue is empty there is nothing to expire and nothing for the subsequent try_pop_if loop to pop, so the entire try_advance cost is wasted. In read-mostly workloads where defer_destroy is rare (the scenario in #1001), this is paid on every periodic collect() and scales O(participants).

Change

Short-circuit collect() when the queue is observed empty, via a new Queue::is_empty(guard) (a single Acquire load of head.next, mirroring the existing test-only helper) — instead of the SeqCst fence + O(participants) traversal.

Why it's sound

Skipping the advance is monotone-conservative: it can only delay epoch advancement, never advance it early, so it can never cause premature reclamation. Liveness is preserved because any thread that pushed a bag is itself on the collect() cadence and will observe a non-empty queue and advance. Worst case is a bounded reclamation delay, in the same spirit as the existing IterError::Stalled early return that already declines to advance.

Verification

  • cargo test -p crossbeam-epoch — lib (44) + doctests (51) green, including collector::tests::pin_holds_advance, incremental, and buffering.
  • ci/crossbeam-epoch-loom.sh (loom, LOOM_MAX_PREEMPTIONS=2) — green.
  • clippy (lib) clean; cargo fmt applied.

The win is deterministic by construction: in the empty-queue case collect() now performs one Acquire load instead of a SeqCst fence plus a traversal of all participants. Happy to add a benchmark or a counted-metric harness if useful.

collect() runs on the pin() hot path every PINNINGS_BETWEEN_COLLECT pins
and unconditionally calls try_advance(), which issues a SeqCst fence and
traverses the intrusive list of every registered participant. Advancing
the epoch is only useful to make queued bags expirable, so when the global
garbage queue is empty the whole traversal is pure overhead -- the common
case in read-mostly workloads where defer_destroy is rare (see crossbeam-rs#1001, crossbeam-rs#852).

Short-circuit collect() when the queue is observed empty. This only delays
epoch advancement, never advances it early, so reclamation stays sound: any
thread that pushed a bag advances the epoch on a later collect(). Adds a
Queue::is_empty(guard) helper mirroring the existing test-only one.
@raphaelroshan
raphaelroshan marked this pull request as ready for review July 15, 2026 06:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

1 participant