scx_p2dq: Add saturation-aware WAKE_SYNC waker CPU handoff #3139
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Implement a fast-path optimization for WAKE_SYNC wakeups that directly assigns wakees to the waker's CPU when the system has capacity. This provides zero-latency handoff for producer-consumer workloads while gracefully degrading at high utilization.
The optimization checks if:
When these conditions are met, the wakee inherits the waker's CPU immediately.
Performance impact (schbench benchmark on 176 CPU system):
Pipe workloads (producer-consumer pairs) see even higher trigger rates with up to 174,000 handoffs/sec at 50% load compared to ~1,000/sec for request-response patterns.
The optimization is placed early in pick_idle_cpu() to take priority over the prev_cpu sticky path, and only activates when beneficial. At saturation, it automatically disables to avoid overhead and allows normal pick-2 load balancing.
Changes:
idle_smtmaskcan_migratechecking LLC min runsTested with schbench and stress-ng across load levels 50-100%.