feat: add priority pools to prevent high-priority task starvation (#702) #704
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
geirsagberg
commented
Jun 10, 2025
db-scheduler/src/main/java/com/github/kagkarlsson/scheduler/FetchCandidates.java
Show resolved
Hide resolved
geirsagberg
commented
Jun 10, 2025
db-scheduler/src/main/java/com/github/kagkarlsson/scheduler/LockAndFetchCandidates.java
Show resolved
Hide resolved
geirsagberg
force-pushed
the
master
branch
2 times, most recently
from
0d3913e to
b6fd571
Compare
geirsagberg
force-pushed
the
master
branch
3 times, most recently
from
b90c754 to
f33a84d
Compare
…gkarlsson#702) Allows high-priority tasks to execute immediately even when all default threads are busy with lower-priority work. Includes capacity-aware fallback routing and comprehensive test coverage. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]>
Oracle 11g is no longer supported
… worker pools When using worker pools, the polling strategies (FetchCandidates and LockAndFetchCandidates) were calculating fetch limits (lowerLimit and upperLimit) based only on the default executor's thread pool size, while aggregating queue state across all executors. This caused severe underutilization of worker pool threads, especially with the LOCK_AND_FETCH strategy. Changes: - Calculate totalThreads by summing default pool + all worker pools - Pass totalThreads to polling strategy constructors - Rename threadpoolSize parameter to totalThreads for clarity - Add getWorkerPoolThreads() to extract thread count from configs - Add third addWorkerPool() overload for custom ExecutorService with explicit thread count - Update documentation to explain limit calculations - Add test verifying full utilization with LOCK_AND_FETCH strategy Example: With 10 default threads + 5 high-priority threads (15 total), the scheduler now fetches up to 15 tasks instead of 10, allowing all pools to be utilized when high-priority tasks are available.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Allows high-priority tasks to execute immediately even when all default threads are busy with lower-priority work. Includes capacity-aware fallback routing and comprehensive test coverage.
🤖 Generated with Claude Code
Brief, plain english overview of your changes here
As described in #702 we had a need for "fast lane" pools for high priority tasks, so I spent the weekend working with Claude and ended up with a solution that uses multiple executors with separate thread pools. Open for feedback and alternative solutions!
Also, I added guidelines for Claude and also Jetbrains Junie.
Fixes
#702
Reminders
mvn spotless:applycc @kagkarlsson