Add IO concurrency limiting for shard cache reads#2163
Open
Conversation
When querying for `*` (all packages), `fetch_package_records` is called concurrently for every package name. The existing `concurrent_requests_semaphore` already gates HTTP downloads, but cache reads (`tokio_fs::read`) were completely unthrottled — potentially opening thousands of file descriptors simultaneously and hitting the OS "too many open files" limit. Reuse the same semaphore to also gate the cache-read path, so the number of concurrently open shard files is bounded by the same limit as HTTP requests. https://claude.ai/code/session_018WhkrxiAzPjv4ByCqhXKg4
… shard cache reads When querying for `*` all shard cache reads fire concurrently, each opening a file descriptor. Previously the fix incorrectly reused the HTTP `concurrent_requests_semaphore` for IO — conflating two different resource types. Follow the same pattern as `rattler::install::InstallDriver`, which keeps a separate `io_concurrency_semaphore` for file operations: - `GatewayBuilder`: add `max_concurrent_io: MaxConcurrency` field with `with_max_concurrent_io` / `set_max_concurrent_io` builder methods. - `GatewayInner`: store the resulting `io_concurrency_semaphore`. - Thread it through `SubdirBuilder::build_sharded` → `ShardedSubdir::new`. - `ShardedSubdir::fetch_package_records`: acquire the IO permit before `tokio_fs::read` so the number of concurrently open shard files is bounded independently of the HTTP request limit. https://claude.ai/code/session_018WhkrxiAzPjv4ByCqhXKg4
baszalmstra
approved these changes
Mar 4, 2026
…st call sites Two test helpers in sharded_subdir/mod.rs call ShardedSubdir::new directly and were missing the new io_concurrency_semaphore parameter added in the previous commit. https://claude.ai/code/session_018WhkrxiAzPjv4ByCqhXKg4
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
This PR adds a separate concurrency limiter for IO operations (specifically reading shard cache files from disk) to prevent exhausting the OS file-descriptor limit when many packages are queried concurrently.
Previously, only HTTP request concurrency was limited via
max_concurrent_requests. When querying for many packages at once (e.g., with wildcard patterns), the gateway could attempt to open many shard cache files simultaneously, leading to file-descriptor exhaustion.Changes:
max_concurrent_iofield toGatewayBuilderwith builder methodswith_max_concurrent_io()andset_max_concurrent_io()io_concurrency_semaphoretoGatewayInnerto track the IO concurrency limitShardedSubdirto acquire an IO semaphore permit before reading cached shard files from diskSubdirBuilderto pass the IO semaphore toShardedSubdirduring constructionThe IO semaphore is acquired before opening shard cache files, ensuring that concurrent file operations are bounded independently from HTTP request concurrency.
How Has This Been Tested?
The changes follow the existing pattern used for
concurrent_requests_semaphoreand integrate with the existing concurrency control infrastructure. Existing tests should continue to pass as the new semaphore is optional and defaults to unlimited when not configured.Checklist:
https://claude.ai/code/session_018WhkrxiAzPjv4ByCqhXKg4