Skip to content

Parallelize size statistics gathering#1081

Merged
paraseba merged 1 commit intomainfrom
push-utulkwwkkqny
Jul 16, 2025
Merged

Parallelize size statistics gathering#1081
paraseba merged 1 commit intomainfrom
push-utulkwwkkqny

Conversation

@paraseba
Copy link
Copy Markdown
Collaborator

This adds concurrency to repo_chunks_storage.

I verified manually this improves performance very significantly, but benchmarks should be able to tell us.

@paraseba paraseba requested review from dcherian and mpiannucci July 15, 2025 22:21
@paraseba paraseba force-pushed the push-utulkwwkkqny branch 2 times, most recently from 23d9760 to 25483b0 Compare July 15, 2025 22:45
This adds concurrency to `repo_chunks_storage`.

I verified manually this improves performance very significantly, but
benchmarks should be able to tell us.
@paraseba paraseba force-pushed the push-utulkwwkkqny branch from 25483b0 to f218cd5 Compare July 15, 2025 23:07
)
.await
})
.try_buffered(process_manifests_concurrently.get() as usize)
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yesss!! I was hoping it would be structured instead of spawning!

@paraseba paraseba merged commit f35582b into main Jul 16, 2025
11 checks passed
@paraseba paraseba deleted the push-utulkwwkkqny branch July 16, 2025 02:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants