Skip to content

feat: add BatchIndex and more metrics for ObjectStore [ON HOLD]#495

Open
dimazhornyk wants to merge 5 commits intomainfrom
dz-add-batch-index
Open

feat: add BatchIndex and more metrics for ObjectStore [ON HOLD]#495
dimazhornyk wants to merge 5 commits intomainfrom
dz-add-batch-index

Conversation

@dimazhornyk
Copy link
Contributor

@dimazhornyk dimazhornyk commented Oct 9, 2025

PR adds the block_number->batch_number and batch_number->(first_block, last_block) indexes that help reduce the amount of S3 reads and significantly lower the network load.
#205 #454 (adds read/write counter and total amount of downloaded data)

@github-actions
Copy link

github-actions bot commented Oct 9, 2025

Test results

112 tests   112 ✅  18m 40s ⏱️
 17 suites    0 💤
  1 files      0 ❌

Results for commit e0e0e90.

♻️ This comment has been updated with latest results.

@dimazhornyk dimazhornyk changed the title feat: add BatchIndex feat: add BatchIndex and more metrics for ObjectStore Oct 10, 2025
@dimazhornyk dimazhornyk marked this pull request as ready for review October 10, 2025 12:56
@RomanBrodetski
Copy link
Collaborator

@dimazhornyk thanks a lot for your contibution! This is currently on-hold as we are addressing this. Plan to tackle it later this week. This PR will used in some form - I'll reach out if I need any advice!

@RomanBrodetski RomanBrodetski changed the title feat: add BatchIndex and more metrics for ObjectStore feat: add BatchIndex and more metrics for ObjectStore [ON HOLD] Oct 13, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants