chore: fix bug with addressbook and improve download days v2 retry logic #2009
+1,371
−265
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
rockysingh
force-pushed
the
1845-fix-addressbook-bugs
branch
3 times, most recently
from
24ea9f8 to
f64c33c
Compare
AlfredoG87
reviewed
Jan 9, 2026
Contributor
AlfredoG87
left a comment
AlfredoG87
left a comment
There was a problem hiding this comment.
Looks good but is failing CI Checks
rockysingh
force-pushed
the
1845-fix-addressbook-bugs
branch
from
f64c33c to
eebe7a0
Compare
Signed-off-by: Rocky Thind <[email protected]>
2. Switched from virtual threads to platform threads and from gRPC to HTTP transport because virtual threads pin carrier threads when hitting synchronized blocks in gRPC/Netty, causing deadlocks after ~6 minutes of high-concurrency downloads.
3. DownloadLive2.java - Complete Rewrite
- HTTP transport + V3 manager - Same stable configuration as download-days-v3
- Auto-detect today from mirror node if no state file exists
- Simplified live flow:
- Queries mirror node for latest block → determines current day
- Downloads blocks sequentially with validation
- Writes to .tar.zstd_partial via ConcurrentTarZstdWriter
- Automatic day rollover with archive finalization
- State persistence for resumability
Signed-off-by: Rocky Thind <[email protected]>
2. Switched from virtual threads to platform threads and from gRPC to HTTP transport because virtual threads pin carrier threads when hitting synchronized blocks in gRPC/Netty, causing deadlocks after ~6 minutes of high-concurrency downloads.
3. DownloadLive2.java - Complete Rewrite
- HTTP transport + V3 manager - Same stable configuration as download-days-v3
- Auto-detect today from mirror node if no state file exists
- Simplified live flow:
- Queries mirror node for latest block → determines current day
- Downloads blocks sequentially with validation
- Writes to .tar.zstd_partial via ConcurrentTarZstdWriter
- Automatic day rollover with archive finalization
- State persistence for resumability
Signed-off-by: Rocky Thind <[email protected]>
rockysingh
force-pushed
the
1845-fix-addressbook-bugs
branch
from
fd58771 to
9c69085
Compare
rockysingh
added
the
Block Node Tools
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Reviewer Notes
Rewrites
download-live2command with HTTP transport (instead of gRPC) to avoid deadlock issues, adds inline validation with signature statistics, and implements automatic day rollover with resumable state.Changes
DownloadDaysV3.java- Transport FixConcurrentDownloadManagerVirtualThreadsV3)DownloadLive2.java- Complete RewriteConcurrentDownloadManagerVirtualThreadstoConcurrentDownloadManagerVirtualThreadsV3(platform threads)signature_statistics.csv).tar.zstdarchive at midnight and starts new dayoutputDir/validateCmdStatus.json(compatible with validate commands)LiveDownloader.javaConcurrentDownloadManagerVirtualThreadsV3Documentation
Build
Test plan
download-live2with--start-dateto catch up from a specific date.tar.zstdarchivesvalidate-with-statscan read state file written bydownload-live2