Skip to content

call hash less often to speed up block processing #1436

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Draft
wants to merge 67 commits into
base: main
Choose a base branch
from
Draft

call hash less often to speed up block processing #1436

wants to merge 67 commits into from

Conversation

@lazynina
Copy link
Member

@lazynina lazynina commented Dec 9, 2024

No description provided.

lazynina and others added 30 commits September 23, 2024 18:50
@lazynina
start by refactoring block index into a read-through cache
fc9a0d3
@lazynina
best chain and header chain with read through cache
f3e8e4c
@lazynina
check bestHeaderChain.ChainMap ONLY when processing pos headers
a2e5b4b
@lazynina
updates for locating headers and faster cache usage
ff6ee24
@lazynina
more fixes
71b1eaa
@lazynina
more fixes
17a56bd
@lazynina
more fix, specifically around .Ancestor
d153927
@lazynina
bump test timeout
47e1958
@lazynina
load all blocks after committed tip in block index
5e5c0c1
@lazynina
fix GetBlockNodeByHashAndHeight - no need for checking height > maxHe…
9a32c26 
…ight
@lazynina
bump timeout to 60m
5359be8
@lazynina
try hashicorp lru cache
89a845e
@lazynina
Fix up some more tests and remove logging
4f7ff12
@lazynina
fix string slice workaround, fix calc next difficulty
722dcc0
@lazynina
use best header chain instead of best chain for calc next difficulty …
1941487 
…target
@lazynina
fix tests
627e882
@lazynina
refactor to just use a block hash for best block chain and best heade…
a469c3a 
…r chain representation
@lazynina
move block index migration to NewBlockchain instead of NewServer, add…
c6ad24a 
… setting committed status for PoW nodes, remove timeout adjustment in test.Dockerfile
@lazynina
merge main into branch
216fd31
@lazynina
Merge branch 'ln/read-through-cache-for-block-index-and-best-chain' i…
1f80f19 
…nto ln/replace-best-chain-with-block-index
@lazynina
handle case where we haven't hit PoS cutover yet in block index migra…
8b73b1c 
…tion
@lazynina
cleanup
7ac67bd
@lazynina
do some sampling for is fully stored and check archival mode
7675765
@lazynina
fix get blocks to store
148bad3
@lazynina
cleanup
8aa6a5b
@lazynina
make a bunch of methods on block index private and expose block index…
53b2d29 
… to backend
@lazynina
don't bother healing orphan pointers if we're syncing
91e22ff
@lazynina
refactor to reduce number of times we have to call hash and don't bot…
9abe238 
…her healing orphan pointers during syncing
@lazynina
upgrade cloudflare/circl
23991c4
@lazynina
Ln/replace best chain plus replace old lru cache (#1414)
8adc624 
* replace decred lru with hashicorp lru

* fix usage of delete -> remove

* fix calc next difficulty target issue

* upgrade cloudflare/circl
lazynina and others added 28 commits October 8, 2024 21:24
@lazynina
fix exit condition for height in get block from best chain by hash
f57a699
@lazynina
allow early return from GetBlockFromBestChainByHash if we're using he…
2bae1a2 
…ader chain and the header is validated if we're syncing
@lazynina
minor enhancement to isTipCurrent
ea243f3
@lazynina
fix txindex issue w/ genesis block
3f73bb6
@lazynina
fix process block pow logic for genesis block
e200407
@lazynina
fix - get next block after tip hash in update process of txindex
3ad68be
@lazynina
reduce direct access to BlockNode.Parent and get rid of heal orphan p…
78034d4 
…ointers since we don't need it
@lazynina
fix postgres get best chain logic
02e9e15
@lazynina
merge in main
dec4675
@lazynina @diamondhands0
Ln/downgrade badger (#1430)
2a82dea 
* downgrade badger, ristretto, and klauspost/compress dependencies

* go mod tidy
@lazynina
resolve conflicts with main
d7b6f93
@lazynina
resolve merge conflicts
453cf51
@lazynina
remove LocateBestBlockChainHeaders and replace with DH suggestion. fl…
35f7a6a 
…ush headers in a batch at the end of each bundle, merge in latest main
@lazynina
make GetUtxoViewAndUtxoOpsAtBlockHash public
69e1825
@lazynina
cleanup
a859c72
@lazynina
change GetBlockFromBestChainByHash to GetBlockFromBestChainByHashAndO…
01cd500 
…ptionalHeight to improve performance when height is available
@lazynina
use GetBlockFromBestChainByHashAndOptionalHeight in GetBlockNodesToFetch
09091df
@lazynina
nuke locator and inventory logic, just return a slice of the tip hash
77a1f57
@lazynina
move check in should verify signatures up in function to reduce amoun…
0585ebc 
…t of times we need to fetch a block node we clearly know isn't there
@lazynina
reduce size of block index cache
139fb2c
@lazynina
use header chain for get blocks to store
4259c39
@lazynina
fix return value from processHeaderPoW
aea51a8
@lazynina
disable deadlock detection around putting block nodes into db after h…
fc63a94 
…andling snapshot
@lazynina
add more logging as we're processing headers after finishing download…
90b5372 
…ing snapshot
@lazynina
iterate backwards to update status of block nodes after finishing hyp…
f9a4f3e 
…ersync
@lazynina
fix set tip call since we reverse backwards through the block nodes a…
f425317 
…t the end of hypersync
@lazynina
small clean up, add some todos
231a6dc
@lazynina
call hash less often to speed up block processing
8eb3e94
Base automatically changed from ln/replace-best-chain-with-block-index to main May 1, 2025 16:04
An error occurred while trying to automatically change base from ln/replace-best-chain-with-block-index to main May 1, 2025 16:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@lazynina