Skip to content

Diamondhands/block sync refactor wip #1434

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 51 commits into
base: main
Choose a base branch
from
Open

Diamondhands/block sync refactor wip #1434

wants to merge 51 commits into from

Conversation

@diamondhands0
Copy link
Member

No description provided.

lazynina and others added 30 commits September 23, 2024 18:50
@lazynina
start by refactoring block index into a read-through cache
fc9a0d3
@lazynina
best chain and header chain with read through cache
f3e8e4c
@lazynina
check bestHeaderChain.ChainMap ONLY when processing pos headers
a2e5b4b
@lazynina
updates for locating headers and faster cache usage
ff6ee24
@lazynina
more fixes
71b1eaa
@lazynina
more fixes
17a56bd
@lazynina
more fix, specifically around .Ancestor
d153927
@lazynina
bump test timeout
47e1958
@lazynina
load all blocks after committed tip in block index
5e5c0c1
@lazynina
fix GetBlockNodeByHashAndHeight - no need for checking height > maxHe…
9a32c26 
…ight
@lazynina
bump timeout to 60m
5359be8
@lazynina
try hashicorp lru cache
89a845e
@lazynina
Fix up some more tests and remove logging
4f7ff12
@lazynina
fix string slice workaround, fix calc next difficulty
722dcc0
@lazynina
use best header chain instead of best chain for calc next difficulty …
1941487 
…target
@lazynina
fix tests
627e882
@lazynina
refactor to just use a block hash for best block chain and best heade…
a469c3a 
…r chain representation
@lazynina
move block index migration to NewBlockchain instead of NewServer, add…
c6ad24a 
… setting committed status for PoW nodes, remove timeout adjustment in test.Dockerfile
@lazynina
merge main into branch
216fd31
@lazynina
Merge branch 'ln/read-through-cache-for-block-index-and-best-chain' i…
1f80f19 
…nto ln/replace-best-chain-with-block-index
@lazynina
handle case where we haven't hit PoS cutover yet in block index migra…
8b73b1c 
…tion
@lazynina
cleanup
7ac67bd
@lazynina
do some sampling for is fully stored and check archival mode
7675765
@lazynina
fix get blocks to store
148bad3
@lazynina
cleanup
8aa6a5b
@lazynina
make a bunch of methods on block index private and expose block index…
53b2d29 
… to backend
@lazynina
don't bother healing orphan pointers if we're syncing
91e22ff
@lazynina
refactor to reduce number of times we have to call hash and don't bot…
9abe238 
…her healing orphan pointers during syncing
@lazynina
upgrade cloudflare/circl
23991c4
@lazynina
Ln/replace best chain plus replace old lru cache (#1414)
8adc624 
* replace decred lru with hashicorp lru

* fix usage of delete -> remove

* fix calc next difficulty target issue

* upgrade cloudflare/circl
lazynina and others added 21 commits October 7, 2024 23:06
@lazynina
Merge branch 'ln/replace-best-chain-with-block-index' of github.com:d…
2608ec5 
…eso-protocol/core into ln/replace-best-chain-with-block-index
@lazynina
go mod tidy
dbad649
@lazynina
expose SetBestChain
ebb1175
@lazynina
fix exit condition for GetBlocksToStore
ae34cff
@lazynina
use bytes.Equal instead of reflect.DeepEqual for comparing block hashes
b48c52d
@lazynina
use get block from best chain by height to speed up syncing
925e44e
@lazynina
fix condition for returning from get block by hash
dddd64c
@lazynina
revert using get block from best chain by height
62503c1
@lazynina
add height check to reduce calls to compare hashes
131f389
@lazynina
fix exit condition for height in get block from best chain by hash
f57a699
@lazynina
allow early return from GetBlockFromBestChainByHash if we're using he…
2bae1a2 
…ader chain and the header is validated if we're syncing
@lazynina
minor enhancement to isTipCurrent
ea243f3
@lazynina
fix txindex issue w/ genesis block
3f73bb6
@lazynina
fix process block pow logic for genesis block
e200407
@lazynina
fix - get next block after tip hash in update process of txindex
3ad68be
@lazynina
reduce direct access to BlockNode.Parent and get rid of heal orphan p…
78034d4 
…ointers since we don't need it
@lazynina
fix postgres get best chain logic
02e9e15
@lazynina
merge in main
dec4675
@lazynina @diamondhands0
Ln/downgrade badger (#1430)
2a82dea 
* downgrade badger, ristretto, and klauspost/compress dependencies

* go mod tidy
@diamondhands0
FIXME to make the mempool thing work
a1a09b8
@diamondhands0
wip
bba95f6
@diamondhands0 diamondhands0 requested a review from a team as a code owner December 4, 2024 06:35
Base automatically changed from ln/replace-best-chain-with-block-index to main May 1, 2025 16:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@diamondhands0 @lazynina