-
Notifications
You must be signed in to change notification settings - Fork 995
Description
Summary
Implement an optimization for bonsai-archive mode that syncs the database in regular bonsai (FULL flat db) mode first, then migrates to archive format after sync is complete. This significantly improves initial sync performance for archive nodes.
Background
Currently, syncing a new node with --data-storage-format=BONSAI and --bonsai-historical-block-limit=0 (archive mode) is slow because archive mode uses seekForPrev for all state reads.
In archive mode, keys are suffixed with block numbers to maintain historical versions. To read state at a given block, the database must use RocksDB's seekForPrev operation to find the most recent version of a key at or before the requested block number. This seek operation is significantly more expensive than direct key lookups, and during sync there are millions of state reads that compound this overhead.
By contrast, regular bonsai FULL mode uses simple direct key lookups without version suffixes, making reads much faster.
Proposed Solution
- Sync in FULL flat db mode: When starting in bonsai-archive mode, use FULL flat db strategy during the initial sync for fast direct key lookups
- Disable archiver during sync: The BonsaiArchiver should not run while the flat db is not yet in archive mode
- Migrate after sync completion: Once the initial sync completes (snap sync heal or full sync), trigger the archive migration
- Process trie logs: The migrator processes all trie logs from genesis to head, adding block number suffixes to create the versioned archive
Metadata
Metadata
Assignees
Labels
Type
Projects
Status