Skip to content

Bonsai archive sync as Bonsai the migrate to archive #9608

@jframe

Description

@jframe

Summary

Implement an optimization for bonsai-archive mode that syncs the database in regular bonsai (FULL flat db) mode first, then migrates to archive format after sync is complete. This significantly improves initial sync performance for archive nodes.

Background

Currently, syncing a new node with --data-storage-format=BONSAI and --bonsai-historical-block-limit=0 (archive mode) is slow because archive mode uses seekForPrev for all state reads.

In archive mode, keys are suffixed with block numbers to maintain historical versions. To read state at a given block, the database must use RocksDB's seekForPrev operation to find the most recent version of a key at or before the requested block number. This seek operation is significantly more expensive than direct key lookups, and during sync there are millions of state reads that compound this overhead.

By contrast, regular bonsai FULL mode uses simple direct key lookups without version suffixes, making reads much faster.

Proposed Solution

  1. Sync in FULL flat db mode: When starting in bonsai-archive mode, use FULL flat db strategy during the initial sync for fast direct key lookups
  2. Disable archiver during sync: The BonsaiArchiver should not run while the flat db is not yet in archive mode
  3. Migrate after sync completion: Once the initial sync completes (snap sync heal or full sync), trigger the archive migration
  4. Process trie logs: The migrator processes all trie logs from genesis to head, adding block number suffixes to create the versioned archive

Metadata

Metadata

Assignees

Type

No type

Projects

Status

In Progress

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions