Skip to content

feat(electrum): optimize merkle proof validation with batching #1957

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 3 commits into
base: master
Choose a base branch
from

Conversation

LagginTimes
Copy link
Contributor

Replaces #1908, originally authored by @Keerthi421.
Fixes #1891.

Description

This PR optimizes the Electrum client's performance by improving Merkle proof validation, addressing the significant performance regression in BDK 1.1.0 where full sync time increased from 4s to 26s.

Key improvements:

  • Implemented batch processing for Merkle proof validations.
  • Added Merkle proof caching to prevent redundant network calls.
  • Optimized header handling with pre-fetching and reuse.
  • Modified core functions to use batch operations instead of individual calls.

Also adds reorg-safe eviction of stale proofs: before each Merkle batch, we verify the highest cached block’s hash against the live tip and, on mismatch, evict proofs until the fork point is reached.

Notes to the reviewers

The optimization approach focuses on three main areas:

  1. Reducing network round trips through batched Merkle proof requests.
  2. Minimizing redundant operations with a new Merkle proof cache.
  3. Improving header handling efficiency with pre-fetching.

The batch size is set to 100 as a balance between performance and memory usage. This value can be adjusted based on testing results.

Changelog notice

  • New Merkle proof cache to prevent redundant network calls.
  • Batch processing for Merkle proof validations.
  • Performance tests to verify sync time improvements.

Checklists

All Submissions:

  • I've signed all my commits
  • I followed the contribution guidelines
  • I ran cargo fmt and cargo clippy before committing

New Features:

  • I've added tests for the new feature
  • I've added docs for the new feature

Bugfixes:

  • This pull request breaks the existing API
  • I've added tests to reproduce the issue which are now passing
  • I'm linking the issue being fixed by this PR

@LagginTimes LagginTimes requested a review from evanlinjin May 15, 2025 19:06
@LagginTimes LagginTimes self-assigned this May 15, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Electrum client Performance issues
2 participants