Skip to content

fix(trie): start partial proofs below known parent#26471

Merged
mediocregopher merged 12 commits into
mainfrom
mediocregopher/proof-v2-child-subtrie
Jul 23, 2026
Merged

fix(trie): start partial proofs below known parent#26471
mediocregopher merged 12 commits into
mainfrom
mediocregopher/proof-v2-child-subtrie

Conversation

@mediocregopher

@mediocregopher mediocregopher commented Jul 22, 2026

Copy link
Copy Markdown
Member

Start known-parent proofs at direct children and batch targets sharing a parent into one bounded traversal. This prevents a stale masked parent node (due to partial persistence) from suppressing live child ranges without splitting sibling targets into independent cursor traversals (which is bad for perf).

For targets aed2 and aed4 under known parent aed:

Before (origin/main)

  • Seek aed and traverse [aed, aee), putting the parent and all 16 child ranges in scope.
  • The database parent can be consulted even though the sparse trie already supplied the current parent.

After

  • Seek the first targeted child, aed2, and traverse the bounded sibling span [aed2, aed5) once. The parent is never read by the trie traversal, while outer siblings aed0aed1 and aed5aedf are excluded from calculation.
  • Targeted children aed2 and aed4 are calculated and retained independently.
  • Untargeted aed3 stays inside the traversal span but is not retained. Usable cached trie data skips its leaves; if it is uncached, the hashed cursor advances through it sequentially instead of performing another seek at aed4.

This narrows the work from the whole parent range while preserving efficient sequential traversal between targeted siblings.

Testing

Ran 1k big blocks, 20k mainnet blocks, and 10k mainnet blocks with reorgs with no issues.

@github-project-automation github-project-automation Bot moved this to Backlog in Reth Tracker Jul 22, 2026
@mediocregopher mediocregopher added the A-trie Related to Merkle Patricia Trie implementation label Jul 22, 2026
mediocregopher added a commit that referenced this pull request Jul 22, 2026
# Conflicts:
#	crates/e2e-test-utils/tests/rocksdb/main.rs
#	crates/engine/primitives/src/config.rs
#	crates/engine/tree/src/tree/state_root_strategy/mod.rs
#	crates/node/core/src/args/engine.rs
@mediocregopher

Copy link
Copy Markdown
Member Author

derek bench

@mediocregopher

Copy link
Copy Markdown
Member Author

derek bench big-blocks bal=true

@decofe

decofe commented Jul 22, 2026

Copy link
Copy Markdown
Member

cc @mediocregopher

✅ Benchmark complete! View run

Benchmark Results

Configuration

  • Derek command: derek bench blocks=500 warmup=125 baseline=main feature=mediocregopher/proof-v2-child-subtrie bal=false cores=0 run-pairs=6 otlp=true metrics=false slack=always

⚠️ Feature is 1 commit behind main. Consider rebasing for accurate results.

Metric main mediocregopher/proof-v2-child-subtrie Change
Mean 34.68ms 34.37ms -0.89% ⚪ (±0.52%, floor 1.20%)
P50 24.09ms 23.74ms -1.46% ⚪ (±0.51%, floor 1.20%)
P90 73.62ms 72.97ms -0.88% ⚪ (±2.32%, floor 1.35%)
P99 188.49ms 193.68ms +2.75% ⚪ (±5.02%, floor 5.00%)
Mgas/s 1146.22 1160.13 +1.21% ⚪ (±0.52%, floor 1.20%)
Wall Clock 105.00s 104.01s -0.94% ⚪ (±0.57%, floor 0.70%)
Persist Wait 22.26ms 22.51ms +1.12% ⚪ (±1.16%, floor 5.00%)

500 blocks, mode: engine, 125 warmup, 6 run pairs

Wait Time Breakdown

Persistence Wait

Metric main mediocregopher/proof-v2-child-subtrie
Mean 22.26ms 22.51ms
P50 0.02ms 0.02ms
P95 227.00ms 224.74ms

Trie Cache Update Wait

Metric main mediocregopher/proof-v2-child-subtrie
Mean 0.00ms 0.00ms
P50 0.00ms 0.00ms
P95 0.00ms 0.00ms

Execution Cache Update Wait

Metric main mediocregopher/proof-v2-child-subtrie
Mean 0.00ms 0.00ms
P50 0.00ms 0.00ms
P95 0.00ms 0.00ms

Observability

Charts

Latency, Throughput & Diff

Latency, Throughput & Diff

Wait Time Breakdown

Wait Time Breakdown

Gas vs Latency

Gas vs Latency

@decofe

decofe commented Jul 22, 2026

Copy link
Copy Markdown
Member

cc @mediocregopher

✅ Benchmark complete! View run

Benchmark Results

Configuration

  • Derek command: derek bench big-blocks=true blocks=30 warmup=7 baseline=main feature=mediocregopher/proof-v2-child-subtrie bal=true cores=0 run-pairs=10 otlp=true metrics=false slack=always

⚠️ Feature is 1 commit behind main. Consider rebasing for accurate results.

Metric main mediocregopher/proof-v2-child-subtrie Change
Mean 268.23ms 279.18ms +4.08% ❌ (±1.10%, floor 1.20%)
P50 256.07ms 264.73ms +3.38% ❌ (±1.60%, floor 1.20%)
P90 331.86ms 343.92ms +3.63% ⚪ (±3.15%, floor 1.35%)
P99 484.96ms 570.37ms +17.61% ⚪ (±6.75%, floor 5.00%, informational)
Mgas/s 3911.68 3798.64 -2.89% ❌ (±1.00%, floor 1.20%)
Wall Clock 82.82s 86.13s +3.99% ❌ (±1.05%, floor 0.70%)
Persist Wait 185.58ms 179.36ms -3.35% ⚪ (±4.04%, floor 5.00%)

30 big blocks, mode: engine, 7 warmup, 10 run pairs, BAL: true

Wait Time Breakdown

Persistence Wait

Metric main mediocregopher/proof-v2-child-subtrie
Mean 185.58ms 179.36ms
P50 0.07ms 0.06ms
P95 1619.51ms 1687.02ms

Trie Cache Update Wait

Metric main mediocregopher/proof-v2-child-subtrie
Mean 0.00ms 0.00ms
P50 0.00ms 0.00ms
P95 0.00ms 0.00ms

Execution Cache Update Wait

Metric main mediocregopher/proof-v2-child-subtrie
Mean 0.00ms 0.00ms
P50 0.00ms 0.00ms
P95 0.00ms 0.00ms

Target Metrics

Metric Baseline Feature Change
reth_sync_block_validation_state_root_histogram mean 0.0479 0.0644 +34.31% ❌ (±9.40%, floor 5.00%, materiality 14.35ms)

Observability

Charts

Latency, Throughput & Diff

Latency, Throughput & Diff

Wait Time Breakdown

Wait Time Breakdown

Gas vs Latency

Gas vs Latency

@mediocregopher

Copy link
Copy Markdown
Member Author

derek bench big-blocks bal=true

@decofe

decofe commented Jul 22, 2026

Copy link
Copy Markdown
Member

cc @mediocregopher

✅ Benchmark complete! View run

Benchmark Results

Configuration

  • Derek command: derek bench big-blocks=true blocks=30 warmup=7 baseline=main feature=mediocregopher/proof-v2-child-subtrie bal=true cores=0 run-pairs=10 otlp=true metrics=false slack=always

⚠️ Feature is 2 commits behind main. Consider rebasing for accurate results.

Metric main mediocregopher/proof-v2-child-subtrie Change
Mean 283.50ms 290.78ms +2.57% ❌ (±1.15%, floor 1.20%)
P50 269.87ms 275.54ms +2.10% ⚪ (±2.28%, floor 1.20%)
P90 368.81ms 369.54ms +0.20% ⚪ (±3.58%, floor 1.35%)
P99 505.93ms 595.29ms +17.66% ⚪ (±6.92%, floor 5.00%, informational)
Mgas/s 3717.22 3655.54 -1.66% ⚪ (±1.22%, floor 1.20%)
Wall Clock 87.45s 89.62s +2.48% ❌ (±1.09%, floor 0.70%)
Persist Wait 185.17ms 188.71ms +1.92% ⚪ (±3.61%, floor 5.00%)

30 big blocks, mode: engine, 7 warmup, 10 run pairs, BAL: true

Wait Time Breakdown

Persistence Wait

Metric main mediocregopher/proof-v2-child-subtrie
Mean 185.17ms 188.71ms
P50 0.07ms 0.07ms
P95 1143.29ms 1157.88ms

Trie Cache Update Wait

Metric main mediocregopher/proof-v2-child-subtrie
Mean 0.00ms 0.00ms
P50 0.00ms 0.00ms
P95 0.00ms 0.00ms

Execution Cache Update Wait

Metric main mediocregopher/proof-v2-child-subtrie
Mean 0.00ms 0.00ms
P50 0.00ms 0.00ms
P95 0.00ms 0.00ms

Target Metrics

Metric Baseline Feature Change
reth_sync_block_validation_state_root_histogram mean 0.0624 0.0701 +12.44% ❌ (±7.03%, floor 5.00%, materiality 14.94ms)

Observability

Charts

Latency, Throughput & Diff

Latency, Throughput & Diff

Wait Time Breakdown

Wait Time Breakdown

Gas vs Latency

Gas vs Latency

@mediocregopher

Copy link
Copy Markdown
Member Author

derek bench big-blocks bal=true run-pairs=1 blocks=100 samply

@decofe

decofe commented Jul 22, 2026

Copy link
Copy Markdown
Member

cc @mediocregopher

✅ Benchmark complete! View run

Benchmark Results

Configuration

  • Derek command: derek bench big-blocks=true blocks=100 warmup=25 baseline=main feature=mediocregopher/proof-v2-child-subtrie bal=true cores=0 run-pairs=1 otlp=true metrics=false slack=always samply

⚠️ Feature is 2 commits behind main. Consider rebasing for accurate results.

Metric main mediocregopher/proof-v2-child-subtrie Change
Mean 285.69ms 293.95ms +2.89% ⚪ (±2.57%, floor 1.20%)
P50 281.63ms 281.87ms +0.09% ⚪ (±4.25%, floor 1.20%)
P90 339.55ms 363.72ms +7.12% ⚪ (±7.39%, floor 1.35%)
P99 445.68ms 555.31ms +24.60% ⚪ (±20.60%, floor 5.00%, informational)
Mgas/s 3640.17 3587.35 -1.45% ⚪ (±1.99%, floor 1.20%)
Wall Clock 29.37s 30.24s +2.94% ⚪ (±2.51%, floor 0.70%)
Persist Wait 239.72ms 236.83ms -1.21% ⚪ (±1.87%, floor 5.00%)

100 big blocks, mode: engine, 25 warmup, 1 run pairs, BAL: true

Wait Time Breakdown

Persistence Wait

Metric main mediocregopher/proof-v2-child-subtrie
Mean 239.72ms 236.83ms
P50 0.09ms 0.09ms
P95 3059.85ms 3019.25ms

Trie Cache Update Wait

Metric main mediocregopher/proof-v2-child-subtrie
Mean 0.00ms 0.00ms
P50 0.00ms 0.00ms
P95 0.00ms 0.00ms

Execution Cache Update Wait

Metric main mediocregopher/proof-v2-child-subtrie
Mean 0.00ms 0.00ms
P50 0.00ms 0.00ms
P95 0.00ms 0.00ms

Observability

Charts

Latency, Throughput & Diff

Latency, Throughput & Diff

Wait Time Breakdown

Wait Time Breakdown

Gas vs Latency

Gas vs Latency

Samply Profiles

@mediocregopher

Copy link
Copy Markdown
Member Author

derek bench big-blocks bal=true

@decofe

decofe commented Jul 22, 2026

Copy link
Copy Markdown
Member

cc @mediocregopher

✅ Benchmark complete! View run

Benchmark Results

Configuration

  • Derek command: derek bench big-blocks=true blocks=30 warmup=7 baseline=main feature=mediocregopher/proof-v2-child-subtrie bal=true cores=0 run-pairs=10 otlp=true metrics=false slack=always

⚠️ Feature is 3 commits behind main. Consider rebasing for accurate results.

Metric main mediocregopher/proof-v2-child-subtrie Change
Mean 282.32ms 272.18ms -3.59% ✅ (±1.04%, floor 1.20%)
P50 264.04ms 254.69ms -3.54% ✅ (±1.70%, floor 1.20%)
P90 358.98ms 345.91ms -3.64% ⚪ (±4.66%, floor 1.35%)
P99 529.46ms 472.10ms -10.83% ⚪ (±5.51%, floor 5.00%, informational)
Mgas/s 3736.26 3865.39 +3.46% ✅ (±1.02%, floor 1.20%)
Wall Clock 87.15s 84.10s -3.51% ✅ (±1.05%, floor 0.70%)
Persist Wait 189.24ms 190.96ms +0.91% ⚪ (±2.73%, floor 5.00%)

30 big blocks, mode: engine, 7 warmup, 10 run pairs, BAL: true

Wait Time Breakdown

Persistence Wait

Metric main mediocregopher/proof-v2-child-subtrie
Mean 189.24ms 190.96ms
P50 0.07ms 0.07ms
P95 1123.38ms 699.64ms

Trie Cache Update Wait

Metric main mediocregopher/proof-v2-child-subtrie
Mean 0.00ms 0.00ms
P50 0.00ms 0.00ms
P95 0.00ms 0.00ms

Execution Cache Update Wait

Metric main mediocregopher/proof-v2-child-subtrie
Mean 0.00ms 0.00ms
P50 0.00ms 0.00ms
P95 0.00ms 0.00ms

Target Metrics

Metric Baseline Feature Change
reth_sync_block_validation_state_root_histogram mean 0.0606 0.0507 -16.37% ✅ (±6.51%, floor 5.00%, materiality 14.53ms)

Observability

Charts

Latency, Throughput & Diff

Latency, Throughput & Diff

Wait Time Breakdown

Wait Time Breakdown

Gas vs Latency

Gas vs Latency

@mediocregopher

Copy link
Copy Markdown
Member Author

derek bench

@decofe

decofe commented Jul 22, 2026

Copy link
Copy Markdown
Member

cc @mediocregopher

✅ Benchmark complete! View run

Benchmark Results

Configuration

  • Derek command: derek bench blocks=500 warmup=125 baseline=main feature=mediocregopher/proof-v2-child-subtrie bal=false cores=0 run-pairs=6 otlp=true metrics=false slack=always

⚠️ Feature is 4 commits behind main. Consider rebasing for accurate results.

Metric main mediocregopher/proof-v2-child-subtrie Change
Mean 35.09ms 34.96ms -0.38% ⚪ (±0.51%, floor 1.20%)
P50 24.01ms 23.70ms -1.30% ⚪ (±0.99%, floor 1.20%)
P90 74.14ms 78.29ms +5.60% ⚪ (±5.06%, floor 1.35%)
P99 184.78ms 184.73ms -0.03% ⚪ (±3.66%, floor 5.00%)
Mgas/s 1141.84 1153.18 +0.99% ⚪ (±0.58%, floor 1.20%)
Wall Clock 106.18s 105.73s -0.43% ⚪ (±0.52%, floor 0.70%)
Persist Wait 22.32ms 22.79ms +2.10% ⚪ (±1.68%, floor 5.00%)

500 blocks, mode: engine, 125 warmup, 6 run pairs

Wait Time Breakdown

Persistence Wait

Metric main mediocregopher/proof-v2-child-subtrie
Mean 22.32ms 22.79ms
P50 0.02ms 0.02ms
P95 225.03ms 227.72ms

Trie Cache Update Wait

Metric main mediocregopher/proof-v2-child-subtrie
Mean 0.00ms 0.00ms
P50 0.00ms 0.00ms
P95 0.00ms 0.00ms

Execution Cache Update Wait

Metric main mediocregopher/proof-v2-child-subtrie
Mean 0.00ms 0.00ms
P50 0.00ms 0.00ms
P95 0.00ms 0.00ms

Observability

Charts

Latency, Throughput & Diff

Latency, Throughput & Diff

Wait Time Breakdown

Wait Time Breakdown

Gas vs Latency

Gas vs Latency

@mediocregopher

Copy link
Copy Markdown
Member Author

derek bench big-blocks bal=true

@mediocregopher

Copy link
Copy Markdown
Member Author

derek bench

@decofe

decofe commented Jul 23, 2026

Copy link
Copy Markdown
Member

cc @mediocregopher

✅ Benchmark complete! View run

Benchmark Results

Configuration

  • Derek command: derek bench big-blocks=true blocks=30 warmup=7 baseline=main feature=mediocregopher/proof-v2-child-subtrie bal=true cores=0 run-pairs=10 otlp=true metrics=false slack=always

⚠️ Feature is 7 commits behind main. Consider rebasing for accurate results.

Metric main mediocregopher/proof-v2-child-subtrie Change
Mean 269.30ms 265.17ms -1.53% ⚪ (±1.17%, floor 1.20%)
P50 251.68ms 248.50ms -1.26% ⚪ (±1.55%, floor 1.20%)
P90 350.55ms 335.14ms -4.40% ⚪ (±3.53%, floor 1.35%)
P99 467.17ms 484.50ms +3.71% ⚪ (±4.90%, floor 5.00%, informational)
Mgas/s 3900.79 3960.51 +1.53% ⚪ (±1.24%, floor 1.20%)
Wall Clock 83.05s 81.87s -1.43% ⚪ (±1.18%, floor 0.70%)
Persist Wait 181.43ms 184.95ms +1.94% ⚪ (±3.03%, floor 5.00%)

30 big blocks, mode: engine, 7 warmup, 10 run pairs, BAL: true

Wait Time Breakdown

Persistence Wait

Metric main mediocregopher/proof-v2-child-subtrie
Mean 181.43ms 184.95ms
P50 0.06ms 0.06ms
P95 1262.95ms 1274.02ms

Trie Cache Update Wait

Metric main mediocregopher/proof-v2-child-subtrie
Mean 0.00ms 0.00ms
P50 0.00ms 0.00ms
P95 0.00ms 0.00ms

Execution Cache Update Wait

Metric main mediocregopher/proof-v2-child-subtrie
Mean 0.00ms 0.00ms
P50 0.00ms 0.00ms
P95 0.00ms 0.00ms

Target Metrics

Metric Baseline Feature Change
reth_sync_block_validation_state_root_histogram mean 0.0496 0.0431 -13.16% ✅ (±7.87%, floor 5.00%, materiality 13.84ms)
reth_storage_providers_database_save_blocks_commit_rocksdb mean 0.0064 0.0059 -8.22% ✅ (±4.66%, floor 2.50%, materiality 2.21ms)

Observability

Charts

Latency, Throughput & Diff

Latency, Throughput & Diff

Wait Time Breakdown

Wait Time Breakdown

Gas vs Latency

Gas vs Latency

@decofe

decofe commented Jul 23, 2026

Copy link
Copy Markdown
Member

cc @mediocregopher

✅ Benchmark complete! View run

Benchmark Results

Configuration

  • Derek command: derek bench blocks=500 warmup=125 baseline=main feature=mediocregopher/proof-v2-child-subtrie bal=false cores=0 run-pairs=6 otlp=true metrics=false slack=always

⚠️ Feature is 7 commits behind main. Consider rebasing for accurate results.

Metric main mediocregopher/proof-v2-child-subtrie Change
Mean 27.02ms 27.09ms +0.26% ⚪ (±0.77%, floor 1.20%)
P50 23.22ms 22.98ms -1.05% ⚪ (±1.14%, floor 1.20%)
P90 44.14ms 44.21ms +0.17% ⚪ (±1.14%, floor 1.35%)
P99 90.63ms 92.05ms +1.57% ⚪ (±4.83%, floor 5.00%)
Mgas/s 1223.93 1228.25 +0.35% ⚪ (±0.66%, floor 1.20%)
Wall Clock 81.61s 81.86s +0.31% ⚪ (±0.76%, floor 0.70%)
Persist Wait 29.33ms 29.42ms +0.31% ⚪ (±1.20%, floor 5.00%)

500 blocks, mode: engine, 125 warmup, 6 run pairs

Wait Time Breakdown

Persistence Wait

Metric main mediocregopher/proof-v2-child-subtrie
Mean 29.33ms 29.42ms
P50 0.02ms 0.02ms
P95 260.67ms 272.00ms

Trie Cache Update Wait

Metric main mediocregopher/proof-v2-child-subtrie
Mean 0.00ms 0.00ms
P50 0.00ms 0.00ms
P95 0.00ms 0.00ms

Execution Cache Update Wait

Metric main mediocregopher/proof-v2-child-subtrie
Mean 0.02ms 0.00ms
P50 0.00ms 0.00ms
P95 0.00ms 0.00ms

Observability

Charts

Latency, Throughput & Diff

Latency, Throughput & Diff

Wait Time Breakdown

Wait Time Breakdown

Gas vs Latency

Gas vs Latency

@mediocregopher

Copy link
Copy Markdown
Member Author

derek bench big-blocks bal=true

@decofe

decofe commented Jul 23, 2026

Copy link
Copy Markdown
Member

cc @mediocregopher

✅ Benchmark complete! View run

Benchmark Results

Configuration

  • Derek command: derek bench big-blocks=true blocks=30 warmup=7 baseline=main feature=mediocregopher/proof-v2-child-subtrie bal=true cores=0 run-pairs=10 otlp=true metrics=false slack=always

⚠️ Feature is 7 commits behind main. Consider rebasing for accurate results.

Metric main mediocregopher/proof-v2-child-subtrie Change
Mean 282.08ms 274.21ms -2.79% ✅ (±1.04%, floor 1.20%)
P50 265.17ms 258.65ms -2.46% ⚪ (±2.19%, floor 1.20%)
P90 368.57ms 355.00ms -3.68% ⚪ (±5.29%, floor 1.35%)
P99 519.39ms 464.03ms -10.66% ⚪ (±6.38%, floor 5.00%, informational)
Mgas/s 3743.92 3834.66 +2.42% ✅ (±1.09%, floor 1.20%)
Wall Clock 86.99s 84.66s -2.67% ✅ (±1.04%, floor 0.70%)
Persist Wait 189.66ms 190.15ms +0.26% ⚪ (±2.51%, floor 5.00%)

30 big blocks, mode: engine, 7 warmup, 10 run pairs, BAL: true

Wait Time Breakdown

Persistence Wait

Metric main mediocregopher/proof-v2-child-subtrie
Mean 189.66ms 190.15ms
P50 0.07ms 0.07ms
P95 1144.34ms 737.13ms

Trie Cache Update Wait

Metric main mediocregopher/proof-v2-child-subtrie
Mean 0.00ms 0.00ms
P50 0.00ms 0.00ms
P95 0.00ms 0.00ms

Execution Cache Update Wait

Metric main mediocregopher/proof-v2-child-subtrie
Mean 0.00ms 0.00ms
P50 0.00ms 0.00ms
P95 0.00ms 0.00ms

Target Metrics

Metric Baseline Feature Change
reth_sync_block_validation_state_root_histogram mean 0.06 0.0527 -12.23% ✅ (±6.19%, floor 5.00%, materiality 14.50ms)

Observability

Charts

Latency, Throughput & Diff

Latency, Throughput & Diff

Wait Time Breakdown

Wait Time Breakdown

Gas vs Latency

Gas vs Latency

@mediocregopher mediocregopher self-assigned this Jul 23, 2026
@mediocregopher mediocregopher added the C-perf A change motivated by improving speed, memory usage or disk footprint label Jul 23, 2026
@mediocregopher
mediocregopher marked this pull request as ready for review July 23, 2026 11:27
@github-project-automation github-project-automation Bot moved this from Backlog to In Progress in Reth Tracker Jul 23, 2026
@mediocregopher
mediocregopher added this pull request to the merge queue Jul 23, 2026
Merged via the queue into main with commit 32de8f9 Jul 23, 2026
37 checks passed
@mediocregopher
mediocregopher deleted the mediocregopher/proof-v2-child-subtrie branch July 23, 2026 14:55
@github-project-automation github-project-automation Bot moved this from In Progress to Done in Reth Tracker Jul 23, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

A-trie Related to Merkle Patricia Trie implementation C-perf A change motivated by improving speed, memory usage or disk footprint

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

3 participants