Skip to content

feat(cli): pair mainnet replay bench with revm - #384

Open
RealiCZ wants to merge 1 commit into
alloy-rs:mainfrom
RealiCZ:cz/feat/mainnet-replay-revm-bench
Open

feat(cli): pair mainnet replay bench with revm#384
RealiCZ wants to merge 1 commit into
alloy-rs:mainfrom
RealiCZ:cz/feat/mainnet-replay-revm-bench

Conversation

@RealiCZ

@RealiCZ RealiCZ commented Sep 1, 2026

Copy link
Copy Markdown

Summary

  • Pair the mainnet_25347446_25347455/replay benchmark with revm. It was the only workload without a revm comparison: BenchCaseKind::BlockchainReplay never entered the EVM2_BENCH_REVM=1 path, so the one bench that resembles real-world load had no paired number.
  • New evm2_cli::replay_bench module drives the same corpus through both engines:
    • an evm2 mirror of the EEST blockchain executor (the EEST hook does not expose per-transaction results, and the timed bench path still uses the unmodified EEST executor);
    • a revm driver aligned step-by-step with the evm2 path: same per-block spec, block env fields, blob base fee via blob_params_for_timestamp, pre/post-block system calls (EIP-2935/4788/7002/7251), withdrawals, and commit cadence (journal accumulation per tx, one commit per block onto CacheDB<EmptyDB>).
  • Parity gate: tests/replay_parity.rs plus a bench-time sanity check compare every transaction's gas used, success flag and log count, and independently anchor both engines to each header's gasUsed. The paired numbers only exist while this passes. Current corpus (10 blocks / 2279 txs / 276,625,341 gas): zero mismatches on both dispatch backends.
  • Add …/replay/setup and …/replay/revm/setup benches measuring the engine-neutral harness work (pre-state + tx decoding, ~31 ms per iteration on the corpus) so the EVM-only ratio can be read separately.
  • Give the replay group its own criterion budget (3 s warm-up / 15 s measurement / 20 samples, overridable via EVM2_BENCH_REPLAY_MEASUREMENT_SECS / EVM2_BENCH_REPLAY_SAMPLES). All other benches keep the existing budget and code paths.

Sample numbers on an Apple M5 Pro (nightly): evm2 109.18 ms vs revm 124.71 ms — 1.14x raw, 1.19x after subtracting the shared setup; with EVM2_DISPATCH_BACKEND=packed: 115.02 ms vs 125.96 ms (1.10x / 1.13x). All CI widths < 2.5%.

Test plan

  • cargo test -p evm2-cli --test replay_parity — zero mismatches across 2279 transactions; also run with EVM2_DISPATCH_BACKEND=packed
  • EVM2_BENCH_REVM=1 cargo bench -p evm2-cli --bench evm -- mainnet
  • cargo fmt --all --check; cargo clippy -p evm2-cli -p evm2 --all-targets and cargo doc -p evm2-cli --no-deps --document-private-items with zero warnings; cargo test -p evm2-cli

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant