chore: merge develop-v2.2-new into develop#192
Open
chee-chyuan wants to merge 1417 commits into
Open
Conversation
Co-authored-by: Soubhik Singha Mahapatra <soubhiksmp2004@gmail.com>
…(#23485) Co-authored-by: Matthias Seitz <matthias.seitz@outlook.de>
Co-authored-by: Ishika Choudhury <117741714+Rimeeeeee@users.noreply.github.com>
Co-authored-by: Alexey Shekhirin <5773434+shekhirin@users.noreply.github.com> Co-authored-by: Alexey Shekhirin <github@shekhirin.com>
Co-authored-by: Soubhik Singha Mahapatra <soubhiksmp2004@gmail.com> Co-authored-by: Soubhik Singha Mahapatra <160333583+Soubhik-10@users.noreply.github.com> Co-authored-by: Brian Picciano <933154+mediocregopher@users.noreply.github.com> Co-authored-by: Amp <amp@ampcode.com> Co-authored-by: steven <corderosteven6@gmail.com> Co-authored-by: Derek Cofausper <256792747+decofe@users.noreply.github.com>
Co-authored-by: Brian Picciano <933154+mediocregopher@users.noreply.github.com>
…ks (#23494) Co-authored-by: mediocregopher <mediocregopher@users.noreply.github.com> Co-authored-by: Amp <amp@ampcode.com>
… (#23356) Co-authored-by: Derek Cofausper <256792747+decofe@users.noreply.github.com> Co-authored-by: Emma Jamieson-Hoare <21029500+emmajam@users.noreply.github.com>
…ror/never) (#23501) Co-authored-by: Amp <amp@ampcode.com>
Co-authored-by: Soubhik Singha Mahapatra <soubhiksmp2004@gmail.com> Co-authored-by: Soubhik Singha Mahapatra <160333583+Soubhik-10@users.noreply.github.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: Alexey Shekhirin <5773434+shekhirin@users.noreply.github.com>
Co-authored-by: Amp <amp@ampcode.com>
Co-authored-by: Federico Gimenez <federico.gimenez@gmail.com> Co-authored-by: Federico Gimenez <fgimenez@users.noreply.github.com> Co-authored-by: klkvr <klkvrr@gmail.com> Co-authored-by: Matthias Seitz <matthias.seitz@outlook.de>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…launch Add logs to aid diagnosing block import issues: - engine::api: trace when NewPayload is sent via ConsensusEngineHandle - engine::tree: trace when engine tree receives NewPayload - engine: debug when engine orchestrator is being built - reth::node: info when engine node launch begins Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Log persistence_gap, threshold, and stall duration at warn level so backpressure is visible without enabling debug/trace on the engine target. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…gh engine Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Adds warn-level logs at every stage after execute_block completes to identify which step is blocking on_new_payload: - After receipt_root wait - Inside validate_post_execution (start, post-receipt-check, pre/post hashed_state.get) - Before state root computation (prints strategy) - Inside await_state_root_with_timeout (pre-wait, no-timeout path) - Inside serial-root task (start, pre/post state_root_with_updates) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
`receipt_tx` was dropped immediately after `execute_transactions` returned, before `executor.finish()` ran. Any receipts appended during finalization (e.g. system transaction receipts on BSC) were never sent to the background receipt-root task, causing it to see fewer receipts than expected and silently fail to compute the root. Fix by carrying `last_sent_len` out of `execute_transactions`, running `finish()` first, then flushing any remaining receipts before dropping the sender — matching the pattern in the upstream develop branch. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…root hang Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…triedb state root Pinpoint whether hang is at terminate_caching, triedb_preftch_result, or to_triedb_hashed_post_state before the blocking state root call. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The prefetch task can take minutes for blocks with large state deltas (e.g. BSC block 1 genesis deployment). Previously this blocked the engine tree thread indefinitely via recv(). Now uses recv_timeout(5s) so the state root computation proceeds without prefetch state on slow prefetchers, allowing the chain to make progress. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…task The changeset background task previously held an MDBX read-only transaction open for its entire duration, including the slow wait_cloned() trie computation phase. This blocked MDBX GC during write commits, causing periodic ~16-second stalls that prevented the validator from producing in-turn blocks on BSC. Pre-warm the overlay_cache on the engine loop thread (preserving the existing changeset-cache eviction race fix), drop the provider immediately, then open a fresh short-lived read transaction inside the task just before compute_trie_changesets. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
… changes Reverts commits 1486308 through 5aa3129: - Remove debug/warn logs added for engine message flow tracing - Remove tracing dependency added to reth-engine-primitives - Remove backpressure warn logs - Remove post-execution and state root hang warn logs - Revert receipt-root task flush added for post-finalization receipts - Remove timeout on triedb prefetch result wait Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…189) * metrics: gate heavy scrape hooks behind RETH_DISABLE_HEAVY_METRICS The two custom hooks registered by `metrics_hooks` walk MDBX page metadata + the freelist DB and iterate every static-file jar header on every Prometheus scrape. On large databases this is expensive enough to stall the metrics endpoint and starve the runtime. Skip registering both hooks when `RETH_DISABLE_HEAVY_METRICS` is set in the environment; the rest of the registry (process, jemalloc, io, chain spec, version) is unaffected and the endpoint still responds normally. The env var is documented on the function so the escape hatch is discoverable without grepping the source. * metrics: run scrape handler and push-gateway render on spawn_blocking The Prometheus metrics handler is fundamentally synchronous: it invokes every registered hook and then runs the prometheus exporter's `render()`, all on the tokio worker that accepted the HTTP request (or on the runtime worker driving the push-gateway loop). The default hooks are cheap (procfs, jemalloc atomic reads), but the two `report_metrics` hooks (DB stat walk, static-file jar enumeration) can take seconds on large archives. Even with those gated out (see preceding patch), `render()` itself is O(total time-series) and will grow over time. A multi-millisecond synchronous block on a runtime worker is not ideal and can become a real engine latency source if hook cost ever regresses. Move the synchronous work off the runtime worker: - Endpoint handler now offloads `handle_request` (which calls the hook + render or the pprof dump) to `spawn_blocking`. On join error, return a 500 instead of letting the connection task panic. - Push-gateway loop offloads the hook + render to `spawn_blocking`; on join error, log a warning and skip this tick rather than killing the loop. The HTTP put itself was already async so it stays inline. No behavioral change to what the endpoint or push-gateway returns; only the thread on which the rendering happens.
# Conflicts: # crates/node/builder/src/launch/common.rs
…int config Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…tion, and lint Remove Optimism jobs and update workflow files to match upstream paradigmxyz/reth v2.2.0: depot runner conditionals, SHA-pinned actions, ethereum-only matrices, and MSRV bumped to 1.93. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Add liburing-dev, pkg-config, and libclang-dev install steps to jobs that build with BSC features (rocksdb, io-uring). Upstream paradigmxyz/reth omits these since it does not use rocksdb or io-uring. Also add rustfmt component to nightly toolchain steps where librocksdb-sys build script requires it. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Remove op-reth from the matrix; upstream already dropped it in v2.2.0. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Restore `if: github.event_name == 'merge_group'` so stage-run-test only runs in merge groups, not on every PR - Remove account-hashing, storage-hashing, and hashing stages: in V2 storage mode (default), these stages unwind the hashed state written by execution but their no-op execute never restores it, causing merkle to produce the genesis state root Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Reverts 2778a06 Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
chee-chyuan
added a commit
to bnb-chain/reth-bsc
that referenced
this pull request
May 28, 2026
Switches all reth-* dependencies from branch = "develop-v2.2-new" to rev = "485e37b738754f96603391f42ee529f9364b47a4" for deterministic builds. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
chee-chyuan
added a commit
to bnb-chain/reth-bsc
that referenced
this pull request
May 28, 2026
Switches all reth-* dependencies from rev de11c921 to rev = "485e37b738754f96603391f42ee529f9364b47a4" for deterministic builds. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
20 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Merges
develop-v2.2-newintodevelop, bringing in the v2.2 upgrade based on upstream reth v2.2.0. This PR pulls in three major upstream releases plus BSC-specific patches developed during the upgrade work.Upstream release notes:
db migrate-v2, revm 38, alloy 2.0, RocksDB sync fixMajor upstream changes by category
Storage v2 (v2.0 default, v2.1 migrator)
reth db migrate-v2for one-shot v1 → v2 migration without full resync (#23422, pruned-node fix #23716).Engine backpressure (v2.0)
--engine.persistence-backpressure-thresholdflag (default16) that stalls new payload processing when the canonical-to-persisted gap reaches this threshold (#23280, #23308). Validated at startup to be strictly greater than--engine.persistence-threshold.--engine.memory-block-buffer-target. On chains with sub-second block intervals, configurations where steady-state gap (≈memory_block_buffer_target) is ≥ backpressure threshold will continuously stall the engine. See Test focus / Known risks below.Performance (v2.0)
Dependency upgrades
gas_refundednon-zero).RocksDB / Storage (v2.1)
WriteOptions::sync=trueby default to prevent corruption on ungraceful shutdown (#23603). Directly affects BSC triedb path.Networking (v2.2)
snap/2wire helpers (#23611).Weakreference to release port on shutdown (#23282).RPC additions
debug_intermediateRoots(#22754)debug_traceBadBlock(#22719)eth_getStorageValuesbatch fetcher (#22186)reth_getBlockExecutionOutcome/reth_forkchoiceUpdated/reth_newPayload(#22397, #22536, #22533)transactionReceiptssubscription ineth_subscribe(v2.1, #23485)admin_nodeInfo.idreturns keccak256 node ID matching go-ethereum format (#23319)Bug fixes worth noting
reth_newPayload(#22239)MDBX_NOTLSwithMDBX_NOSTICKYTHREADS(#23378)eth_createAccessListandeth_estimateGas(#23026, #22959)Breaking SDK changes (selected, full list in release notes)
reth-primitivescrate removed, replaced byreth-ethereum-primitives(#23220)SerdeBincodeCompatremoved; ExEx WAL now uses RLP (#23158)PayloadBuilderAttributesremoved (#23202)TaskSpawnertrait removed in favor of concreteRuntime(#22052)reth-engine-servicecrate removed (#22187)reth-primitives-traitsmoved to separatereth-corerepo (#23186, #23210)BSC-specific additions in this PR
perf(metrics): prevent multi-second engine stalls from scrape hooks viaspawn_blockingParliaSnapshotBlobandoverlay_cachereferencesTest focus
High priority — likely BSC-affected behaviour
migrate-v2from existing v1 datadir — both paths must produce identical canonical chain. Seereth db migrate-v2docs. Note thatmigrate-v2resetsSenderRecovery/TransactionLookup/IndexAccountHistory/IndexStorageHistory/MerkleExecute/MerkleUnwindcheckpoints to 0 by design; pipeline rebuilds these locally from preserved Headers/Bodies in static files.--engine.persistence-backpressure-threshold > --engine.persistence-threshold ≥ --engine.memory-block-buffer-targetin deploy scripts, especially on fast block-time testnets. Steady-statepersistence_gapshould stay below backpressure threshold or engine will stall on every block.sync=true— measure triedb persist latency, ensure no regression in block import / payload build time.Medium priority
reth db migrate-v2on a pruned datadir (#23716).reth_*,debug_intermediateRoots,transactionReceiptssubscription).NOSTICKYTHREADSsemantics change — verify long-lived RPC read transactions still behave correctly.Sanity checks
make build,make maxperf,cargo clippy --workspace --tests --all-features, ef-tests.Known risks / compatibility
db migrate-v2.gas_refundednon-zero.