Skip to content

refactor(l1): deduplicate the account-update loop in Store - #7184

Open
ilitteri wants to merge 4 commits into
mainfrom
dedup-store-account-update-loop
Open

refactor(l1): deduplicate the account-update loop in Store#7184
ilitteri wants to merge 4 commits into
mainfrom
dedup-store-account-update-loop

Conversation

@ilitteri

@ilitteri ilitteri commented Aug 21, 2026

Copy link
Copy Markdown
Collaborator

Motivation

Store::apply_account_updates_from_trie_batch and Store::apply_account_updates_from_trie_with_witness were two hand-maintained copies of the same per-update algorithm: remove the account if update.removed; decode-or-default the AccountState; reset storage_root on removed_storage; copy nonce/balance/code_hash and collect code_updates; apply added_storage to a storage trie and collect_changes_since_last_hash; insert the account; finally collect state changes into an AccountUpdatesList. Any future change to the update semantics had to be made twice, and the copies had already drifted cosmetically (hashed-address type, blank-line style), which obscured that the only real difference is how the storage trie is acquired.

Description

Extracts the loop into a private apply_account_updates_from_trie_inner(&self, state_trie, account_updates, storage_tries: Option<&mut StorageTries>) and turns both public functions into thin wrappers with their exact original signatures. crates/storage/store.rs is the only file touched: +46/−79, a net −35 lines of code.

Line accounting (code lines only — blank and comment-only lines counted as zero, the way tokei counts):

before after
apply_account_updates_from_trie_batch 59 7 (wrapper)
apply_account_updates_from_trie_with_witness 67 13 (wrapper)
apply_account_updates_from_trie_inner 71
total 126 91

Of that, the duplicated loop bodies go 113 → 64 code lines; the three signatures plus their delegating calls cost 27 where the two originals cost 13. Blank lines were not touched to move the number: the diff's blank-line count is +1 and its comment count +1, and the surviving loop keeps the blank-line structure of apply_account_updates_from_trie_with_witness, the function it was extracted from.

The merged body is byte-identical (modulo blank lines) to main's batch loop body everywhere except the storage-trie acquisition block, and identical to main's witness body except for that block, the hashed-address type, and the return shape. Both diffs are small enough to read line by line, which is how each invariant below was checked.

The two variants' only genuine differences are preserved verbatim behind the Option:

  • Storage-trie acquisition (the load-bearing difference): None (batch) opens a fresh trie per account via open_storage_trie at that account's current storage_root, exactly as before. Some(map) (witness) uses the entry API on the map keyed by unhashed update.address: Occupied reuses the cached (TrieWitness, Trie) — the cached trie's in-memory state intentionally wins over account_state.storage_root, including ignoring an EMPTY_TRIE_HASH reset from removed_storage — and Vacant opens the trie after the removed_storage reset, wraps it in TrieLogger::open_trie, and inserts it into the map so witness nodes for storage tries first touched during update application are recorded; the mutated map is returned to the caller as before.
  • The shared open_storage_trie call lives in a closure so it stays lazy: an Occupied hit never opens a trie, just as it never did before.
  • Hashed address: unified on hash_address_fixed (H256). The witness copy used hash_address (Vec<u8>) and converted back with H256::from_slice; both are the same keccak digest of the address, so trie keys and storage_updates keys are byte-identical. This also drops the H256::from_slice length-panic path.
  • state_root passed to open_storage_trie remains the single pre-loop hash_no_commit snapshot, not recomputed per iteration.
  • Borrow checking: the Option<&mut StorageTries> is reborrowed per iteration with as_deref_mut(), and the batch arm parks its fresh trie in a per-iteration Option local so both arms unify as &mut Trie.

Public API, RLP encoding, DB schema, and STORE_SCHEMA_VERSION are untouched. No tests were deleted or modified.

If this change were wrong, one of these would have to be true:

  • hash_address and hash_address_fixed would have to produce different bytes for some address — they are both keccak(address.to_fixed_bytes()) (store.rs), one returning Vec<u8>, the other H256.
  • The witness variant would have to depend on re-opening a cached storage trie at the post-removed_storage root — it cannot: the Occupied arm never consulted account_state.storage_root before this change either, and the closure that would do the opening is not called on that arm.
  • A caller would have to key StorageTries by hashed address — callers in blockchain.rs populate and drain it keyed by unhashed Address, which this change keeps.
  • The batch variant would have to rely on storage-trie state carrying across accounts — it opened a fresh trie per account before and still does, and its body is otherwise byte-identical to what it replaced.

How to test

cargo fmt --all --check
cargo clippy -- -D warnings                  # what CI's L1 Lint job runs
cargo clippy --workspace --all-targets -- -D warnings
cargo test -p ethrex-storage
cargo test -p ethrex-blockchain              # covers the batch and witness call sites
cargo test -p ethrex-test --test ethrex_tests batch

Results on this branch:

  • cargo fmt --all --check — clean.
  • cargo clippy -- -D warnings — clean, exit 0. This is the exact command the L1 Lint job runs, and it is the gate that previously failed on this branch with clippy::empty_line_after_doc_comments.
  • cargo clippy --workspace --all-targets -- -D warnings — clean.
  • cargo test -p ethrex-storage — 91 passed, 0 failed (plus doctests: 1 passed, 1 ignored).
  • cargo test -p ethrex-blockchain — 58 passed, 0 failed.
  • cargo test -p ethrex-test --test ethrex_tests batch — 30 passed, 0 failed. Includes batch_selfdestruct_created_account_no_spurious_state, which exercises exactly the default-AccountState + removed_storage + state-root-parity semantics of this loop end to end, plus account_batch_parity_in_memory and storage_batch_parity_in_memory.
  • ef-tests: the blockchain and engine suites run green on this branch. Nothing in this diff touches serialization or API surface.
  • Not covered by any local test, before or after: witness completeness for storage tries first touched during update application. There is no fast fixture for it — the stateless harness needs the separate vectors_zkevm bundle — so that invariant was checked by reading the Vacant arm against the original line by line: it still wraps in TrieLogger::open_trie, still inserts into the map, and the wrapper still returns the mutated map.

Checklist

  • No Store schema change, so STORE_SCHEMA_VERSION (crates/storage/lib.rs) is untouched — this is a private helper extraction; public API, RLP encoding and DB layout are identical, so no re-sync is required.

apply_account_updates_from_trie_batch and
apply_account_updates_from_trie_with_witness ran the same per-update
algorithm, copy-pasted. Extract it into a private
apply_account_updates_from_trie_inner and turn both public functions
into thin wrappers with unchanged signatures.

The variants' only real differences stay behind an
Option<&mut StorageTries>: None opens a fresh storage trie per account
(batch); Some caches tries in the map keyed by unhashed address,
reusing the logged trie on hit and wrapping newly opened ones in
TrieLogger::open_trie so witness recording spans updates. Hashed
addresses unify on hash_address_fixed, the same keccak digest the
witness copy re-wrapped with H256::from_slice.
@github-actions

github-actions Bot commented Aug 21, 2026

Copy link
Copy Markdown

⚠️ Known Issues — intentionally skipped tests

Source: docs/known_issues.md

rpc-compat log-bearing cases excluded

Where: KNOWN_EXCLUDED_TESTS in .github/scripts/check-hive-results.sh counts out
eight hive rpc-compat cases — the four eth_getLogs cases, eth_getBlockReceipts/get-block-receipts-latest,
and three eth_getTransactionReceipt cases. They are exactly the cases whose recorded
response contains at least one log object; every case with an empty log array still runs.
Note this leaves eth_getLogs with no rpc-compat coverage at all, since all four of its
cases are in the set.

Why: ethrex populates blockTimestamp on log objects, as geth, besu, nethermind, reth
and erigon all do. hive's rpc-compat compares responses byte-exactly (jsondiff.FullMatch;
the lenient checkJSONStructure path applies only to cases upstream marks speconly), and
the corpus is pinned to execution-apis d08382ae (2025-02-10), whose recordings predate the
field — it entered the schema in execution-apis#639 and the fixtures in #846 (2026-07-22).
So the extra key cannot match, and this is a property of the pin rather than of the response.

The pin cannot move, and this is not temporary. The pin sits one commit before
execution-apis#627, which moved the test chain to a pre-merge genesis: the current corpus has
~36 proof-of-work blocks before its terminal total difficulty. ethrex does not support
pre-merge chains and will not, so importing that chain.rlp fails at block 1 —
validate_block_header has no pre-London base-fee path. Every revision carrying
blockTimestamp in its fixtures also carries that chain, so there is no revision that
satisfies both. Nor can the corpus be patched locally: rpc-compat's Dockerfile clones
ethereum/execution-apis by hard-coded URL, so the branch buildarg cannot point at a fork.

Coverage: the field itself is pinned by
block_timestamp_is_on_the_log_and_not_on_the_receipt in
crates/networking/rpc/types/receipt.rs, which asserts it is present on each log and absent
from the receipt level.

Removal: delete the entries if ethrex ever gains pre-merge chain import, or if upstream
marks these cases speconly so they are type-checked instead of compared byte-for-byte.


The stateless schema id does not identify the encoding

Where: STATELESS_INPUT_SCHEMA_ID in crates/common/types/stateless_ssz.rs.

Upstream keeps the stateless input schema id at 0x1501
(fork_index 0x15 << 8 | revision 0x01) across incompatible body changes. Three
encodings have now shipped under it: tests-zkevm@v0.6.2, then #3248 + #3278,
then #3356, which moved state, codes and public_keys from SszList to
ProgressiveList. ethrex speaks the last one.

The consequence is that the 2-byte prefix cannot be used to detect a stale or
mismatched bundle. A wrong-dialect input is accepted by the id check and then
fails later — in SSZ decode, or on a root that does not match — rather than being
rejected up front for what it is. only_amsterdam_schema_id_decodes therefore
proves less than its name suggests.

Worth raising upstream: a revision field that does not move across a body change
provides no version negotiation at all.


ZisK guest program hash changes with the unsync_cell gate

Where: crates/common/types/block.rs, transaction.rs.

The gate on the single-threaded unsync_cell::OnceCell moved from
all(feature = "eip-8025", target_arch = "riscv64") to
all(feature = "zisk", target_arch = "riscv64") when the eip-8025 feature was removed.

The guest ELFs were previously built --features "<zkvm>-build-elf,ci", which never enabled
eip-8025, so they compiled the atomic once_cell variant. bin/zisk/Cargo.toml does enable
ethrex-common/zisk, so the ZisK guest now compiles the unsafe impl Sync cell instead.
That changes the ELF bytes and therefore the program hash and verification key.

This is intended (the guest is single-threaded, so the unsync cell is sound and cheaper), but it
is a VK change rather than a no-op refactor, and the diffstat presents it as a file rename
(eip8025_cell.rsunsync_cell.rs). Anyone pinning a ZisK VK across this change must
re-register it. The stateless-validator crate now forwards ethrex-common/zisk from its own
zisk feature so the two ZisK guests do not disagree on the cell type.


Release signing key is an unprotected repository secret

Where: .github/workflows/tag_release.yaml.

MINISIGN_SECRET_KEY is a plain repository secret. There is no environment: on
finalize-release or dry-run-release-assets, and gh api repos/lambdaclass/ethrex/rulesets
shows only branch-targeted rulesets, so the github.ref_type == 'tag' condition is a workflow
check rather than an enforced boundary: anyone who can push a tag can reach the signing key.

This is a repository-settings change, not a code change, so it is recorded here rather than
fixed in the tree. Recommended:

  1. Move MINISIGN_SECRET_KEY / MINISIGN_PASSWORD into a GitHub Environment with required
    reviewers, and add environment: to the two jobs that sign.
  2. Add a ruleset targeting refs/tags/v* restricting who may create release tags.

Until then, the compromise of that key is silent and durable: signatures would still verify
against the committed .github/minisign.pub.

@github-actions github-actions Bot added the L1 Ethereum client label Aug 21, 2026
@github-actions

Copy link
Copy Markdown

🤖 Codex Code Review

  1. crates/storage/store.rs:2444-2457 introduces a state-corruption path in the witness-enabled flow. You set account_state.storage_root = EMPTY_TRIE_HASH at 2429-2431, but if storage_tries already contains that account, Entry::Occupied reuses the previously opened trie anyway. That is wrong for accounts with removed_storage=true (emitted for “destroyed then modified” accounts in crates/vm/levm/src/db/gen_db.rs:629-644 and 737-752). A realistic case is: read an existing slot during execution, then selfdestruct/recreate or selfdestruct/send ETH, then write new storage. The cached trie still points at the old storage root, so old slots can leak into the recomputed root/diff instead of starting from an empty trie. This is not just a witness artifact: crates/blockchain/blockchain.rs:1961-1971 persists the returned account_updates_list. The occupied entry needs to be reopened/reset when removed_storage is true (more generally, whenever the cached trie no longer matches account_state.storage_root), and this needs a regression test.

Other than that, the refactor looks mechanically equivalent.


Automated review by OpenAI Codex · gpt-5.4 · custom prompt

@github-actions

Copy link
Copy Markdown

🤖 Claude Code Review

I have enough to complete the review based on careful manual trace of the diff (the two original implementations vs. the unified inner function), confirming key equivalences (hash_address vs hash_address_fixed, storage trie acquisition strategy, borrow-checker pattern with local_trie/as_deref_mut).

Review: PR 7184 — deduplicate account-update loop in Store

This is a clean, mechanical refactor with no behavioral changes intended, and the trace confirms it delivers on that.

Correctness

  • Hash equivalence (Item 1): hash_address_fixed (H256, used via .as_bytes()) now replaces hash_address (Vec<u8>) in the witness path. Both are keccak(address.to_fixed_bytes()), just different return types, so trie keys and the (H256, storage_updates) pairs pushed into ret_storage_updates are byte-identical to before. Verified hash_address is still used elsewhere (blockchain.rs, block_execution_witness.rs, tests), so it's not now dead code.
  • Storage-trie acquisition split (store.rs:2442-2465): the None arm (batch) opens a fresh trie every iteration via open_storage_trie, matching the pre-refactor batch behavior exactly (no caching across iterations). The Some(tries) arm preserves the original entry-API caching/witness-wrapping behavior verbatim, including the subtlety that an Occupied entry's in-memory storage root wins over account_state.storage_root (i.e., a removed_storage reset to EMPTY_TRIE_HASH is ignored on cache hit) — this matches the original witness function's behavior, so no regression, though it's a preexisting sharp edge worth a comment if not already documented elsewhere.
  • Deferred-init borrow pattern (local_trie at store.rs:2443, reborrow via storage_tries.as_deref_mut() at store.rs:2444): sound. as_deref_mut() reborrows Option<&mut StorageTries> each iteration instead of moving it, and local_trie is only read after being assigned along the only path that reaches it. Both match arms produce &mut Trie, so the types unify correctly.
  • Loop-invariant state_root: still computed once via hash_no_commit before the loop in the unified function, not per-iteration — matches both originals.

Style / maintainability

  • Good use of impl IntoIterator + Option<&mut StorageTries> to unify the two call shapes without behavior-changing generalization.
  • The doc comment on apply_account_updates_from_trie_inner (store.rs:2401-2406) accurately describes the semantic difference between the two modes — helpful since this is exactly the kind of divergence that caused the original duplication/drift.
  • Both public wrappers keep their exact original signatures, so this is invisible to callers (blockchain.rs, tests) — no ripple-effect changes needed elsewhere, which the diff confirms.

Minor nit

  • apply_account_updates_from_trie_batch (store.rs:2377-2383) is pub while apply_account_updates_from_trie_inner is private — consistent with pre-existing visibility, not introduced by this PR, so no action needed.

No correctness, security, or performance issues found. The refactor is a faithful, well-verified deduplication — net reduction in code with preserved semantics.


Automated review by Claude (Anthropic) · sonnet · custom prompt

@github-actions

github-actions Bot commented Aug 21, 2026

Copy link
Copy Markdown

Lines of code report

Total lines added: 0
Total lines removed: 35
Total lines changed: 35

Detailed view
+--------------------------------+-------+------+
| File                           | Lines | Diff |
+--------------------------------+-------+------+
| ethrex/crates/storage/store.rs | 5072  | -35  |
+--------------------------------+-------+------+

Comment thread crates/storage/store.rs
)?))
}

pub fn apply_account_updates_from_trie_batch<'a>(

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this function can be inlined. I don't see any uses apart from the one in apply_account_updates_batch.

Comment thread crates/storage/store.rs
account_state.balance = info.balance;

account_state.code_hash = info.code_hash;

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This isn't a good way to reduce lines of code.

Comment thread crates/storage/store.rs
Comment on lines -2529 to +2489
let account_updates_list = AccountUpdatesList {
Ok(AccountUpdatesList {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is also not a good way to reduce lines of code

Comment thread crates/storage/store.rs
Comment on lines +2481 to +2484
state_trie.insert(
hashed_address.as_bytes().to_vec(),
account_state.encode_to_vec(),
)?;

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This would be a good way to reduce lines of code:

Suggested change
state_trie.insert(
hashed_address.as_bytes().to_vec(),
account_state.encode_to_vec(),
)?;
let hashed_address_vec = hashed_address.as_bytes().to_vec();
state_trie.insert(hashed_address_vec, account_state.encode_to_vec())?;

@github-project-automation github-project-automation Bot moved this to In Progress in ethrex_l1 Aug 21, 2026
…cal blocks is deliberate readability structure and never counted as lines of code — the reduction here is the deduplicated logic, not whitespace.
…elong

The previous commit's blank lines were inserted mechanically rather than at
logical-block boundaries: one landed between a doc comment and the item it
documents, which clippy rejects as empty_line_after_doc_comments and which
failed `cargo clippy -- -D warnings`; others split a `let` binding from its
right-hand side, split a call's argument list, and sat immediately after an
opening brace or before a closing one. Two more landed in
setup_genesis_state_trie, a function this change does not otherwise touch.

Restore the spacing from the source the shared loop was actually extracted
from: apply_account_updates_from_trie_with_witness. Its body is the one that
survives in apply_account_updates_from_trie_inner, so its blank lines go back
exactly where they were and nowhere else. The deleted batch copy had no such
spacing to preserve, and setup_genesis_state_trie returns to its original
form. Blank lines are not lines of code either way; the reduction remains the
deduplicated logic.
…te loop

The extraction landed the per-variant switch as a match wrapping a match
wrapping a block, with the open_storage_trie call spelled out in full in
both arms. Collapsing it — one closure for the shared fresh-open call,
`&mut ....1` instead of re-destructuring the cached (witness, trie) pair,
and Option::insert instead of a deferred-init local — takes the
acquisition from 24 code lines to 13 and drops a nesting level, with both
variants' semantics untouched: Occupied still reuses the cached trie
without consulting account_state.storage_root, Vacant still wraps the new
trie in TrieLogger::open_trie and inserts it into the map keyed by the
unhashed address, and the storage-tries-less variant still opens a fresh
trie per account at that account's current storage root. The closure keeps
the open lazy, so an Occupied hit still never opens a trie.

With this, the shared loop body is byte-identical to the batch loop it
replaced everywhere except the acquisition block, and the whole
deduplication is 126 -> 91 code lines across the two entry points.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

L1 Ethereum client

Projects

Status: In Progress

Development

Successfully merging this pull request may close these issues.

2 participants