Skip to content

refactor(test-fill): rebuild the Engine X drift check on typed fixture models - #3265

Draft
spencer-tb wants to merge 6 commits into
ethereum:forks/amsterdamfrom
spencer-tb:refactor/engine-x-checks
Draft

refactor(test-fill): rebuild the Engine X drift check on typed fixture models#3265
spencer-tb wants to merge 6 commits into
ethereum:forks/amsterdamfrom
spencer-tb:refactor/engine-x-checks

Conversation

@spencer-tb

Copy link
Copy Markdown
Contributor

Description

Rebuild engine_x_checks.py per the review feedback on #3219: fixtures were compared as black-box JSON dicts, the BAL parent-hash mask was a byte-pattern match over the whole RLP tree, and drift failures gave no indication of cause.

  • Parse both fixture trees with BlockchainEngineXFixture/BlockchainEngineFixture; a fixture that fails typed validation is a loud error naming file and test id, never a silent degradation of the check.
  • Decode BALs with BlockAccessList.from_rlp and mask only the EIP-2935 history-contract write whose value equals the payload's own parent hash (integer comparison, so the leading-zero trimming special case disappears). A parent-hash-valued write to any other account is drift again, and there is a regression test for it.
  • Classify and aggregate drifts by cause: a leaked account is traced through the sibling fixture's pre and the pre_alloc group file to the address, the group, and the tests that packed it; identical causes collapse into one diagnosis instead of one failure per fixture.
  • verify_engine_x_execution always returns a result with an explicit skip_reason, simplifying the pytest_sessionfinish caller; the not-clean-exit gate now logs that the check was skipped.
  • The unit tests build real serialized fixtures via the models instead of hand-rolled payload dicts, and guard the field names the check depends on so model renames fail in CI.

Example failure output after injecting a leak into a real fill:

1 of 13 Engine X fixtures execute differently against their packed pre-allocation group's genesis than against their own pre-allocation (1 distinct cause(s)):

[1x] payload 0: account 0xf6c3...b3ff appears in the packed fixture's BAL only
  tests: tests/amsterdam/eip7928_block_level_access_lists/test_block_access_lists_eip2935.py::test_bal_2935_empty_block[fork_Amsterdam-blockchain_test_engine_x]
  the account is absent from the test's own pre-allocation but present in pre-alloc group 0xa58f4980b4cd5258 (11 tests, e.g. ...): an account introduced by packing leaked into this test's execution. Isolate the test with @pytest.mark.pre_alloc_group("separate") or declare the account in its pre-allocation

Validated with tests/amsterdam/eip7928_block_level_access_lists filled at Amsterdam with --generate-all-formats: 1013 Engine X fixtures execute identically against their packed group's genesis, including the corrupted-BAL negative tests; injected gas and BAL-leak drifts produce the diagnostics above and the output verifies green again once restored.

Related Issues or PRs

Stacked on #3219 (includes its commits until it merges; only the last commit is new). Addresses marioevz's engine_x_checks.py review comments on #3219.

Checklist

  • Ran fast static checks to avoid CI fails, see Code Standards & Verifying Changes: just static
  • PR title has the form <type>(<area>): <title>, where <type> and <area> come from an appropriate C-<type>, respectively A-<area>, label. The title should match the target squash commit message.

Cute Animal Picture

Put a link to a cute animal picture inside the parenthesis-->

@codecov

codecov Bot commented Jul 30, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 93.42%. Comparing base (946f45c) to head (a411c77).
⚠️ Report is 2 commits behind head on forks/amsterdam.

Additional details and impacted files
@@               Coverage Diff                @@
##           forks/amsterdam    #3265   +/-   ##
================================================
  Coverage            93.42%   93.42%           
================================================
  Files                  625      625           
  Lines                37032    37032           
  Branches              3385     3385           
================================================
  Hits                 34597    34597           
  Misses                1666     1666           
  Partials               769      769           
Flag Coverage Δ
unittests 93.42% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant