feat(fill): implement address tag resolution for static test fillers

[fselmo]

🗒️ Description

Hard-coded address conversion in .yml and .json static fillers:

• Add convert_addresses.py script to automate tag conversion
• The correct way to run this is with the CONVERT_COINBASE flag set to False as this allows the same coinbase for all tests (just as python tests do). If we decide we want to handle the coinbase setting on the python side, we can turn this flag on and hard-code on the python side... but the current approach seems correct.
• Convert static test YAML/JSON files to use address tags
• Use fine tuning for certain tests. Selective script options:
  • DO_NOT_TAG_ADDRESSES: Don't tag certain addresses crucial for some tests but only for addresses that were made sure to not appear in any other test.
  • SHORT_NAME_FILLERS: This finds and replaces short name for addresses (e.g. 0x0000...000dead1 -> 0xdead1). This turns on a few more fillers that were not working.
    NO_TAGS_IN_CODE: Don't tag matches inside bytecode sections for these tests: data, code, storage.

Generate deterministic addresses from tags coming from static test fillers in python code:

• Resolve tags to deterministic addresses in the same way python tests do - via pytest static filler plugin
• Add BlockchainEngineXFixture support for pre-allocation groups

This enables static tests to use symbolic address tags instead of hardcoded addresses, minimizing muddied context across tests when running via pre-alloc sharing.

---

Note: This is a very large change with very many nuances. This is a first pass at a generalized approach. I think maybe we can add complexity and attempt at turning on more tests eventually.

---

🔗 Related Issues

related to #1750

✅ Checklist

• All: Set appropriate labels for the changes.
• All: Considered squashing commits to improve commit history.
• All: Added an entry to CHANGELOG.md.
• All: Considered updating the online docs in the ./docs/ directory.
• Tests: All converted JSON/YML tests from ethereum/tests/tests/static have been assigned @ported_from marker.
• Tests: A PR with removal of converted JSON/YML blockchain tests from ethereum/tests have been opened.
• Tests: Included the type and version of evm t8n tool used to locally execute test cases: e.g., ref with commit hash or geth 1.13.1-stable-3f40e65.
• Tests: Ran mkdocs serve locally and verified the auto-generated docs for new tests in the Test Case Reference are correctly formatted.
• Tests: For PRs implementing a missed test case, update the post-mortem document to add an entry the list.

[danceratopz]

Excited about this! Just a few minor comments after a fleeting review. I can try it out and have a closer look tomorrow!

[fselmo]

I've managed to fine tune quite a few fillers that were not passing the tagging system. There are still edge cases I think can be cleaned up and I think some of the stTransactionTest cases I can still fine tune. All tests are passing if ran with -k "not stTransactionTest" as I still think I can turn more of those on but this is ready for another set of eyes. Commits will be cleaned up as well. These are the stats on tagged vs not tagged, excluding /stTransactionTest/ (so for all passing tests):

Total filler files in tests/static (excluding stTransactionTest): 2,444
   Successfully tagged files: 1,751 (71.6%)
   Untagged files: 693 (28.4%)

My hope is there are more tests that can be turned on as well for tagging but examining all the tests and figuring out general approaches was my first attempt and I wanted to get this to a good first pass since it has taken up a lot of dev time.

The list of currently incompatible tests is here. This could be converted to a list in the documentation somewhere once it's finalized for this first pass.

[fselmo]

OK, I made my last pass through the stTransactionTest tests and turned off all the ones that are incompatible. This is good to review for a first pass.

[fselmo]

Hmm it looks like running with -m "not slow and tagged" --generate-pre-alloc-groups -n 20 has a few failures, a handful of them being read timeouts and I'm not sure if it's a domino effect or what is happening there. Will need to investigate.

---

edit: I actually need to re-investigate the enginex flow altogether after the pydantic changes. It doesn't seem to be working as expected.

[fselmo]

Ok @danceratopz @marioevz, there are a few things to consider here.

• The commit directly after Mario's fixes the issues post-pydantic refactor. This had to turn off some more tests but this was expected on my end as they related mostly to create tests (shouldnotexist in post state) and I had to account for those pre-pydantic refactor still.
• The commit after that is for using a pytest plugin to be able to use -m "tagged" or -m "untagged" markers for collecting tagged static tests. This has been very useful for filling with --generate-pre-alloc-groups.
• The very last commit is for hashing the pre and post states of a coinbase address and resolving its tag to based on this hash so that tests like stRandom where the coinbase has set code in the pre are able to use the same shared pre alloc. Before this change, something like uv run fill --fill-static-tests --clean --generate-pre-alloc-groups tests/static -m "not slow and tagged" -k stRandom -n 20 would create something like 501 groups for 503 tests. After this change it goes to 11 groups for 503 tests.

As of now, even before the last commit for the shared coinbase logic, I am getting timeout failures when running the tests with the shared pre alloc: uv run fill --fill-static-tests --clean --generate-pre-alloc-groups -m "not slow and tagged" tests/static -n 20. This is something to consider when reviewing but otherwise the tests fill if no shared pre flag is passed. I'm not entirely sure where these failures are coming from so any clarity would be good here. Thanks!

[fselmo]

I just pushed a rebase with main. Still looking into why --generate-pre-alloc-groups fails for transStorageResetFiller but doesn't fail when running without shared pre.

[marioevz]

Amazing work! 🎉

This paves the way to executing every single Ethereum test in history in less than an hour compared to tens of hours!