feat(unit-tests): verify transition tool support

[winsvega]

🗒️ Description

Despite checks, and unit test pass, evmone t8n input is still inconsistent
geth t8n requires 0x20000 format for hex input and support decimals
evmone t8n requires 0x020000 format for hex input and does not support decimals
eest t8n supports all

first I introduce a unit test CI to verify that test filling works correctly on t8n's
TODO
design a simple test that triggers interesting t8n input on all forks. (with blockheader history and all fork fields in env) to run as a reference

🔗 Related Issues

Current coverage script is broken because of t8n format inconsistency with evmone after changing fields from int to hex in:

• https://github.com/ethereum/execution-spec-tests/pull/1027/files

✅ Checklist

• All: Set appropriate labels for the changes.
• All: Considered squashing commits to improve commit history.
• All: Added an entry to CHANGELOG.md.
• All: Considered updating the online docs in the ./docs/ directory.
• Tests: All converted JSON/YML tests from ethereum/tests have been added to converted-ethereum-tests.txt.
• Tests: A PR with removal of converted JSON/YML tests from ethereum/tests have been opened.
• Tests: Included the type and version of evm t8n tool used to locally execute test cases: e.g., ref with commit hash or geth 1.13.1-stable-3f40e65.
• Tests: Ran mkdocs serve locally and verified the auto-generated docs for new tests in the Test Case Reference are correctly formatted.

[danceratopz]

Thanks @winsvega for identifying and fixing the bug. I'm not sure about the approach here for testing the t8n tool (more below), so I moved your fix to a separate PR #1268 to get that merged more quickly.

I agree that it's really important to add better testing for our t8n support/compatibility. I would suggest however, to move all these tests to our unit tests under ./src/ and improve the unit testing flow so that it can run with any t8n tool.

Then, we add jobs to the existing tox_verify.yaml that runs these unit tests with every t8n tool we support.

[winsvega]

The issue is that it requires to build geth, evmone, ... t8ns.
I don't think unit tests are building the t8ns

[winsvega]

I update the PR so now it is a unit test. but it will require t8n binaries to be installed before the run.

[spencer-tb]

This will be a nice addition for the future. Note it would have caught this issue: #1276

Some thoughts below:

1. I think we could add a new tox step in tox.ini for this (that only runs when specified locally - not by default). We will likely want to skip running these tests within pytest-framework.ini.
2. Once added to tox.ini we should add this as step to the tox_verify.yaml workflow, but only if we have a docker image churning daily for each client t8n tool. This would be to make sure CI remains fast and we don't have to wait to long for the t8n's to build. Keeping the test run on the PR level.
3. If we don't create docker images, then I think we should instead create a new workflow tox_t8n_verify.yaml that only runs the t8n tests in CI when PRs are merged to main, i.e only on main. I don't mind slow CI on main, but on PRs directly I'd like to keep it fast! Personally I prefer 2).

[marioevz]

Pushed a fix for transaction type 4 not working in Geth's t8n.

[marioevz]

Still failing for some reason, and I didn't have time to debug it.

[winsvega]

Now it fails because eels on Prague rejects the transaction.
Can't reproduce locally.

 test_t8n_support[ExecutionSpecsTransitionTool-Prague]

shall we fix t8n's commit hashes to not crash on t8n changes?

Ah I see eels prague resolution in CI is not done correctly

[marioevz]

This will be a nice addition for the future. Note it would have caught this issue: #1276

Some thoughts below:

1. I think we could add a new tox step in `tox.ini` for this (that only runs when specified locally - not by default). We will likely want to skip running these tests within `pytest-framework.ini`.

2. Once added to `tox.ini` we should add this as step to the `tox_verify.yaml` workflow, but **only if** we have a docker image churning daily for each client t8n tool. This would be to make sure CI remains fast and we don't have to wait to long for the t8n's to build. Keeping the test run on the PR level.

3. If we don't create docker images, then I think we should instead create a new workflow `tox_t8n_verify.yaml` that only runs the t8n tests in CI when PRs are merged to main, i.e only on main. I don't mind slow CI on main, but on PRs directly I'd like to keep it fast! Personally I prefer 2).

@spencer-tb I agree that we should keep CI fast, but if you check the actions run for this PR, the longest one takes 6 minutes, and since they are running in parallel I think it should be ok.

I think the biggest offender is not the building of evmone but rather the run_in_serial tests in my opinion.

Most of the issues is that they instantiate an EELS resolver instance per parametrized test, so I actually have an idea on how to address those:

• We should add a parameter to fill that accepts a transition tool server endpoint.
• Run the run_in_serial tests by passing the server endpoint as a a parameter to avoid re-instantiating each time.

Let me know what you think, I'm going to go ahead and merge this PR and we can revisit speeding up in the way I mention above.

[marioevz]

LGTM, thanks for the changes.