bug(fill): fix misleading test collection reporting in terminal output

[danceratopz]

After the pytest header and test collection (but before test execution) pytest incorrectly reports how many tests are collected. This is visible with and without --collect-only.

For example:

fill tests/istanbul/eip1344_chainid/ --fork=Cancun --collect-only

outputs (incorrect output is marked as such):

================================== test session starts ===================================
platform linux -- Python 3.10.12, pytest-7.4.4, pluggy-1.5.0
Executing tests for: Cancun 
Only executing tests with stable/deployed forks: Specify an upcoming fork via --until=fork to add forks under development.
rootdir: /home/dtopz/code/github/danceratopz/execution-spec-tests
configfile: pytest.ini
plugins: cov-4.1.0, xdist-3.6.1
collecting ... start
collected 3 items / 1 deselected / 2 selected  # <-- INCORRECT, should be "collected 3 items"

<Package eip1344_chainid>
  <Module test_chainid.py>
    <Function test_chainid[fork_Cancun-blockchain_test]>
    <Function test_chainid[fork_Cancun-blockchain_test_hive]>
    <Function test_chainid[fork_Cancun-state_test]>

====================== 2/3 tests collected (1 deselected) in 0.01s =======================   # <-- INCORRECT, should be "3 tests collected in 0.01 s"

For every test module collected, one additional "test" will be incorrectly reported as deselected (above we only collect one module).

The error is due to the fact that we automatically add EIP version tests in the spec_version_checker plugin (src/pytest_plugins/spec_version_checker/spec_version_checker.py) but then also disable them by default in fill's pytest.ini (L9 and L11):

execution-spec-tests/pytest.ini

Lines 6 to 11 in 4783e6e

	addopts =
	-p pytest_plugins.test_filler.test_filler
	-p pytest_plugins.forks.forks
	-p pytest_plugins.spec_version_checker.spec_version_checker
	-p pytest_plugins.test_help.test_help
	-m "not eip_version_check"

The most reasonable solution would be to simply not use this plugin when executing fill and have yet another pytest command dedicated to version checking that can be ran regularly in CI. The reason to not enable these tests by default is that each of these version checking tests makes a request to the Github API (to retrieve info from ethereum/EIPs); without authentication setup, fill quickly hits the Github API rate limit.

Note, as of 2024-04-29, eip version checking is currently entirely unused in our CI. To run these tests, it's required to explicitly select tests marked as eip_version_check with:

fill tests/istanbul/eip1344_chainid/ --fork=Cancun -m eip_version_check -v

[danceratopz]

Related issue #963.

[spencer-tb]

Cc-ing comment from here: #1357 (review)

How to not run into API call issues:

I'm in favour of removing this from fill too, but approving so we can fill Prague!

A separate command in CI would be nice or something that just runs for all the EL EIPs and pings for changes?

The solution around the API throttling is to use the Trees API endpoint. This would allow us to fetch the SHA hashes for all spec files in a single API call rather than making individual requests per file, /git/trees/{branch}?recursive=1 to get all files in one request.