feat(tests): converting `calldataload` and `calldatasize` tests #1236

pacrob · 2025-02-19T23:13:04Z

🗒️ Description

🔗 Related Issues

✅ Checklist

All: Set appropriate labels for the changes.
All: Considered squashing commits to improve commit history.
All: Added an entry to CHANGELOG.md.
All: Considered updating the online docs in the ./docs/ directory.
Tests: All converted JSON/YML tests from ethereum/tests have been added to converted-ethereum-tests.txt.
Tests: A PR with removal of converted JSON/YML tests from ethereum/tests have been opened.
remove calldataload and calldatasize tests tests#1477
Tests: Included the type and version of evm t8n tool used to locally execute test cases: e.g., ref with commit hash or geth 1.13.1-stable-3f40e65.
used geth version 1.15.2-stable-c8c62daf
Tests: Ran mkdocs serve locally and verified the auto-generated docs for new tests in the Test Case Reference are correctly formatted.

winsvega · 2025-03-20T09:53:54Z

tests/frontier/opcodes/test_calldatasize.py

+            Account(storage={0x00: 0x11}),
+        ),
+        (
+            b"\x1a\x84Q\xe6\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00!\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x01",


is there better way to declare bytes ? more readable

shortened and labeled!

winsvega · 2025-03-20T09:54:32Z

evmone build script failing

pacrob · 2025-03-21T20:31:49Z

OK, I think it's gtg. Still getting coverage loss though. How can I see what's lost , and is there a solution? I think my previous PR had the same issue.

===============
COVERAGE REPORT
/tests/DIFF/index.html
New covered Lines: 180
New covered function Paths: 77

ERROR: coverage is lost!
Lost coverage Lines: 26
Lost coverage function Paths: 5

marioevz

Hey, some comments which I think should make the test a bit more EEST-like, but we might lose irrelevant coverage (Like the SHA3 function which is IMO is out-of-scope for these tests) but that's ok.

marioevz · 2025-03-27T18:05:33Z

tests/frontier/opcodes/test_calldataload.py

+
+    Based on https://github.com/ethereum/tests/blob/ae4791077e8fcf716136e70fe8392f1a1f1495fb/src/GeneralStateTestsFiller/VMTests/vmTests/calldatacopyFiller.yml
+    """
+    address_a = pre.deploy_contract(code_for_address_a)


It might be unnecessary to get the entire code as a parameter since it's very similar in all cases, so we could change it only having a calldataload_offset parameter and change this to:

Suggested change

address_a = pre.deploy_contract(code_for_address_a)

address_a = pre.deploy_contract(Op.SSTORE(0, Op.CALLDATALOAD(calldataload_offset)) + Op.STOP)

marioevz · 2025-03-27T18:19:45Z

tests/frontier/opcodes/test_calldataload.py

+            (Op.MSTORE8(offset=0x0, value=0x25) + Op.MSTORE8(offset=0x1, value=0x60)),
+            0x2,


It might be better to change these two parameters to a single one called calldata, which could contain a bytes value as follows:

two_bytes: b'\x25\x60'

word_n_byte: b'\xff' * 32 + b'\x23'

34_bytes: bytes.fromhex('123456789ABCDEF00000000000000000000000000000000000000000000000000024')

or something similar, then in the code we could use the Om.MSTORE(calldata, 0x00) (Macros can be imported by from ethereum_test_tools import Macros as Om) to put the call data in memory and args_size = len(calldata), then finally Op.CALL to the address_a.

Advantages to this method are:

IMO it makes it easier to understand what the test is trying to verify by showing exactly what the calldata is when calldataload is being called.

We can parametrize for generate a test where address_a is the entry point of the transaction and calldata is the data of the transaction.

marioevz · 2025-03-27T18:20:49Z

tests/frontier/opcodes/test_calldataload.py

+    )
+
+    tx = Transaction(
+        data=tx_data,


I think tx_data is not really used anywhere and could be safely removed.

But ideally it should be replaced by calldata as described in the previous comment 👍

marioevz · 2025-03-27T18:21:42Z

tests/frontier/opcodes/test_calldataload.py

+            Account(
+                storage={0x00: 0x2560000000000000000000000000000000000000000000000000000000000000}
+            ),


Could be simplified to:

Suggested change

Account(

storage={0x00: 0x2560000000000000000000000000000000000000000000000000000000000000}

),

0x2560000000000000000000000000000000000000000000000000000000000000,

And then the function defines the Account(storage=... part.

marioevz · 2025-03-27T18:23:16Z

tests/frontier/opcodes/test_calldatasize.py

+@pytest.mark.parametrize(
+    "args_size,sha3_len,storage",
+    [
+        ("0002", "01", Account(storage={0x00: 0x02})),


Suggested change

("0002", "01", Account(storage={0x00: 0x02})),

("0002", "01", 0x02),

Similar comment to previous file, Account(storage=... can be defined inside the function to make the parametrization more readable.

marioevz · 2025-03-27T18:24:48Z

tests/frontier/opcodes/test_calldataload.py

+    tx = Transaction(
+        data=tx_data,
+        gas_limit=100_000,
+        gas_price=0x0A,


Suggested change

gas_price=0x0A,

Can be safely removed, so we can execute this same test in other environments.

marioevz · 2025-03-27T18:25:13Z

tests/frontier/opcodes/test_calldataload.py

+        protected=fork >= Byzantium,
+        sender=pre.fund_eoa(),
+        to=to,
+        value=0x01,


Suggested change

value=0x01,

I think this can also be safely removed since it's not used anywhere.

marioevz · 2025-03-27T18:38:04Z

tests/frontier/opcodes/test_calldatasize.py

+            )
+        )
+    )
+    tx_data = bytes.fromhex("1a8451e6" + "00" * 30 + args_size + "00" * 31 + sha3_len)


No need to encode this as call data to be decoded by the contract, args_size and sha3_len could be integers and they could be placed inside of the contract directly:

to = pre.deploy_contract( code=( Op.MSTORE(offset=0x0, value=Op.SHA3(offset=0x0, size=sha3_len)) + Op.CALL( gas=Op.SUB(Op.GAS(), 0x100), address=address, value=0x0, args_offset=0x0, args_size=args_size, ret_offset=0x0, ret_size=0x0, ) ) )

Although I'm having trouble understanding why the original test is using SHA3 in the first place since the memory contents are irrelevant, and my best guess is to just have a non-zero filled memory.

I think we should replace all of this with calldata just like the comment in test_calldataload.py, and also parametrize this function to test calldatasize at the top-level call (returns len(tx.data)) or a sub-level call (returns arg_size).

@marioevz I simplified it a bit, but I'm pretty sure it's still checking correctly. Anything I'm missing?

winsvega · 2025-04-07T10:44:38Z

lets add push32, mstore8, add opcodes to default opcodes exempt from coverage check requirements.
it is in the file tests/homestead/coverage/test_coverage.py

pacrob · 2025-04-08T00:13:10Z

lets add push32, mstore8, add opcodes to default opcodes exempt from coverage check requirements. it is in the file tests/homestead/coverage/test_coverage.py

Not all fixed, but better. Thanks!

before:

ERROR: coverage is lost!
Lost coverage Lines: 34
Lost coverage function Paths: 11

after:

ERROR: coverage is lost!
Lost coverage Lines: 33
Lost coverage function Paths: 3

pacrob · 2025-04-09T19:51:28Z

Latest coverage fail:

Lost coverage Lines: 56
Lost coverage function Paths: 9

Looking into it:

expected:

Lost 22 lines and 6 functions from lib/evmone due to no longer using keccak or calldataload in the calldatasize test
Lost 7 lines test/state/host.cpp, 3 lines and one function in test/state/host.cpp due to not including a value in the transaction in the calldatasize test.

unknown:

Lost 2 lines in evmc/include/evmc, showing that the test is no longer checking Prague. Why might that be happening?
Leaving 22 lines and 2 functions across mpt.cpp, state.cpp, and system_contracts.cpp. Not sure of source.

I'll keep poking, but if you know whether the last 2 can be fixed or should be ignored, let me know!

winsvega · 2025-04-10T13:40:56Z

About sha3

I don't get what are the inputs for call in Oris test.
I suggest we debug original test, derrive the inputs and reproduce in python.

pacrob force-pushed the calldata-tests branch from 89f3e94 to 5344526 Compare March 19, 2025 17:41

pacrob mentioned this pull request Mar 19, 2025

remove calldataload and calldatasize tests ethereum/tests#1477

Open

pacrob force-pushed the calldata-tests branch 4 times, most recently from 91c493a to f6a91be Compare March 19, 2025 21:44

pacrob marked this pull request as ready for review March 19, 2025 21:45

winsvega reviewed Mar 20, 2025

View reviewed changes

pacrob force-pushed the calldata-tests branch from f6a91be to bfe80f7 Compare March 21, 2025 19:48

marioevz reviewed Mar 27, 2025

View reviewed changes

pacrob force-pushed the calldata-tests branch 2 times, most recently from f21d975 to de67c5c Compare April 4, 2025 20:14

pacrob added 6 commits April 4, 2025 14:34

convert calldataload and calldatasize tests

fcc2017

pass just calldata_offset, not all the code

71338e5

compress mstore and args_size into calldata

776c174

replace tx_data with calldata

470aa61

simplify expected storage

7785ec6

cleanup

982f52f

pacrob force-pushed the calldata-tests branch from eb7658b to 982f52f Compare April 4, 2025 20:34

pacrob force-pushed the calldata-tests branch from c3de54c to 13ce646 Compare April 7, 2025 20:27

add push32, mstore8, and add to exempt from coverage

b07e05c

pacrob force-pushed the calldata-tests branch from 13ce646 to b07e05c Compare April 7, 2025 23:55

pacrob added 4 commits April 8, 2025 10:42

put args_size and sha3_len in contract

47b0067

simplify calldatasize

9b825e2

see what coverage is lost by not sending value

b00596d

remove value

06fbd1b

spencer-tb changed the title ~~converting calldataload and calldatasize tests~~ feat(tests): converting calldataload and calldatasize tests Apr 14, 2025

spencer-tb added scope:tests Scope: Changes EL client test cases in `./tests` port Related to porting ethereum/tests to EEST type:feat type: Feature and removed port Related to porting ethereum/tests to EEST labels Apr 14, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(tests): converting `calldataload` and `calldatasize` tests #1236

feat(tests): converting `calldataload` and `calldatasize` tests #1236

pacrob commented Feb 19, 2025 •

edited

Loading

winsvega Mar 20, 2025

pacrob Mar 21, 2025

winsvega commented Mar 20, 2025

pacrob commented Mar 21, 2025

marioevz left a comment

marioevz Mar 27, 2025

marioevz Mar 27, 2025

marioevz Mar 27, 2025

marioevz Mar 27, 2025

marioevz Mar 27, 2025

marioevz Mar 27, 2025

marioevz Mar 27, 2025

marioevz Mar 27, 2025

pacrob Apr 8, 2025

winsvega commented Apr 7, 2025 •

edited

Loading

pacrob commented Apr 8, 2025

pacrob commented Apr 9, 2025

winsvega commented Apr 10, 2025

	address_a = pre.deploy_contract(code_for_address_a)
	address_a = pre.deploy_contract(Op.SSTORE(0, Op.CALLDATALOAD(calldataload_offset)) + Op.STOP)

		(Op.MSTORE8(offset=0x0, value=0x25) + Op.MSTORE8(offset=0x1, value=0x60)),
		0x2,

	("0002", "01", Account(storage={0x00: 0x02})),
	("0002", "01", 0x02),

feat(tests): converting calldataload and calldatasize tests #1236

Are you sure you want to change the base?

feat(tests): converting calldataload and calldatasize tests #1236

Conversation

pacrob commented Feb 19, 2025 • edited Loading

🗒️ Description

🔗 Related Issues

✅ Checklist

Choose a reason for hiding this comment

Choose a reason for hiding this comment

winsvega commented Mar 20, 2025

pacrob commented Mar 21, 2025

marioevz left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

winsvega commented Apr 7, 2025 • edited Loading

pacrob commented Apr 8, 2025

pacrob commented Apr 9, 2025

winsvega commented Apr 10, 2025

feat(tests): converting `calldataload` and `calldatasize` tests #1236

feat(tests): converting `calldataload` and `calldatasize` tests #1236

pacrob commented Feb 19, 2025 •

edited

Loading

winsvega commented Apr 7, 2025 •

edited

Loading