[Bug-Candidate]: Parsing error when using Medusa corpus

[nican0r]

Describe the issue:

I'm getting the following parsing error when trying to run the tool on a corpus from a Medusa run

and the Foundry test it creates just contains the setup function for the target contract

When using an Echidna corpus it gives no parsing error but creates the same Foundry test

I'm unsure of whether this is due to a misconfiguration on my part or an actual bug in the tool.

Code example to reproduce the issue:

This repo is the one on which the issue appeared, it uses the default Foundry template and the only thing that's been added is an invariant in the FuzzCounter contract: https://github.com/nican0r/fuzz-utils-repro

The FuzzCounter contract also includes the commands I used for running the tool along with the command I used for running Echidna. Please lmk if there's anything else I can add that might help.

Version:

0.0.1

Relevant log output:

Generating Foundry unit tests based on the Medusa reproducers...
Parsing fail on [{'call': {'from': '0x0000000000000000000000000000000000030000', 'to': '0xa647ff3c36cfab592509e13860ab8c4f28781a66', 'nonce': 1, 'value': '0x0', 'gasLimit': 12500000, 'gasPrice': '0x1', 'gasFeeCap': '0x0', 'gasTipCap': '0x0', 'data': '0x3fb5c1cb0000000000000000000000000000000000000000000000000000000000000000', 'dataAbiValues': {'methodSignature': 'setNumber(uint256)', 'inputValues': ['0']}, 'AccessList': None, 'SkipAccountChecks': False}, 'blockNumberDelay': 30636, 'blockTimestampDelay': 98207}]: index: 0
Parsing fail on [{'call': {'from': '0x0000000000000000000000000000000000010000', 'to': '0xa647ff3c36cfab592509e13860ab8c4f28781a66', 'nonce': 0, 'value': '0x0', 'gasLimit': 12500000, 'gasPrice': '0x1', 'gasFeeCap': '0x0', 'gasTipCap': '0x0', 'data': '0x3fb5c1cb0000000000000000000000000000000000000000000000000000000000000000', 'dataAbiValues': {'methodSignature': 'setNumber(uint256)', 'inputValues': ['0']}, 'AccessList': None, 'SkipAccountChecks': False}, 'blockNumberDelay': 4563, 'blockTimestampDelay': 45901}]: index: 1
Parsing fail on [{'call': {'from': '0x0000000000000000000000000000000000030000', 'to': '0xa647ff3c36cfab592509e13860ab8c4f28781a66', 'nonce': 1, 'value': '0x0', 'gasLimit': 12500000, 'gasPrice': '0x1', 'gasFeeCap': '0x0', 'gasTipCap': '0x0', 'data': '0x3fb5c1cb0000000000000000000000000000000000000000000000000000000000000000', 'dataAbiValues': {'methodSignature': 'setNumber(uint256)', 'inputValues': ['0']}, 'AccessList': None, 'SkipAccountChecks': False}, 'blockNumberDelay': 40210, 'blockTimestampDelay': 360624}]: index: 2
Parsing fail on [{'call': {'from': '0x0000000000000000000000000000000000020000', 'to': '0xa647ff3c36cfab592509e13860ab8c4f28781a66', 'nonce': 0, 'value': '0x0', 'gasLimit': 12500000, 'gasPrice': '0x1', 'gasFeeCap': '0x0', 'gasTipCap': '0x0', 'data': '0x3fb5c1cb0000000000000000000000000000000000000000000000000000000000000000', 'dataAbiValues': {'methodSignature': 'setNumber(uint256)', 'inputValues': ['0']}, 'AccessList': None, 'SkipAccountChecks': False}, 'blockNumberDelay': 23883, 'blockTimestampDelay': 157607}]: index: 3
Parsing fail on [{'call': {'from': '0x0000000000000000000000000000000000020000', 'to': '0xa647ff3c36cfab592509e13860ab8c4f28781a66', 'nonce': 0, 'value': '0x0', 'gasLimit': 12500000, 'gasPrice': '0x1', 'gasFeeCap': '0x0', 'gasTipCap': '0x0', 'data': '0x3fb5c1cb0000000000000000000000000000000000000000000000000000000000000000', 'dataAbiValues': {'methodSignature': 'setNumber(uint256)', 'inputValues': ['0']}, 'AccessList': None, 'SkipAccountChecks': False}, 'blockNumberDelay': 1, 'blockTimestampDelay': 56038}]: index: 4
Parsing fail on [{'call': {'from': '0x0000000000000000000000000000000000010000', 'to': '0xa647ff3c36cfab592509e13860ab8c4f28781a66', 'nonce': 0, 'value': '0x0', 'gasLimit': 12500000, 'gasPrice': '0x1', 'gasFeeCap': '0x0', 'gasTipCap': '0x0', 'data': '0x3fb5c1cb0000000000000000000000000000000000000000000000000000000000000000', 'dataAbiValues': {'methodSignature': 'setNumber(uint256)', 'inputValues': ['0']}, 'AccessList': None, 'SkipAccountChecks': False}, 'blockNumberDelay': 0, 'blockTimestampDelay': 267796}]: index: 5
Parsing fail on [{'call': {'from': '0x0000000000000000000000000000000000030000', 'to': '0xa647ff3c36cfab592509e13860ab8c4f28781a66', 'nonce': 1, 'value': '0x0', 'gasLimit': 12500000, 'gasPrice': '0x1', 'gasFeeCap': '0x0', 'gasTipCap': '0x0', 'data': '0x3fb5c1cb0000000000000000000000000000000000000000000000000000000000000000', 'dataAbiValues': {'methodSignature': 'setNumber(uint256)', 'inputValues': ['0']}, 'AccessList': None, 'SkipAccountChecks': False}, 'blockNumberDelay': 0, 'blockTimestampDelay': 2}]: index: 6
Parsing fail on [{'call': {'from': '0x0000000000000000000000000000000000010000', 'to': '0xa647ff3c36cfab592509e13860ab8c4f28781a66', 'nonce': 0, 'value': '0x0', 'gasLimit': 12500000, 'gasPrice': '0x1', 'gasFeeCap': '0x0', 'gasTipCap': '0x0', 'data': '0x3fb5c1cb0000000000000000000000000000000000000000000000000000000000000000', 'dataAbiValues': {'methodSignature': 'setNumber(uint256)', 'inputValues': ['0']}, 'AccessList': None, 'SkipAccountChecks': False}, 'blockNumberDelay': 0, 'blockTimestampDelay': 415822}]: index: 7
Parsing fail on [{'call': {'from': '0x0000000000000000000000000000000000030000', 'to': '0xa647ff3c36cfab592509e13860ab8c4f28781a66', 'nonce': 1, 'value': '0x0', 'gasLimit': 12500000, 'gasPrice': '0x1', 'gasFeeCap': '0x0', 'gasTipCap': '0x0', 'data': '0x3fb5c1cb0000000000000000000000000000000000000000000000000000000000000000', 'dataAbiValues': {'methodSignature': 'setNumber(uint256)', 'inputValues': ['0']}, 'AccessList': None, 'SkipAccountChecks': False}, 'blockNumberDelay': 8467, 'blockTimestampDelay': 15389}]: index: 8
Parsing fail on [{'call': {'from': '0x0000000000000000000000000000000000010000', 'to': '0xa647ff3c36cfab592509e13860ab8c4f28781a66', 'nonce': 0, 'value': '0x0', 'gasLimit': 12500000, 'gasPrice': '0x1', 'gasFeeCap': '0x0', 'gasTipCap': '0x0', 'data': '0x3fb5c1cb0000000000000000000000000000000000000000000000000000000000000000', 'dataAbiValues': {'methodSignature': 'setNumber(uint256)', 'inputValues': ['0']}, 'AccessList': None, 'SkipAccountChecks': False}, 'blockNumberDelay': 18801, 'blockTimestampDelay': 413907}]: index: 9
Generated a test file in ./test/FuzzCounter_Medusa_Test.t.sol
Done!

[tuturu-tech]

Hey @nican0r, thanks for opening an issue!

The parsing failure you're seeing is due to Medusa 0.1.3 deprecating one of the fields from the corpus call sequences, so the tool tries to parse a field that doesn't exist. I've included a fix in this branch, you can install the branch by cloning it and running python3 -m pip install . in the repo root.

We're working on refactoring the Medusa call sequence parsing and will include a fix for this along with a couple of other issues in the next release. For Echidna it should work as expected, but the tool will only create unit tests for failing properties. Is the property failing for Echidna? The echidna corpus included in the repro repo doesn't seem to include failing sequences

[nican0r]

Hey @tuturu-tech, appreciate the quick response. Will try out the fix for medusa.

You were right about Echidna not failing, it was failing without any calls being made since the default value of number in the contract is 0

I hadn't been paying attention to the message in the Echidna output that was saying this. After changing the default value of number it breaks the invariant and the fuzz-utils tool does generate the correct Foundry test 👍

This makes me wonder if Medusa and Echidna treat checking properties after a contract is deployed differently since the Medusa run ignored the initial state and started trying call sequences whereas the Echidna run failed on the initial state. Is this described in the documentation somewhere?