Test some jump scenarios + baseline; log resource utilization

[codygunton]

🗒️ Description

Investigate zkVM cycle costs for various JUMP-heavy scenarios. Results from the branch in a Reth fork we have for counting these:

[crates/zkvm-benchmarks/host/src/main.rs:57:9] &reports = [
    WorkloadMetrics {
        name: "tests/zkevm/test_worst_compute.py::test_jumpdest_push0_pop_jump[fork_Cancun-blockchain_test-gas_limit_36000000]-1",
        total_num_cycles: 251870152,
        region_cycles: {
            "read_input": 250976181,
            "validation": 890043,
            "verify-witness": 371061,
        },
    },
    WorkloadMetrics {
        name: "tests/zkevm/test_worst_compute.py::test_push0_pop_no_jump[fork_Cancun-blockchain_test-gas_limit_36000000]-1",
        total_num_cycles: 252769788,
        region_cycles: {
            "read_input": 250976210,
            "validation": 1789650,
            "verify-witness": 327848,
        },
    },
    WorkloadMetrics {
        name: "tests/zkevm/test_worst_compute.py::test_jumpdest_jump[fork_Cancun-blockchain_test-gas_limit_36000000]-1",
        total_num_cycles: 251931756,
        region_cycles: {
            "verify-witness": 432756,
            "read_input": 250976166,
            "validation": 951662,
        },
    },
    WorkloadMetrics {
        name: "tests/zkevm/test_worst_compute.py::test_jumpdest_only[fork_Cancun-blockchain_test-gas_limit_36000000]-1",
        total_num_cycles: 252548548,
        region_cycles: {
            "read_input": 250976204,
            "verify-witness": 500029,
            "validation": 1568416,
        },
    },
]

Note that the included log file shows

===== UTILIZATION SUMMARY =====

TEST PATTERN                                  BYTECODE                         GAS        SIZE  ITERATIONS
----------------------------------------------------------------------------------------------------
JUMPDEST only                      24576/24576 (100.00%)        24576/35979000 (  0.07%)     1       24576
JUMPDEST + JUMP                    24576/24576 (100.00%)       110592/35979000 (  0.31%)     2       12288
JUMPDEST + PUSH0 + POP + JUMP      24576/24576 (100.00%)        86016/35979000 (  0.24%)     4        6144
PUSH0 + POP                        24576/24576 (100.00%)        61440/35979000 (  0.17%)     2       12288

so, e.g., we are are 1428x short of the gas limit in the JUMPDEST only example. I think this means we can get to 2.24B cycles in a block full of JUMPDESTs?

🔗 Related Issues

✅ Checklist

• All: Set appropriate labels for the changes.
• All: Considered squashing commits to improve commit history.
• All: Added an entry to CHANGELOG.md.
• All: Considered updating the online docs in the ./docs/ directory.
• Tests: All converted JSON/YML tests from ethereum/tests have been added to converted-ethereum-tests.txt.
• Tests: A PR with removal of converted JSON/YML tests from ethereum/tests have been opened.
• Tests: Included the type and version of evm t8n tool used to locally execute test cases: e.g., ref with commit hash or geth 1.13.1-stable-3f40e65.
• Tests: Ran mkdocs serve locally and verified the auto-generated docs for new tests in the Test Case Reference are correctly formatted.

[jsign]

Left some thoughts!

[jsign]

I think I've never seen this kind of file generation in this repo, but maybe is just a temp debug one? We can ping Mario when opening for review to ask him.

[codygunton]

Yeah for sure I will get rid of this, was just a temporary kludge until I learned about the -s flag.

[jsign]

I think if iteration_gas_cost is zero, it would be better to raise an error, since doesn't sounds like something expected.

[jsign]

If the max is max_iters_by_size, then this isn't a worst-case attack, right?

Some napkin math: the max number of JUMPDEST in 24KiB is 241024 JUMPDESTs since each costs 1 gas. That means that execution used 241024 of gas, which is pretty low compared with the gas limit 36M.

We could call 36M/24K contracts this style to use all gas, around 1464 contracts. So the real worst case sounds like some setup that can do those 1464 CALL to contracts full of JUMPDESTs or similar.

I had to solve a similar problem in the bytecode attack. But it is a bit trickier.
If you're interested, this is the PR: #1521

We could think of generalizing that setup to use it here, too. We can try merging your PR, and I can help if it sounds too complicated for you.

[codygunton]

Right, yeah agree this is not a worst case, none of these are without solving the issues around the testing framework that you've mentioned in other channels. Will try to extend your setup to this setting and will reach out if I get stuck, thanks

[jsign]

Uhm, this isn't correct since the JUMPs are not built correctly so this will fail quite fast. See here to know how JUMP works.

You can also see the gist I shared yesterday. In that commented line, you can see how I make the jump destination of JUMP be the PC+x to target the JUMPDEST.

I think the first link I shared with you explains this correctly, but ping me via DM if you want to chat more about it or have any question!

[codygunton]

Right, I actually expected this to fail, but it didn't fail in the testing framework--I was able to generate the json fixture and execute in the Reth fork. Is that expected?

[jsign]

Yep, it's expected for "everything to work", mainly because it would be a block execution that has tx that fail. That's something that could be intentional from the tester side.

Usually, tests have some side effects that can be detected with the postState assertions, so the test can fail because of that. Unfortunately, in all these test cases we don't do that. We could do some kind of "final SSTORE" and then check it as a postState assertion, but that feels it will complicate stuff a bit in the attack design and use some extra gas for that.

Might still be worth it though -- I don't like having to manually debug tests to be sure they're doing what we want.

Another strategy is having a way for the testing framework to check that the transaction doing the attack failed because of "out of gas reason" and not any other reason. I think with that I would be happy enough. In your case, this would be detected since the tx failure isn't for that reason.

cc @marioevz is something like this possible?

[jsign]

What is the theory behind this setup? Do you think that mixing some PUSH0/POP with JUMPs can create some particular interesting case?

[codygunton]

No this can go away

[jsign]

This sounds fine to me. The reason I did it in my gist was to compare the cycles of these simple opcodes with the JUMP/JUMPDEST ones. Kind of as a baseline.

[kevaundray]

This seems like a new thing being added into the repo as a whole -- If so, I don't think we want it here as the intention is to have these tests look like every other test

[codygunton]

Right never intended to be permanent, just seeking feedback on the tests :)

[kevaundray]

At first, I think we want to have basic workloads, so a block filled with one particular opcode.

Once we have that baseline down, then it would be good to explore interesting usecases that take advantage of the particular configuration of zkEVMs and or respectively EVMs like the PR is doing.