investigate hevm memory usage / performance

[MrChico]

recently, a couple of memory intensive tests started failing on ci with error message:

Return50000_d0g1v0_Istanbul hevm: internal error: Unable to commit 1048576 bytes of memory

although the error message reports this as a ghc bug, people suggest this is rarely the case. Granted, these are pretty extreme test cases, but would still be interesting to see if there are any easy ways to tackle the memory footprint

[d-xo]

I ran the Return5000 tests with profiling enabled (cabal new-run hevm -- compliance --tests ~/code/github.com/ethereum/tests/ --timeout 999999999 --match Return50000 +RTS -p) and got the following trace: https://ipfs.io/ipfs/QmeLEYpgYahNMxaxtDz74z5FPzrS1w358Y2evjUkXoKNcs/hevm.prof

It looks like a lot (~40%) of the allocation is happening in <!> from Options.Applicative.Internal

[MrChico]

hmmm, I wonder how to interpret that... hevm compliance is kinda weird because it invokes hevm bc-test through a bash script
we might get better results if the logic of the run-blockchain tests was just done in hevm directly. It would also be pretty straightforward to run the tests in parallell then as well

[d-xo]

Maybe it’s better to just profile hevm bc-test directly?

[MrChico]

probably. Unless the culprit is actually the launchscript oddity

[d-xo]

results of running cabal new-run hevm -- bc-test --file ~/code/github.com/ethereum/tests/BlockchainTests/GeneralStateTests/stQuadraticComplexityTest/Return50000.json --timeout 99999999 +RTS -p:

https://ipfs.io/ipfs/QmcMTQ6jEHk65xnGFHVvxxjCgtvwSxPaFDmRatn1D8hkCP/hevm.prof

This time we're getting 40% of the alloc from byteStringSliceWithDefaultZeroes in Concrete.hs

[MrChico]

wowow

[MrChico]

maybe https://stackoverflow.com/questions/18350177/why-creating-and-disposing-temporal-bytestrings-eats-up-my-memory-in-haskell

[MrChico]

something should be strict that isn't here

[d-xo]

So I made byteStringSliceWithDefaultZeroes as strict as I know how (see here), and the memory usage didn't seem to change anything (profiling output here).

Not really sure how to proceed.

[MrChico]

some useful benchmarking scripts can be found at https://github.com/MrChico/evmbench/

[rainbreak]

Last year I considered changing the memory stuff to work with LazyByteString, which could help here (?), but it seemed like a lot of work.