eth_simulateV1 fails for op-nmc

[mralj]

Description
Calling eth_simulateV1 on op-NMC node fails.

Steps to Reproduce

curl https://url.dev \
        -X POST \
        -H "Content-Type: application/json" \
        --data '{"jsonrpc":"2.0","method":"eth_simulateV1","params":[{"blockStateCalls":[{"blockOverrides":{"baseFeePerGas":"0x9"},"stateOverrides":{"0xc000000000000000000000000000000000000000":{"balance":"0x4a817c420"}},"calls":[{"from":"0xd8dA6BF26964aF9D7eEd9e03E53415D37aA96045","to":"0x014d023e954bAae7F21E56ed8a5d81b12902684D","maxFeePerGas":"0xf","value":"0x1"}]}],"validation":true,"traceTransfers":true},"latest"],"id":1}'

Actual behavior

Request errors with:

{"jsonrpc":"2.0","error":{"code":0,"message":"System.InvalidCastException: Unable to cast object of type 'Nethermind.Core.TxReceipt' to type 'Nethermind.Optimism.OptimismTxReceipt'.\n   at Nethermind.Optimism.OptimismReceiptMessageDecoder.GetLength(TxReceipt item, RlpBehaviors rlpBehaviors) in /src/Nethermind/Nethermind.Optimism/OptimismReceiptMessageDecoder.cs:line 200\n   at Nethermind.State.Proofs.ReceiptTrie`1.CalculateRoot(IReceiptSpec receiptSpec, TReceipt[] txReceipts, IRlpStreamDecoder`1 decoder) in /src/Nethermind/Nethermind.State/Proofs/ReceiptTrie.cs:line 62\n   at Nethermind.Consensus.Processing.BlockProcessor.ProcessBlock(Block block, IBlockTracer blockTracer, ProcessingOptions options) in /src/Nethermind/Nethermind.Consensus/Processing/BlockProcessor.cs:line 319\n   at Nethermind.Consensus.Processing.BlockProcessor.ProcessOne(Block suggestedBlock, ProcessingOptions options, IBlockTracer blockTracer) in /src/Nethermind/Nethermind.Consensus/Processing/BlockProcessor.cs:line 286\n   at Nethermind.Consensus.Processing.BlockProcessor.Process(Hash256 newBranchStateRoot, List`1 suggestedBlocks, ProcessingOptions options, IBlockTracer blockTracer) in /src/Nethermind/Nethermind.Consensus/Processing/BlockProcessor.cs:line 148\n   at Nethermind.Facade.Simulate.SimulateBridgeHelper.TrySimulate(BlockHeader parent, SimulatePayload`1 payload, SimulateBlockTracer simulateOutputTracer, IBlockTracer tracer, SimulateReadOnlyBlocksProcessingEnv env, String& error) in /src/Nethermind/Nethermind.Facade/Simulate/SimulateBridgeHelper.cs:line 76\n   at Nethermind.Facade.Simulate.SimulateBridgeHelper.TrySimulate(BlockHeader parent, SimulatePayload`1 payload, SimulateBlockTracer simulateOutputTracer, IBlockTracer tracer, String& error) in /src/Nethermind/Nethermind.Facade/Simulate/SimulateBridgeHelper.cs:line 65\n   at Nethermind.Facade.BlockchainBridge.Simulate(BlockHeader header, SimulatePayload`1 payload, CancellationToken cancellationToken) in /src/Nethermind/Nethermind.Facade/BlockchainBridge.cs:line 176","data":[]},"id":1}⏎

Expected behavior

curl https://docs-demo.base-mainnet.quiknode.pro/ \
        -X POST \
        -H "Content-Type: application/json" \
        --data '{"jsonrpc":"2.0","method":"eth_simulateV1","params":[{"blockStateCalls":[{"blockOverrides":{"baseFeePerGas":"0x9"},"stateOverrides":{"0xc000000000000000000000000000000000000000":{"balance":"0x4a817c420"}},"calls":[{"from":"0xd8dA6BF26964aF9D7eEd9e03E53415D37aA96045","to":"0x014d023e954bAae7F21E56ed8a5d81b12902684D","maxFeePerGas":"0xf","value":"0x1"}]}],"validation":true,"traceTransfers":true},"latest"],"id":1}'

Returns

{"jsonrpc":"2.0","id":1,"result":[{"baseFeePerGas":"0x9","blobGasUsed":"0x0","calls":[{"returnData":"0x","logs":[{"address":"0xeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee","topics":["0xddf252ad1be2c89b69c2b068fc378daa952ba7f163c4a11628f55a4df523b3ef","0x000000000000000000000000d8da6bf26964af9d7eed9e03e53415d37aa96045","0x000000000000000000000000014d023e954baae7f21e56ed8a5d81b12902684d"],"data":"0x0000000000000000000000000000000000000000000000000000000000000001","blockNumber":"0x1a84935","transactionHash":"0x60b1244032eaded5fbb0856294c22d61c3ea45f27f7857eb42eb70f1c0a5f5c6","transactionIndex":"0x0","blockHash":"0x830bcc820289f0323496b34151a50195f5288a4fa7edcaf559c1ef253ad15778","logIndex":"0x0","removed":false}],"gasUsed":"0x5208","status":"0x1"}],"difficulty":"0x0","excessBlobGas":"0x0","extraData":"0x","gasLimit":"0x6ea0500","gasUsed":"0x5208","hash":"0x830bcc820289f0323496b34151a50195f5288a4fa7edcaf559c1ef253ad15778","logsBloom":"0x00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000","miner":"0x4200000000000000000000000000000000000011","mixHash":"0x0000000000000000000000000000000000000000000000000000000000000000","nonce":"0x0000000000000000","number":"0x1a84935","parentBeaconBlockRoot":"0x0000000000000000000000000000000000000000000000000000000000000000","parentHash":"0xfcbb695622570b065d78993abb2a2714e1b67d1dc086d99e1e1cefbfff499c0e","receiptsRoot":"0xf78dfb743fbd92ade140711c8bbc542b5e307f0ab7984eff35d751969fe57efa","sha3Uncles":"0x1dcc4de8dec75d7aab85b567b6ccd41ad312451b948a7413f0a142fd40d49347","size":"0x272","stateRoot":"0xd51eb4900668403903bcd79ab5ca44139c7e8d460c0805ee9ffbd80f41a37742","timestamp":"0x67daef57","transactions":["0x60b1244032eaded5fbb0856294c22d61c3ea45f27f7857eb42eb70f1c0a5f5c6"],"transactionsRoot":"0x345da63fc97ee942af41863b79b59d2bf1d19f407d684741546414f5c977082f","uncles":[],"withdrawals":[],"withdrawalsRoot":"0x56e81f171bcc55a6ff8345e692c0f86e5b48e01b996cadc001622fb5e363b421"}]}

Using NMC v1.31.4

[rjnrohit]

Is there a way to reproduce this error?

[rjnrohit]

Can we try it on the latest master branch ?

[mralj]

Is there a way to reproduce this error?

I'll send you node URL to Slack :)

Can we try it on the latest master branch ?

I asked about this and will get back to you

[mralj]

There is one more error I've noticed and this happens if I set Validation: true:

Request:

{"method":"eth_simulateV1","params":[{"blockStateCalls":[{"blockOverrides":{},"stateOverrides":null,"calls":[{"from":"0x3643CE6e2336aAf8f14346f19c6b21ef703d8AF1","to":"0x653FA433176796dEAbe624E8B04F07Aa446eE44a","maxFeePerGas":"0xb2d05e00","value":"0x86bd009fa28832"},{"from":"0xF7b463D51A4d6242D61046c3d3cC0ACbF039Bb0f","to":"0x3643CE6e2336aAf8f14346f19c6b21ef703d8AF1","maxFeePerGas":
"0xb2d05e00","value":"0x21c5febac09d94"}]}],"validation":true,"traceTransfers":true},"latest"],"id":0,"jsonrpc":"2.0"}

Resp:

&{2.0 <nil> -32602: Nethermind.Blockchain.InvalidBlockException: Transaction 0xe675c0e5fe8a432ccd034533b394b4cecc84691b4bc6a3f6bf222d0c6cbf41d8 at index 1 failed with error Block gas limit
 exceeded
   at Nethermind.Consensus.Processing.BlockProcessor.BlockValidationTransactionsExecutor.ProcessTransaction(BlockExecutionContext& blkCtx, Transaction currentTx, Int32 index, BlockReceiptsTrace
r receiptsTracer, ProcessingOptions processingOptions) in /src/Nethermind/Nethermind.Consensus/Processing/BlockProcessor.BlockValidationTransactionsExecutor.cs:line 52
   at Nethermind.Facade.Simulate.SimulateBlockValidationTransactionsExecutor.ProcessTransaction(BlockExecutionContext& blkCtx, Transaction currentTx, Int32 index, BlockReceiptsTracer receiptsTr
acer, ProcessingOptions processingOptions) in /src/Nethermind/Nethermind.Facade/Simulate/SimulateReadOnlyBlocksProcessingEnv.cs:line 42
   at Nethermind.Consensus.Processing.BlockProcessor.BlockValidationTransactionsExecutor.ProcessTransactions(Block block, ProcessingOptions processingOptions, BlockReceiptsTracer receiptsTracer
, IReleaseSpec spec) in /src/Nethermind/Nethermind.Consensus/Processing/BlockProcessor.BlockValidationTransactionsExecutor.cs:line 42
   at Nethermind.Consensus.Processing.BlockProcessor.ProcessBlock(Block block, IBlockTracer blockTracer, ProcessingOptions options) in /src/Nethermind/Nethermind.Consensus/Processing/BlockProce
ssor.cs:line 319
   at Nethermind.Consensus.Processing.BlockProcessor.ProcessOne(Block suggestedBlock, ProcessingOptions options, IBlockTracer blockTracer) in /src/Nethermind/Nethermind.Consensus/Processing/Blo
ckProcessor.cs:line 286
   at Nethermind.Consensus.Processing.BlockProcessor.Process(Hash256 newBranchStateRoot, List`1 suggestedBlocks, ProcessingOptions options, IBlockTracer blockTracer) in /src/Nethermind/Nethermi
nd.Consensus/Processing/BlockProcessor.cs:line 148
   at Nethermind.Facade.Simulate.SimulateBridgeHelper.TrySimulate(BlockHeader parent, SimulatePayload`1 payload, SimulateBlockTracer simulateOutputTracer, IBlockTracer tracer, SimulateReadOnlyB
locksProcessingEnv env, String& error) in /src/Nethermind/Nethermind.Facade/Simulate/SimulateBridgeHelper.cs:line 76
   at Nethermind.Facade.Simulate.SimulateBridgeHelper.TrySimulate(BlockHeader parent, SimulatePayload`1 payload, SimulateBlockTracer simulateOutputTracer, IBlockTracer tracer, String& error) in
 /src/Nethermind/Nethermind.Facade/Simulate/SimulateBridgeHelper.cs:line 65
   at Nethermind.Facade.BlockchainBridge.Simulate(BlockHeader header, SimulatePayload`1 payload, CancellationToken cancellationToken) in /src/Nethermind/Nethermind.Facade/BlockchainBridge.cs:li
ne 176 0}

This doesn't happen on Quiknode (Geth/v1.101503.1-stable-fbc739c3/linux-amd64/go1.24.1) I've been using as backup till this is sorted out.
For the context I'm sending 2 transactions fro the simulation, both are simple transfers, and gasLimits are capped at 21k.
This happens even if I manually override Block gasLimit to smth. big, e.g. 1M

[emlautarom1]

I'm able to reproduce the issue on Nethermind v1.31.4 with base-sepolia as the config and the following request:

curl --request POST \
  --url http://localhost:8545/ \
  --header 'content-type: application/json' \
  --data '{"id": 123,"jsonrpc": "2.0","method": "eth_simulateV1","params": [{"blockStateCalls":[{"blockOverrides":{"baseFeePerGas":"0x9"},"stateOverrides":{"0xc000000000000000000000000000000000000000":{"balance":"0x4a817c420"}},"calls":[{"from":"0xd8dA6BF26964aF9D7eEd9e03E53415D37aA96045","to":"0x014d023e954bAae7F21E56ed8a5d81b12902684D","maxFeePerGas":"0xf","value":"0x1"}]}],"validation":true,"traceTransfers":true},"latest"]}'

If you can run Nethermind from source, it would be great if you could try the following branch: https://github.com/NethermindEth/nethermind/tree/fix/op-eth_simulate. It only contains the required fixes on top of release/v1.31.4.

[rjnrohit]

I ran this on my fully synced sepolia instance. I got the following response:

{
    "jsonrpc": "2.0",
    "result": [
        {
            "calls": [
                {
                    "status": "0x1",
                    "returnData": "0x",
                    "gasUsed": "0x5208",
                    "logs": [
                        {
                            "address": "0xeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee",
                            "topics": [
                                "0xddf252ad1be2c89b69c2b068fc378daa952ba7f163c4a11628f55a4df523b3ef",
                                "0xe8d4b3c704000000c0d1a60cd8da6bf26964af9d7eed9e03e53415d37aa96045",
                                "0xf87c8f1001000000d8d7f76e014d023e954baae7f21e56ed8a5d81b12902684d"
                            ],
                            "data": "0x0000000000000000000000000000000000000000000000000000000000000001",
                            "blockNumber": "0x79acba",
                            "transactionHash": "0xbc95f76ec516c1a98f4dbcee30f57f095212f17e22a18c6044772bade96e0414",
                            "transactionIndex": "0x1",
                            "blockHash": "0x2018f060503ac5f7edfe88987fe26f9812817e17bdd503843088b52e25f7c949",
                            "logIndex": "0x0",
                            "removed": false
                        }
                    ]
                }
            ],
            "difficulty": "0x0",
            "extraData": "0x",
            "gasLimit": "0x22550dd",
            "gasUsed": "0x5208",
            "hash": "0x68ac6159e2898d0e12fe9f527a4c902684fdddfbe84e71460c008fed18e07ef3",
            "logsBloom": "0x00000000000000000000000000000000001000000000000000000004000000000000000000000000000000000000000000000000000000040000000000000000000000000000000000000008000000000000000000040000000000000000000000000000000000000000000000000000000000000000000000000010000000000000000000000000000000000000000000000000000000000000000000000000000010000000000000000000000000000000000000000000000000000000840000000002000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000100000000000000002000",
            "miner": "0xe276bc378a527a8792b353cdca5b5e53263dfb9e",
            "mixHash": "0x0000000000000000000000000000000000000000000000000000000000000000",
            "nonce": "0x0000000000000000",
            "number": "0x79acba",
            "parentHash": "0x543ce2455d7c6a4b351397be9eca188fe46358b7e890118a28a5b4f7d2635d5e",
            "receiptsRoot": "0x9de67f87f745b09690d66c138d42774a527e56b2f4ccf82bf32cc2b068315340",
            "sha3Uncles": "0x1dcc4de8dec75d7aab85b567b6ccd41ad312451b948a7413f0a142fd40d49347",
            "size": "0x271",
            "stateRoot": "0xc562e42d8a23a1d556467fca47a7516af7c8401d9789f43a76a571735f8329cb",
            "totalDifficulty": "0x3c656d23029ab0",
            "timestamp": "0x67e1e060",
            "baseFeePerGas": "0x9",
            "transactions": [
                "0xbc95f76ec516c1a98f4dbcee30f57f095212f17e22a18c6044772bade96e0414"
            ],
            "transactionsRoot": "0x4c052af8310b71205d15f1f850f00cafa10b7f2b6b8972bdf7ef1f53ed7a242c",
            "uncles": [],
            "withdrawals": [],
            "withdrawalsRoot": "0x56e81f171bcc55a6ff8345e692c0f86e5b48e01b996cadc001622fb5e363b421",
            "blobGasUsed": "0x0",
            "excessBlobGas": "0x0",
            "parentBeaconBlockRoot": "0x0000000000000000000000000000000000000000000000000000000000000000",
            "requestsHash": "0xe3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855"
        }
    ],
    "id": 1
}

[mralj]

Hey @emlautarom1 @rjnrohit I confirm that the original issue has been resolved, but the other one still persists.

[rjnrohit]

Hey @emlautarom1 @rjnrohit I confirm that the original issue has been resolved, but the other one still persists.

I have tried that one as well:

I'm getting the following response:

{
    "jsonrpc": "2.0",
    "error": {
        "code": -38014,
        "message": "invalid transaction index: 1 at block number: 7977040, Reason: Transaction cost (88882843249002132) is higher than sender balance (0)",
        "data": []
    },
    "id": 0
}

[mralj]

I was getting this error, but not anymore 🤔
So for the time being @rjnrohit @emlautarom1 please disregard my previous comment , I'll test more and report if I see the issue again

[mralj]

Manget to repro the 2nd issue, but I think it unrelated to the op-nmc, so I've opened separate issue

[emlautarom1]

We currently have a PR that partially fixes this issue due to misconfiguration when dealing with OP chains (#8423). The PR is marked as draft since I want to add some tests to avoid regressions.

Now that Nethermind does not crash, we see that responses differ from op-geth:

• PR Merge eth_simulate_v1 to master #8153 is needed since it fixes an issue with the simulated block timestamp which is not part of any release yet (v1.31.6 being the latest).
• Nethermind still has a bug with the above listed PRs since the simulated block timestamp is twice the block time (ex. if parent's block time is 10 and block time is 2 it should simulate a block with timestamp 12, instead we're seeing 14 (10 + 2 + 2). Currently working on figuring out why this is the case.
• OP-geth has a bug where the timestamp is computed incorrectly since it uses Ethereum Mainnet's block time instead of the one configured per chain (in case of Base Sepolia it's 2 seconds). A ticket was submitted to the corresponding repo: eth_simulateV1 produces incorrect timestamp ethereum-optimism/op-geth#563

This issue affects all chains.