feat: E2E gas profiling native and erc20 #123
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,76 +1,78 @@ | ||
| AddOwnerAddressTest:testEmitsAddOwner() (gas: 95829) | ||
| AddOwnerAddressTest:testIncreasesOwnerIndex() (gas: 93811) | ||
| AddOwnerAddressTest:testRevertsIfAlreadyOwner() (gas: 97900) | ||
| AddOwnerAddressTest:testRevertsIfCalledByNonOwner() (gas: 13315) | ||
| AddOwnerAddressTest:testSetsIsOwner() (gas: 94340) | ||
| AddOwnerAddressTest:testSetsOwnerAtIndex() (gas: 97345) | ||
| AddOwnerPublicKeyTest:testEmitsAddOwner() (gas: 119224) | ||
| AddOwnerPublicKeyTest:testFuzzIsOwnerPublicKey(bytes32,bytes32) (runs: 260, μ: 120409, ~: 120409) | ||
| AddOwnerPublicKeyTest:testRevertsIfAlreadyOwner() (gas: 121567) | ||
| AddOwnerPublicKeyTest:testRevertsIfCalledByNonOwner() (gas: 13468) | ||
| AddOwnerPublicKeyTest:testSetsIsOwner() (gas: 117612) | ||
| AddOwnerPublicKeyTest:testSetsOwnerAtIndex() (gas: 120856) | ||
| CoinbaseSmartWallet1271InputGeneratorTest:testGetReplaySafeHashForDeployedAccount() (gas: 321780) | ||
| CoinbaseSmartWallet1271InputGeneratorTest:testGetReplaySafeHashForUndeployedAccount() (gas: 306564) | ||
| CoinbaseSmartWalletFactoryTest:testDeployDeterministicPassValues() (gas: 278647) | ||
| CoinbaseSmartWalletFactoryTest:test_CreateAccount_ReturnsPredeterminedAddress_WhenAccountAlreadyExists() (gas: 302620) | ||
| CoinbaseSmartWalletFactoryTest:test_RevertsIfLength32ButLargerThanAddress() (gas: 312891) | ||
| CoinbaseSmartWalletFactoryTest:test_constructor_revertsIfImplementationIsNotDeployed(address) (runs: 260, μ: 40036, ~: 40046) | ||
| CoinbaseSmartWalletFactoryTest:test_constructor_setsImplementation(address) (runs: 260, μ: 548238, ~: 548238) | ||
| CoinbaseSmartWalletFactoryTest:test_createAccountDeploysToPredeterminedAddress() (gas: 281841) | ||
| CoinbaseSmartWalletFactoryTest:test_createAccountSetsOwnersCorrectly() (gas: 293368) | ||
| CoinbaseSmartWalletFactoryTest:test_createAccount_emitsAccountCreatedEvent(uint256) (runs: 260, μ: 284216, ~: 284216) | ||
| CoinbaseSmartWalletFactoryTest:test_exitIfAccountIsAlreadyInitialized() (gas: 282240) | ||
| CoinbaseSmartWalletFactoryTest:test_implementation_returnsExpectedAddress() (gas: 7992) | ||
| CoinbaseSmartWalletFactoryTest:test_initCodeHash() (gas: 8307) | ||
| CoinbaseSmartWalletFactoryTest:test_revertsIfNoOwners() (gas: 30019) | ||
| ERC1271Test:test_returnsExpectedDomainHashWhenProxy() (gas: 23618) | ||
| ERC1271Test:test_static() (gas: 5038825) | ||
| EndToEndTest:test_transfer_erc20() (gas: 462176) | ||
| EndToEndTest:test_transfer_native() (gas: 355253) | ||
| MultiOwnableInitializeTest:testRevertsIfLength32ButLargerThanAddress() (gas: 83336) | ||
| MultiOwnableInitializeTest:testRevertsIfLength32NotAddress() (gas: 83255) | ||
| MultiOwnableInitializeTest:testRevertsIfLengthNot32Or64() (gas: 105695) | ||
| RemoveLastOwnerTest:test_emitsRemoveOwner() (gas: 52621) | ||
| RemoveLastOwnerTest:test_removesOwner() (gas: 52383) | ||
| RemoveLastOwnerTest:test_removesOwnerAtIndex() (gas: 53221) | ||
| RemoveLastOwnerTest:test_revert_whenCalledByNonOwner(address) (runs: 260, μ: 20652, ~: 20652) | ||
| RemoveLastOwnerTest:test_revert_whenNoOwnerAtIndex() (gas: 50014) | ||
| RemoveLastOwnerTest:test_revert_whenWrongOwnerAtIndex() (gas: 37450) | ||
| RemoveLastOwnerTest:test_reverts_whenNotLastOwner() (gas: 127056) | ||
| RemoveOwnerAtIndexTest:test_emitsRemoveOwner() (gas: 57775) | ||
| RemoveOwnerAtIndexTest:test_removesOwner() (gas: 57349) | ||
| RemoveOwnerAtIndexTest:test_removesOwnerAtIndex() (gas: 57914) | ||
| RemoveOwnerAtIndexTest:test_revert_whenCalledByNonOwner(address) (runs: 260, μ: 20674, ~: 20674) | ||
| RemoveOwnerAtIndexTest:test_revert_whenNoOwnerAtIndex() (gas: 35018) | ||
| RemoveOwnerAtIndexTest:test_revert_whenWrongOwnerAtIndex() (gas: 40657) | ||
| RemoveOwnerAtIndexTest:test_reverts_ifIsLastOwner() (gas: 8529792) | ||
| TestCanSkipChainIdValidation:test_approvedSelectorsReturnTrue() (gas: 20637) | ||
| TestCanSkipChainIdValidation:test_otherSelectorsReturnFalse() (gas: 13631) | ||
| TestExecuteWithoutChainIdValidation:testExecute() (gas: 587208) | ||
| TestExecuteWithoutChainIdValidation:testExecuteBatch() (gas: 1087434) | ||
| TestExecuteWithoutChainIdValidation:testExecuteBatch(uint256) (runs: 260, μ: 5622658, ~: 5415134) | ||
| TestExecuteWithoutChainIdValidation:test__codesize() (gas: 65702) | ||
| TestExecuteWithoutChainIdValidation:test_revertsWithReservedNonce() (gas: 87228) | ||
| TestExecuteWithoutChainIdValidation:test_reverts_whenCallerNotEntryPoint() (gas: 11407) | ||
| TestExecuteWithoutChainIdValidation:test_reverts_whenOneCallReverts() (gas: 475054) | ||
| TestExecuteWithoutChainIdValidation:test_reverts_whenOneSelectorNotApproved() (gas: 183931) | ||
| TestExecuteWithoutChainIdValidation:test_reverts_whenSelectorNotApproved() (gas: 108819) | ||
| TestExecuteWithoutChainIdValidation:test_succeeds_whenSelectorAllowed() (gas: 440590) | ||
| TestImplementation:testImplementation() (gas: 12896) | ||
| TestInitialize:testInitialize() (gas: 22939) | ||
| TestInitialize:test_cannotInitImplementation() (gas: 4597308) | ||
| TestIsValidSignature:testReturnsInvalidIfPasskeySigButWrongOwnerLength() (gas: 50912) | ||
| TestIsValidSignature:testRevertsIfEthereumSignatureButWrongOwnerLength() (gas: 29788) | ||
| TestIsValidSignature:testRevertsIfOwnerIsInvalidEthereumAddress() (gas: 29384) | ||
| TestIsValidSignature:testSmartWalletSigner() (gas: 4950306) | ||
| TestIsValidSignature:testValidateSignatureWithEOASigner() (gas: 30653) | ||
| TestIsValidSignature:testValidateSignatureWithEOASignerFailsWithWrongSigner() (gas: 27974) | ||
| TestIsValidSignature:testValidateSignatureWithPasskeySigner() (gas: 275434) | ||
| TestIsValidSignature:testValidateSignatureWithPasskeySignerFailsBadOwnerIndex() (gas: 46521) | ||
| TestIsValidSignature:testValidateSignatureWithPasskeySignerFailsWithWrongBadSignature() (gas: 270588) | ||
| TestUpgradeToAndCall:testUpgradeToAndCall() (gas: 26537) | ||
| TestValidateUserOp:test_reverts_whenReplayableNonceKeyInvalidForSelector() (gas: 17983) | ||
| TestValidateUserOp:test_reverts_whenSelectorInvalidForReplayableNonceKey() (gas: 19057) | ||
| TestValidateUserOp:test_reverts_whenUpgradeToImplementationWithNoCode(address) (runs: 260, μ: 30315, ~: 30315) | ||
| TestValidateUserOp:test_succeedsWithEOASigner() (gas: 687856) | ||
| TestValidateUserOp:test_succeedsWithPasskeySigner() (gas: 854583) | ||
| TestValidateUserOp:test_succeeds_whenUpgradeToImplementationWithCode() (gas: 4688457) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,26 @@ | ||
| { | ||
| "lib/account-abstraction": { | ||
| "rev": "abff2aca61a8f0934e533d0d352978055fddbd96" | ||
| }, | ||
| "lib/forge-std": { | ||
| "tag": { | ||
| "name": "v1.10.0", | ||
| "rev": "8bbcf6e3f8f62f419e5429a0bd89331c85c37824" | ||
| } | ||
| }, | ||
| "lib/openzeppelin-contracts": { | ||
| "rev": "5705e8208bc92cd82c7bcdfeac8dbc7377767d96" | ||
| }, | ||
| "lib/p256-verifier": { | ||
| "rev": "29475ae300ec95d98d5c7cc34c094846f0aa2dcd" | ||
| }, | ||
| "lib/safe-singleton-deployer-sol": { | ||
| "rev": "cf2b89c33fed536c4dd6fef2fb84f39053068868" | ||
| }, | ||
| "lib/solady": { | ||
| "rev": "c4c96607cb3aa3807b14c81ae2015bcba061f8fc" | ||
|
Contributor
There was a problem hiding this comment. if this is a commit, we actually want this to be |
||
| }, | ||
| "lib/webauthn-sol": { | ||
| "rev": "619f20ab0f074fef41066ee4ab24849a913263b2" | ||
| } | ||
| } | ||
Submodule forge-std
updated
55 files
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,6 @@ | ||
| { | ||
| "e2e_transfer_erc20_baseAccount": "159402", | ||
| "e2e_transfer_erc20_eoa": "50910", | ||
| "e2e_transfer_native_baseAccount": "134635", | ||
| "e2e_transfer_native_eoa": "17652" | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,111 @@ | ||
| // SPDX-License-Identifier: MIT | ||
| pragma solidity ^0.8.4; | ||
|
Contributor
There was a problem hiding this comment. make at parity with codebase so |
||
|
|
||
| import {MockERC20} from "../../lib/solady/test/utils/mocks/MockERC20.sol"; | ||
| import {CoinbaseSmartWallet} from "../../src/CoinbaseSmartWallet.sol"; | ||
| import {CoinbaseSmartWalletFactory} from "../../src/CoinbaseSmartWalletFactory.sol"; | ||
| import {SmartWalletTestBase} from "../CoinbaseSmartWallet/SmartWalletTestBase.sol"; | ||
|
|
||
| import {Static} from "../CoinbaseSmartWallet/Static.sol"; | ||
| import {MockTarget} from "../mocks/MockTarget.sol"; | ||
| import {UserOperation} from "account-abstraction/interfaces/UserOperation.sol"; | ||
| import {console2} from "forge-std/Test.sol"; | ||
|
|
||
| /// forge-config: default.isolate = true | ||
|
|
||
| contract EndToEndTest is SmartWalletTestBase { | ||
| address eoaUser = address(0xe0a); | ||
|
|
||
| MockERC20 usdc; | ||
| MockTarget target; | ||
| CoinbaseSmartWalletFactory factory; | ||
|
|
||
| function setUp() public override { | ||
| vm.etch(0x5FF137D4b0FDCD49DcA30c7CF57E578a026d2789, Static.ENTRY_POINT_BYTES); | ||
|
|
||
| CoinbaseSmartWallet implementation = new CoinbaseSmartWallet(); | ||
| factory = new CoinbaseSmartWalletFactory(address(implementation)); | ||
|
|
||
| signerPrivateKey = 0xa11ce; | ||
| signer = vm.addr(signerPrivateKey); | ||
| owners.push(abi.encode(signer)); | ||
| account = factory.createAccount(owners, 0); | ||
|
|
||
| vm.deal(address(account), 100 ether); | ||
| vm.deal(eoaUser, 100 ether); | ||
|
|
||
| usdc = new MockERC20("USD Coin", "USDC", 6); | ||
| usdc.mint(address(account), 10000e6); | ||
| usdc.mint(eoaUser, 10000e6); | ||
|
|
||
| target = new MockTarget(); | ||
| } | ||
|
|
||
| // Native ETH transfer comparison between ERC-4337 and EOA | ||
| function test_transfer_native() public { | ||
|
Contributor
There was a problem hiding this comment. would benefit from comments to annotate the code sections here for easier skimming |
||
| // Dust recipient to control for gas increase for first non-zero balance | ||
| vm.deal(address(0x1234), 1 wei); | ||
| usdc.mint(eoaUser, 1 wei); | ||
|
|
||
| userOpCalldata = abi.encodeCall(CoinbaseSmartWallet.execute, (address(0x1234), 1 ether, "")); | ||
| UserOperation memory op = _getUserOpWithSignature(); | ||
|
|
||
| bytes memory handleOpsCalldata = abi.encodeCall(entryPoint.handleOps, (_makeOpsArray(op), payable(bundler))); | ||
| console2.log("test_transfer_native Base Account calldata size:", handleOpsCalldata.length); | ||
|
|
||
| vm.startSnapshotGas("e2e_transfer_native_baseAccount"); | ||
| _sendUserOperation(op); | ||
| uint256 gas4337 = vm.stopSnapshotGas(); | ||
| console2.log("test_transfer_native Base Account gas:", gas4337); | ||
|
|
||
| console2.log("test_transfer_native EOA calldata size:", uint256(0)); | ||
|
|
||
| vm.prank(eoaUser); | ||
| vm.startSnapshotGas("e2e_transfer_native_eoa"); | ||
| payable(address(0x1234)).transfer(1 ether); | ||
| uint256 gasEOA = vm.stopSnapshotGas(); | ||
| console2.log("test_transfer_native EOA gas:", gasEOA); | ||
| console2.log("Gas overhead (4337 Base Account / EOA):", (gas4337 * 100) / gasEOA, "%"); | ||
| } | ||
|
|
||
| // ERC20 transfer comparison between ERC-4337 and EOA | ||
| function test_transfer_erc20() public { | ||
|
Contributor
There was a problem hiding this comment. in the https://github.com/coinbase/solidity-style-guide there's a preference for tests only testing one thing. not necessarily a blocker here especially since these are just benchmarks, but if I were to write this I would split it out into the 4 explicit cases of native/erc20, scw/eoa for better org and readability. |
||
| // Dust recipient to control for gas increase for first non-zero balance | ||
| vm.deal(address(0x5678), 1 wei); | ||
| usdc.mint(eoaUser, 1 wei); | ||
|
|
||
| userOpCalldata = abi.encodeCall( | ||
| CoinbaseSmartWallet.execute, (address(usdc), 0, abi.encodeCall(usdc.transfer, (address(0x5678), 100e6))) | ||
| ); | ||
| UserOperation memory op = _getUserOpWithSignature(); | ||
|
|
||
| bytes memory handleOpsCalldata = abi.encodeCall(entryPoint.handleOps, (_makeOpsArray(op), payable(bundler))); | ||
| console2.log("test_transfer_erc20 Base Account calldata size:", handleOpsCalldata.length); | ||
|
|
||
| vm.startSnapshotGas("e2e_transfer_erc20_baseAccount"); | ||
| _sendUserOperation(op); | ||
| uint256 gas4337 = vm.stopSnapshotGas(); | ||
| console2.log("test_transfer_erc20 Base Account gas:", gas4337); | ||
|
|
||
| bytes memory eoaCalldata = abi.encodeCall(usdc.transfer, (address(0x5678), 100e6)); | ||
| console2.log("test_transfer_erc20 EOA calldata size:", eoaCalldata.length); | ||
|
|
||
| vm.prank(eoaUser); | ||
| vm.startSnapshotGas("e2e_transfer_erc20_eoa"); | ||
| usdc.transfer(address(0x5678), 100e6); | ||
| uint256 gasEOA = vm.stopSnapshotGas(); | ||
| console2.log("test_transfer_erc20 EOA gas:", gasEOA); | ||
| console2.log("Gas overhead (4337 Base Account / EOA):", (gas4337 * 100) / gasEOA, "%"); | ||
| } | ||
|
|
||
| function _makeOpsArray(UserOperation memory op) internal pure returns (UserOperation[] memory) { | ||
| UserOperation[] memory ops = new UserOperation[](1); | ||
| ops[0] = op; | ||
| return ops; | ||
| } | ||
|
|
||
| function _sign(UserOperation memory userOp) internal view override returns (bytes memory signature) { | ||
|
Contributor
There was a problem hiding this comment. this should already be available on the SmartWalletTestBase parent contract so I don't think we need to re-write here? |
||
| bytes32 toSign = entryPoint.getUserOpHash(userOp); | ||
| (uint8 v, bytes32 r, bytes32 s) = vm.sign(signerPrivateKey, toSign); | ||
| signature = abi.encode(CoinbaseSmartWallet.SignatureWrapper(0, abi.encodePacked(r, s, v))); | ||
| } | ||
| } | ||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@amiecorso did we update .gas-snapshot when pushing v1.1? Feels weird for the diff to be so big unless its fuzzing variance?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
we did actually... See the file on the v1.1 commit here
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
differences could have to do with how things are compiled before snapshotting though... since control of that stuff is a mess