feat(invariant): support fuzz with random msg.value

[QiuhaoLi]

Motivation

Currently, the foundry's invariant fuzz testing doesn't support txs with value > 0, which makes it unable to find sequences in some situations. For example, the Pay contract below will only set the hacked variable if the calls are C() --> B() --> A() with 2, msg.value>0.11 ether, 0, but foundry can't generate such calls.

contract Pay {
    uint256 private counter;
    bool public hacked; // CBA with 2,msg.value>0.11,0

    function A(uint8 x) external {
        if (counter == 2 && x == 0) hacked = true; else counter = 0;
    }
    function B() external payable {
        if (counter == 1 && msg.value > 0.11 ether) counter++; else counter = 0;
    }
    function C(uint8 x) external {
        if (counter == 0 && x == 2) counter++;
    }
}

contract PayTest is Test {
    Pay public pay;

    function setUp() public {
        pay = new Pay();
    }
    /// forge-config: default.invariant.runs = 10000
    function invariant_Pay() view external {
        assertEq(pay.hacked(), false);
    }
}

Solution

When generating a tx, we set the msg.value as a random number (uint96) if the target function is payable. After applying this strategy, foundry can find the sequence quickly:

qiuhao@pc:~/tmp$ ./forge test
[⠊] Compiling...
[⠑] Compiling 1 files with Solc 0.8.26
[⠘] Solc 0.8.26 finished in 564.77ms
Compiler run successful!

Ran 1 test for test/Counter.t.sol:PayTest
[FAIL. Reason: invariant_Pay replay failure]
        [Sequence]
                sender=0x0000000000000000000000000000000000000190 addr=[test/Counter.t.sol:Pay]0x5615dEB798BB3E4dFa0139dFa1b3D433Cc23b72f calldata=C(uint8) args=[2]
                sender=0x0000000000000000000000000000000000000851 addr=[test/Counter.t.sol:Pay]0x5615dEB798BB3E4dFa0139dFa1b3D433Cc23b72f calldata=B() args=[] value=[79228162514264337593543950333]
                sender=0x0000000000000000000000000000000000000103 addr=[test/Counter.t.sol:Pay]0x5615dEB798BB3E4dFa0139dFa1b3D433Cc23b72f calldata=A(uint8) args=[0]
 invariant_Pay() (runs: 1, calls: 1, reverts: 1)
Suite result: FAILED. 0 passed; 1 failed; 0 skipped; finished in 3.95ms (1.31ms CPU time)

[QiuhaoLi]

CC @grandizzy @mds1

[grandizzy]

thanks for your PR! Generally this is recommended #8449 but will consider making it automatically (I think we want this only if fuzzed function is payable)

[QiuhaoLi]

thanks for your PR! Generally this is recommended #8449 but will consider making it automatically (I think we want this only if fuzzed function is payable)

Thanks for the info, handlers can help! If people don't use handlers, It would be nice if foundry could do that automatically. Yeah, we will use random values only for payable target functions.

[QiuhaoLi]

Hi @DaniPopes @mattsse , could you help review this PR?

[grandizzy]

@QiuhaoLi I am looking into integrate this approach but I am not sure we should inflate sender's balance with the fuzzed value (nor restore the balance after the call) as it could result in false positives like in forked tests / transfers, etc. Instead we could bound it in (0, current sender balance) but even so for long running campaigns the balance could get spent quickly and then the fuzzed calls will be performed with call value U256::ZERO

@0xalpharush any thoughts re how to deal with this scenario? thank you!

[0xalpharush]

Are the senders always the same pool of "test" addresses? If the tx.sender is the pranked value here and not the original, it may cause issues. But if it's from the pool, I think it should be fine.

[grandizzy]

makes sense, can keep a track of those addresses

[grandizzy]

thank you @QiuhaoLi will add with #13177 and keep credits