feat(balance_overrides): use debug_traceCall to find slots #3937

kaze-cow · 2025-12-01T04:06:39Z

Description

Implement a faster and more reliable approach for detecting solidity balance override storage addresses.

Rather than bulk storage override/scanning many storage slots for a match, a new detector strategy DirectSlot is added to detect the storage slot through a single debug_traceCall on the ERC20 balanceOf function, and reading the resolved SLOAD slots. This approach is very similar to one followed by Foundry for their deal cheatcode.

This method has advantages:

Assuming that there is only one SLOAD in a balanceOf call (most cases), only a single debug_traceCall is required for a single token/address pair. In the case of more than one SLOAD, an additional call is required for each slot to validate the correct storage, which is still inexpensive.
Basically any token that stores a user's balance in a single uint256 will be supported by this method.
Since this strategy can be reliably used anywhere, we can probably deprecate/remove the other strategies

Since slot detection using this method is only able to find the storage slot for a specific account in question, the interface needed to be updated in a couple places to reflect this. This also means that there is a potential performance disadvantage with this method since balance overrides for different addresses cannot be detected due to not computing via Solidity mapping. This could be mitigated by only supporting balance overrides on the spardose contract (followed by a transfer call) to the needed account as required.

This method requires a node that supports the debug_traceCall API. I believe this is the case for our infra, but please double check 🙏.

For now I left the original SolidityMapping detector strategy as a backup in case DirectSlot fails. However, there should theoretically be no cases where SolidityMapping would detect when DirectSlot does not, so it would be worth removing and relying solely on DirectSlot.

I added a E2E test and contract to validate that the storage slot detection is working as expected.

Changes

add DirectSlot detector
update caching interface to cache by the pair (token address, overridden balance address)
add E2E test to verify the detector works in practice

How to test

Run just test-e2e local_node_trace_based_balance_detection

Implement a better approach for detecting. Rather than scanning solidity storage slots for a mapping, a new detector strategy `DirectSlot` is added to detect the storage slot through a single `debug_traceCall` on the ERC20 `balanceOf` function, and reading the resolved `SLOAD` slots. This approach is very similar to [one followed by Foundry for their `deal` cheatcode](https://github.com/foundry-rs/foundry/blob/9b13b811849e73654fae046986b8730df8a0d64d/crates/anvil/src/eth/api.rs#L2259). This method has a few advantages: * Assuming that there is only one `SLOAD` in a `balanceOf` call (most cases), only a single `debug_traceCall` is required for a single token/address pair. In the case of more than one SLOAD, an additional call is required for each slot to validate the correct storage, which is still inexpensive. * Basically any token that stores a user's balance in a single `uint256` will be supported by this method. Since slot detection using this method is only able to find the storage slot for a specific account in question, the interface needed to be updated in a couple places to reflect this. This also means that there is a potential performance disadvantage with this method since balance overrides for different addresses cannot be detected due to not computing via Solidity mapping. This could be mitigated by only supporting balance overrides on the spardose contract (followed by a `transfer` call) to the needed account as required. This method requires a node that supports the `debug_traceCall` API. I believe this is the case for our infra, but please double check 🙏. For now I left the original `SolidityMapping` detector strategy as a backup in case `DirectSlot` fails. However, there should theoretically be no cases where `SolidityMapping` would detect when `DirectSlot` does not, so it would be worth removing and relying solely on `DirectSlot`. I added a E2E test and contract to validate that the storage slot detection is working as expected.

kaze-cow · 2025-12-01T04:09:30Z

crates/ethrpc/src/extensions.rs

+
+/// Custom deserializer for stack values that handles variable-length hex
+/// strings (side note: I don't know why this has to be so complicated...)
+fn deserialize_hex_stack<'de, D>(deserializer: D) -> Result<Vec<primitive_types::H256>, D::Error>


is there a better way to do this? ran into some issues with doing somethat should be extremely basic (parsing hex string to a B256)

While this does not answer your specific question I think this would go away if we use eth_createAccessList instead of debug_traceCall to figure out the relevant storage slots. That should be a lot simpler and not rely on non-standard modules to be enabled in the node client.

🤦‍♂️ had no idea this API existed. Apparently AI didn't know about it either 😆 Looks super standardized too.

ok looking at this again, I actually think the current API call is still worth considering. In general access lists give us what we need, but there is not requirement that the accessed storage slots be returned in a particular order. So, for example on etherscan https://etherscan.io/tx/0x46ef9c9771286847cb95dc4003b9f6ffae2e0d8630b6edcd19a7872875758484#accesslist , they are returned in alphabetical order, and now the scanning can be very expensive.

with trace we can tell which SLOAD(s) were last, and in a balanceOf call, it seems reasonable to expect the last SLOADs will be the slot we are looking for, right?

I'm still somewhat worried about debug_ calls being less supported by external node providers. Let's go with trying debug_traceCall first and if that fails fall back to eth_createAccessList.
Also you actually don't have to implement debug_traceCall yourself since alloy provides that out of the box: https://docs.rs/alloy/latest/alloy/providers/ext/trait.DebugApi.html#tymethod.debug_trace_call.

Reminder on this comment since the PR currently re-implements a bunch of stuff that alloy has out of the box (unless there is a reason for this which I don't see).

crates/shared/src/price_estimation/trade_verifier/balance_overrides/detector.rs

MartinquaXD · 2025-12-01T09:57:04Z

crates/shared/src/price_estimation/trade_verifier/balance_overrides/mod.rs

+    /// debug_traceCall. This is similar to Foundry's `deal` approach where
+    /// we trace a balanceOf call to find which storage slot is accessed for
+    /// a given account.
+    DirectSlot { slot: H256 },


I think this needs to also store a target_contract. IIRC tokens like EURe on gnosis chain store the balances in a completely different contract.

yea i probably should have mentioned this. for right now in order to keep the implementation simpler I decided not to update storage slots in other contracts since its somewhat rare and I thought it would amke things a lot more complicated with the trace early on. But its not like it would be hard to add looking at it now.

so I added target contract as you suggest, but in order to properly encode the overrides object in the case that its DirectSlot, both SolidityMapping and SoladyMapping also need to have the target_contract as well. This turned out to be a somewhat big refactor after all. LMK if we should keep it or revert. a5f70bc

crates/shared/src/price_estimation/trade_verifier/balance_overrides/detector.rs

MartinquaXD · 2025-12-01T10:05:28Z

crates/shared/src/price_estimation/trade_verifier/balance_overrides/detector.rs

+                "detected balance slot via trace (single SLOAD) for token {:?}: slot {:?}",
+                token, slot
+            );
+            return Ok(Strategy::DirectSlot { slot });


IMO in a follow up PR we should get rid of the old brute force approach and use the same idea here.
That way we'll only have 1 strategy that is able to detect the balances storage slot for superior caching or just return the specific slot if brute forcing a couple of storage slots doesn't work.

…wprotocol/services into feat/tracing-balance-override

crates/contracts/solidity/Makefile

crates/shared/src/price_estimation/trade_verifier/balance_overrides/detector.rs

MartinquaXD · 2025-12-08T09:06:16Z

crates/shared/src/price_estimation/trade_verifier/balance_overrides/detector.rs

+        // changing unnecessary storage slots could negatively affect the execution (ex.
+        // overriding an upgradable proxy contract target)
+        for (i, slot) in storage_slots.iter().rev().enumerate() {
+            if let Ok(strategy) = self.verify_slot_is_balance(token, holder, *slot).await {


To properly marry the "brute force" approach with this one this is where we should start brute forcing.
So if we find the n where hash(user_address || n) == balance_slot we know where the balance mapping lives.

great idea! I suppose that gives us a major reason to continue using the existing solidity mapping method

biggest fear I have is you could still end up in a situation where you do the mapping filter and still end up with matching slots that just so happen to be related to something separate from the actual balance mapping (ex. if balanceOf checks an epoch before getting the actual balance, which is not actually stored in a solidity mapping. something convoluted.). ofc these are extreme cases. but yea, the performance benefit is high, worthwhile change.

fyi I implemetned this in such a way that it doesn't reuse the code in SolidityMapping. I did this to hopefully make a possible later refactor of this code a bit easier.

While using the brute force idea to pick more likely storage slots might be a neat optimization to save a few eth_calls the real benefit only comes from also being able to return the SolidityMapping strategy from detect_with_trace(). Only if we store the SolidityMapping strategy can we avoid all future eth_calls to compute the correct storage slot for new owner addresses.

So what I'm suggesting is:

use debug_traceCall / eth_createAccessList to figure out the relevant storage slots

optionally use the brute forcing stuff to find more likely candidates

find the EXACT storage slot for the given owner

use the brute force logic to see if we can find the storage slot that holds the balances mapping
if that works return SolidityMapping with the found mapping slot
else return DirectSlot

Step 4 should also check the Solady mapping but IIRC that only requires 1 check.

got it, yea that shouldn't be too hard. my brain is pretty much on the track now that these strategies were going out the door so I was going all in on the DirectSlot stuff + only using spardose (rather than filling any address, which delivers inconsistent performance)

…wprotocol/services into feat/tracing-balance-override

for now not intermingling this with the SolidityMapper strategy because we are likely going to refactor this out in the future according to discussions.

…wprotocol/services into feat/tracing-balance-override

crates/shared/src/price_estimation/trade_verifier/balance_overrides/detector.rs

…rides/detector.rs Co-authored-by: Jan [Yann] <[email protected]>

MartinquaXD · 2025-12-09T13:16:29Z

crates/shared/src/price_estimation/trade_verifier/balance_overrides/detector.rs

+    storage_slots.sort_by(|a, b| {
+        heuristic_slots
+            .contains(&b.1)
+            .cmp(&heuristic_slots.contains(&a.1))
+    });


This can be replaced with sort_by_key() for less ambiguity if the comparison was implemented correctly or even sort_by_cached_key() if frequent hashmap look ups are actually too slow.

so much easier! done.

MartinquaXD · 2025-12-09T13:18:49Z

crates/ethrpc/src/extensions.rs

+
+/// Custom deserializer for stack values that handles variable-length hex
+/// strings (side note: I don't know why this has to be so complicated...)
+fn deserialize_hex_stack<'de, D>(deserializer: D) -> Result<Vec<primitive_types::H256>, D::Error>


Reminder on this comment since the PR currently re-implements a bunch of stuff that alloy has out of the box (unless there is a reason for this which I don't see).

fafk · 2025-12-09T13:35:44Z

crates/shared/src/price_estimation/trade_verifier/balance_overrides/detector.rs

+            let mut map_slot = U256::from(start_slot);
+            for _ in 0..self.probing_depth {
+                strategies.push(Strategy::SolidityMapping {
+                    target_contract: H160::default(),


I am a bit out of my element here, but the default strategies above pass target_contract and this one defaults to zero. Is that right?

yea... something about this seems wrong. probably DirectSlot is taking over and so SolidityMapping strategy issues are no longer showing.

kaze-cow requested a review from a team as a code owner December 1, 2025 04:06

kaze-cow requested a review from MartinquaXD December 1, 2025 04:06

kaze-cow commented Dec 1, 2025

View reviewed changes

crates/shared/src/price_estimation/trade_verifier/balance_overrides/detector.rs Show resolved Hide resolved

remove unnecessary code and warnings

aec31dc

MartinquaXD reviewed Dec 1, 2025

View reviewed changes

kaze-cow added 5 commits December 2, 2025 14:05

changes corresponding to comments

a5f70bc

Merge branch 'main' into feat/tracing-balance-override

ef22a7d

fix log, comment

56d7410

just fmt

9896321

Merge branch 'feat/tracing-balance-override' of https://github.com/co…

04d07a5

…wprotocol/services into feat/tracing-balance-override

kaze-cow requested a review from MartinquaXD December 3, 2025 06:16

Merge branch 'main' into feat/tracing-balance-override

a2421ba

kaze-cow requested a review from a team December 4, 2025 09:52

Merge branch 'main' into feat/tracing-balance-override

a4919a4

MartinquaXD reviewed Dec 8, 2025

View reviewed changes

kaze-cow added 8 commits December 9, 2025 00:20

undo unintended makefile change

4979f0b

Merge branch 'feat/tracing-balance-override' of https://github.com/co…

eded10d

…wprotocol/services into feat/tracing-balance-override

add even smarter heuristic detection

2f83e73

for now not intermingling this with the SolidityMapper strategy because we are likely going to refactor this out in the future according to discussions.

Merge branch 'main' into feat/tracing-balance-override

6da8fd3

fix lint

02c314b

Merge branch 'feat/tracing-balance-override' of https://github.com/co…

e9ad7c3

…wprotocol/services into feat/tracing-balance-override

fix clippy

0118199

just fmt

91031ea

kaze-cow requested a review from MartinquaXD December 9, 2025 08:09

kaze-cow enabled auto-merge December 9, 2025 08:09

fafk reviewed Dec 9, 2025

View reviewed changes

crates/shared/src/price_estimation/trade_verifier/balance_overrides/detector.rs Outdated Show resolved Hide resolved

kaze-cow disabled auto-merge December 9, 2025 11:00

Update crates/shared/src/price_estimation/trade_verifier/balance_over…

c5afa35

…rides/detector.rs Co-authored-by: Jan [Yann] <[email protected]>

beef up the test

7ebd82b

MartinquaXD reviewed Dec 9, 2025

View reviewed changes

fafk reviewed Dec 9, 2025

View reviewed changes

kaze-cow added 3 commits December 9, 2025 22:56

working much better

0901089

final cleanups

6ef2797

proper makefile change

69a7f61

feat(balance_overrides): use debug_traceCall to find slots #3937

Are you sure you want to change the base?

feat(balance_overrides): use debug_traceCall to find slots #3937

Uh oh!

Conversation

kaze-cow commented Dec 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Changes

How to test

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

kaze-cow Dec 2, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

kaze-cow commented Dec 1, 2025 •

edited

Loading

kaze-cow Dec 2, 2025 •

edited

Loading