autoload() never resolves on some Diamond proxies

[yog4a]

When calling whatsabi.autoload() the Promise never resolves for certain contracts (notably Diamond proxy patterns).
The function halts after the "loadDiamondFacets" phase and no result is returned nor error thrown.

--- Logs

For 0x3914bdb4130306f80f5d8ee099c180442d19680d
[whatsabi][onProgress] getCode { address: '0x3914bdb4130306f80f5d8ee099c180442d19680d' }
[whatsabi][onProgress] loadDiamondFacets { address: '0x3914bdb4130306f80f5d8ee099c180442d19680d' }

--> Then nothing happened until node process crash.

--- Details
The issue happens when followProxies: true and false
The issue happens with any viem provider (and with timeout set)
The issue happens only for this contract
Tested on Node.js v20.15.1, @shazow/whatsabi@0.11.2.

// Imports
// ===========================================================

import { whatsabi, proxies } from '@shazow/whatsabi';
import { createPublicClient, http, chains } from 'viem';

const provider = createPublicClient({
    chain: chains.mainnet,
    transport: http()
});

// Functions
// ===========================================================

export const resolveProxy = async (address: `0x${string}`, bytecode: `0x${string}`): Promise<ResolveProxyOutput> => {
    try {
        const cachedCodeProvider = whatsabi.providers.WithCachedCode(provider, {
            [address]: bytecode,
        });

        const result = await whatsabi.autoload(address, {
            provider: cachedCodeProvider,
            abiLoader: false, // Skip ABI loaders
            signatureLookup: false, // Skip looking up selector signatures
            followProxies: true, // Follow proxy
        });

        const selectors = result.abi.map((func: any) => func.selector);
        const proxies = result.proxies.map((proxy: any) => ({ name: proxy.name, address: proxy.resolvedAddress }));

        return { selectors, proxies };
        
    } catch (error) {
        const _selectors = whatsabi.selectorsFromBytecode(bytecode) as `0x${string}`[]; 
        return { selectors: _selectors, proxies: [] };
    }
}

[shazow]

Need to investigate, but I've run into scenarios where the diamond proxy has so much history that it takes many hundreds or even thousands of RPC requests to traverse them all, takes forever.

Not sure what the best approach is in this case, maybe add a limit by default so that at least it will fail early?

[yog4a]

I get what you’re saying about some diamond proxies having huge facet histories, but what’s strange in my case is that once the call hangs, nothing in userland can abort it.

Even if I wrap whatsabi.autoload() in a Promise.race with a 10s timeout, the timeout never fires.
The whole Node process just keeps waiting — no rejection, no error, and even Viem’s { timeout } transport option doesn’t trigger.

It looks like the main event loop is completely blocked while waiting for one of the RPC calls deep inside facets() to finish.
At that point the only way out is to run the call in a separate worker and kill it manually.

So maybe it’s not just “takes a long time,” but rather that the resolver can get stuck somewhere

[shazow]

Any ideas of how to fix this? Want to give it a try? :)

[yog4a]

I already tried digging into it a bit today but couldn’t find the exact cause yet — looks like something deeper than just the number of facets.

I’ll keep investigating and share what I find

[shazow]

Appreciated, thank you! Wonder if it happens in other runtimes like deno

[yog4a]

Reproductible example:

// Imports
// ===========================================================

import { whatsabi, proxies } from '../src/index.js';
import { createPublicClient, http } from 'viem';
import { mainnet } from 'viem/chains';

const provider = createPublicClient({
    chain: mainnet,
    transport: http('https://ethereum-rpc.publicnode.com')
});

// Functions
// ===========================================================

(async () => {
    const address = '0x3914bdb4130306f80f5d8ee099c180442d19680d';
    try {
        const cachedCodeProvider = whatsabi.providers.WithCachedCode(provider, {
            [address]: "0x608060405236600a57005b600080356001600160e01b03191681527fc8fcad8db84d3cc18b4c41d551ea0ee66dd599cde068d998e57d5e09332c131c6020819052604090912054819060601c8060805760405162461bcd60e51b8152602060048201526002602482015261533160f01b604482015260640160405180910390fd5b3660008037600080366000845af43d6000803e808015609e573d6000f35b3d6000fdfea2646970667358221220e1c422b895f3ed3b2083b6e98ae66ddcf5050f45159879385136dda8e89e146864736f6c63430008170033",
        });

        const result = await whatsabi.autoload(address, {
            provider: cachedCodeProvider,
            abiLoader: false,
            signatureLookup: false,
            followProxies: false,
            onProgress: (phase, ...args) =>
              console.log(`[worker][onProgress] ${phase}`, ...args),
            onError: (phase, error) => {
              console.error(`[worker][onError] ${phase}`, error.message);
              return false;
            },
        });

        const selectors = result.abi.map((func: any) => func.selector);
        const proxies = result.proxies.map((proxy: any) => ({ name: proxy.name, address: proxy.resolvedAddress }));

    } catch (error) {
        console.error(error);
    }
})();

Problem come from autoload.js

  // TODO: Respect config.followProxies, see https://github.com/shazow/whatsabi/issues/132
  onProgress("loadDiamondFacets", { address });
  const diamondProxy = result.proxies[0] as DiamondProxyResolver;
  const f = await diamondProxy.facets(provider, address);
  Object.assign(facets, f);

Inside then this in proxies.js, facets is 54 array length

  const selectorWidth = 4;
  const facetSelectors : Record<string, string[]> = {};
  const slot = addSlotOffset(storageStart, 1); // facetToSelector in 2nd slot
  for (const f of facets) {
      const facet = addressFromPadded(f);
      const facetSelectorsSlot = joinSlot([facet, slot]);
      const selectors = await readArray(provider, address, facetSelectorsSlot, selectorWidth);
      facetSelectors[addressWithChecksum(facet)] = selectors.map(s => "0x" + s);

      if (--limit === 0) break;
  }

then in slots.js,

getStorage return: 0xcdffacc652ef6b2cadfca15e7a0ed62701ffc9a77ad467059c02504a66a0c394
this lead to num = 9.317587240826328e+76

Then this lead to loop for a long long time....

export async function readArray(provider: StorageProvider, address: string, pos: number|string, width: number=32): Promise<string[]> {
    // Based on https://gist.github.com/banteg/0cee21909f7c1baedfa6c3d96ffe94f2
    const num = Number(await provider.getStorageAt(address, pos));
    const start = keccak256(pos.toString(16)); // toString(16) does the right thing on strings too (no-op) (:
    const itemsPerWord = Math.floor(32 / width);

    const promises : Promise<string>[] = [];
    for (let i=0; i<num; i++) {
        const itemSlot = addSlotOffset(start, Math.floor(i / itemsPerWord));
        promises.push(provider.getStorageAt(address, itemSlot));
    }

    const words : string[] = await Promise.all(promises);

    return words.map((wordHex: string, i: number) => {
        // TODO: Extract multiple words if they fit in a slot?
        const itemOffset = 2 + 64 - (i % itemsPerWord + 1) * width * 2; // 0x + 2 hex per byte
        return wordHex.slice(itemOffset, itemOffset + width * 2);
    });
}

[yog4a]

I don't know why, I've found this one, same proxy, but this don't execute (the facets variable return empty array, compared to the other that return 54 elements)

0x222222fD79264BBE280b4986F6FEfBC3524d0137

[yog4a]

Extracted from the contract

    bytes32 constant DIAMOND_STORAGE_POSITION = keccak256("diamond.standard.diamond.storage");

    struct DiamondStorage {
        // maps function selectors to the facets that execute the functions.
        // and maps the selectors to their position in the selectorSlots array.
        // func selector => address facet, selector position
        mapping(bytes4 => bytes32) facets;
        // array of slots of function selectors.
        // each slot holds 8 function selectors.
        mapping(uint256 => bytes32) selectorSlots;
        // The number of function selectors in selectorSlots
        uint16 selectorCount;
        // Used to query if a contract implements an interface.
        // Used to implement ERC-165.
        mapping(bytes4 => bool) supportedInterfaces;
        // owner of the contract
        address contractOwner;
    }

What's the structure looks like normally

        // 1. Read the DiamondStorage.facets array
        //
        // struct DiamondStorage {
        //   mapping(bytes4 => SelectorToFacet) selectorToFacet;
        //   mapping(address => FacetToSelectors) facetToSelectors;
        //   address[] facets;
        //   bool isFrozen;
        // }
``

[yog4a]

@shazow I don’t have time to make all the changes right now, but I’m pretty sure the issue comes from DIAMOND_STORAGE_1 integration, the structure for DIAMOND_STORAGE is

  // struct DiamondStorage {
  //   mapping(bytes4 => SelectorToFacet) selectorToFacet;
  //   mapping(address => FacetToSelectors) facetToSelectors;
  //   address[] facets;
  //   bool isFrozen;
  // }

But for DIAMOND_STORAGE_1

  struct DiamondStorage {
      mapping(bytes4 => bytes32) facets;
      mapping(uint256 => bytes32) selectorSlots;
      uint16 selectorCount;
      mapping(bytes4 => bool) supportedInterfaces;
      address contractOwner;
  }

https://github.com/mudgen/diamond-2-hardhat/blob/bf67b736ad5fa3366551f599e204784856fb3069/contracts/libraries/LibDiamond.sol#L15

[yog4a]

/// author Matter Labs
/// notice The helper library for managing the EIP-2535 diamond proxy.

Diamond
https://github.com/mudgen/diamond-1-hardhat/blob/main/contracts/Diamond.sol

/// Authors: Nick Mudge nick@perfectabstractions.com (https://twitter.com/mudgen),
/// EIP-2535 Diamonds: https://eips.ethereum.org/EIPS/eip-2535

Diamond 1
https://github.com/mudgen/diamond-1-hardhat/blob/main/contracts/libraries/LibDiamond.sol

Diamond 2
https://github.com/mudgen/diamond-2-hardhat/blob/bf67b736ad5fa3366551f599e204784856fb3069/contracts/libraries/LibDiamond.sol#L15

Diamond 3
https://github.com/mudgen/diamond-3-hardhat/blob/main/contracts/libraries/LibDiamond.sol

[shazow]

@shazow I don’t have time to make all the changes right now, but I’m pretty sure the issue comes from DIAMOND_STORAGE_1 integration, the structure for DIAMOND_STORAGE is

Ah that sounds plausible, hrmm. I won't have time for a bit to dive into this for a little while, but please share if you have ideas for how to better distinguish the correct storage slot.

[shazow]

Ah I see, there's multiple implementations with different layouts that use the same origin storage slot. Hmmmm.

[yog4a]

I haven’t really worked on proxies, so I’m not too sure how all of this actually works.
But one possibility could be to test it —
based on the structure first, to see which ones fit best.

Because in my case, from the test, the values are so large and the array is full of zeros that it might mean it’s wrong.