-
Notifications
You must be signed in to change notification settings - Fork 50
feat(utils): batch balance queries via multicall #1557
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change | ||
---|---|---|---|---|
@@ -0,0 +1,230 @@ | ||||
import { ERC20__factory } from "@across-protocol/contracts"; | ||||
import { providers } from "ethers"; | ||||
import { callViaMulticall3 } from "../../api/_utils"; | ||||
import { getProvider } from "./providers"; | ||||
import { getMulticall3, ZERO_ADDRESS } from "./sdk"; | ||||
import { useQuery } from "@tanstack/react-query"; | ||||
|
||||
export type MultiCallResult = { | ||||
blockNumber: providers.BlockTag; | ||||
// { [walletAddress]: { [tokenAddress]: balanceString } } | ||||
balances: Record<string, Record<string, string>>; | ||||
}; | ||||
|
||||
/** | ||||
* Fetches the balances for an array of addresses on a particular chain, for a particular erc20 token | ||||
* @param chainId The blockchain Id to query against | ||||
* @param addresses An array of valid Web3 wallet addresses | ||||
* @param tokenAddress The valid ERC20 token address on the given `chainId` or ZERO_ADDRESS for native balances | ||||
* @param blockTag Block to query from, defaults to latest block | ||||
* @returns a Promise that resolves to an array of BigNumbers | ||||
*/ | ||||
export const getBatchBalanceViaMulticall3 = async ( | ||||
chainId: string | number, | ||||
addresses: string[], | ||||
tokenAddresses: string[], | ||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. It might make sense to use symbols instead of addresses here? |
||||
blockTag: providers.BlockTag = "latest" | ||||
): Promise<MultiCallResult> => { | ||||
const chainIdAsInt = Number(chainId); | ||||
const provider = getProvider(chainIdAsInt); | ||||
|
||||
const multicall3 = getMulticall3(chainIdAsInt, provider); | ||||
|
||||
if (!multicall3) { | ||||
throw new Error("No Multicall3 deployed on this chain"); | ||||
} | ||||
|
||||
let calls: Parameters<typeof callViaMulticall3>[1] = []; | ||||
|
||||
for (const tokenAddress of tokenAddresses) { | ||||
if (tokenAddress === ZERO_ADDRESS) { | ||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. We'll need to add logic somewhere to handle ETH since we use WETH address internally not the zero address There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Wdyt of using the symbols instead of addresses in the context of our FE? Based on the chain id we can infer which symbols are native |
||||
// For native currency | ||||
calls.push( | ||||
...addresses.map((address) => ({ | ||||
contract: multicall3, | ||||
functionName: "getEthBalance", | ||||
args: [address], | ||||
})) | ||||
); | ||||
} else { | ||||
// For ERC20 tokens | ||||
const erc20Contract = ERC20__factory.connect(tokenAddress, provider); | ||||
calls.push( | ||||
...addresses.map((address) => ({ | ||||
contract: erc20Contract, | ||||
functionName: "balanceOf", | ||||
args: [address], | ||||
})) | ||||
); | ||||
} | ||||
} | ||||
|
||||
const inputs = calls.map(({ contract, functionName, args }) => ({ | ||||
target: contract.address, | ||||
callData: contract.interface.encodeFunctionData(functionName, args), | ||||
})); | ||||
Comment on lines
+50
to
+65
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. @gsteenkamp89 I think this could be reordered to squash it down. It should be possible to instantiate a single generic ERC20 contract (not connected to any specific address or provider) and then generate all the balanceOf() calldata. Then that same calldata can be reused across all of the ERC20s we want to query. So I think it could be something like:
If it's done that way then that should save instantiating a bunch of ephemeral ERC20 contract instances. It does require some special handling for the snowflake "getEthBalance()" call though. |
||||
|
||||
const [blockNumber, results] = await multicall3.callStatic.aggregate(inputs, { | ||||
blockTag, | ||||
}); | ||||
|
||||
const decodedResults = results.map((result, i) => | ||||
calls[i].contract.interface.decodeFunctionResult( | ||||
calls[i].functionName, | ||||
result | ||||
) | ||||
); | ||||
|
||||
let balances: Record<string, Record<string, string>> = {}; | ||||
|
||||
let resultIndex = 0; | ||||
for (const tokenAddress of tokenAddresses) { | ||||
addresses.forEach((address) => { | ||||
if (!balances[address]) { | ||||
balances[address] = {}; | ||||
} | ||||
balances[address][tokenAddress] = decodedResults[resultIndex].toString(); | ||||
resultIndex++; | ||||
}); | ||||
} | ||||
|
||||
return { | ||||
blockNumber: blockNumber.toNumber(), | ||||
balances, | ||||
}; | ||||
}; | ||||
Comment on lines
+22
to
+95
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. copied from here (since we cannot share code between api and frontend easily) Line 1393 in 9c29867
|
||||
|
||||
// Define the type representing an individual balance request. | ||||
interface BalanceRequest { | ||||
token: string; | ||||
address: string; | ||||
resolve: (balance: string) => void; | ||||
reject: (error: any) => void; | ||||
} | ||||
|
||||
// Define the type for a batch request that aggregates multiple balance requests. | ||||
interface BatchRequest { | ||||
chainId: number; | ||||
blockTag: providers.BlockTag; | ||||
aggregatedTokens: Set<string>; | ||||
aggregatedAddresses: Set<string>; | ||||
requests: BalanceRequest[]; | ||||
timer?: NodeJS.Timeout; | ||||
} | ||||
|
||||
const DEFAULT_BATCH_INTERVAL = 100; | ||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. this interval adds an implicit delay. is this too aggressive? is 10ms better? |
||||
|
||||
export type BalanceBatcherFetchFn = typeof getBatchBalanceViaMulticall3; | ||||
|
||||
export class BalanceBatcher { | ||||
private batchQueue: Record<number, BatchRequest> = {}; | ||||
|
||||
constructor( | ||||
readonly fetcher: BalanceBatcherFetchFn, | ||||
readonly batchInterval = DEFAULT_BATCH_INTERVAL | ||||
) {} | ||||
|
||||
/** | ||||
* Queues an individual balance request. All requests for the same chain | ||||
* (and assumed same blockTag) within batchInterval (ms) are batched together. | ||||
* | ||||
* @param chainId The chain ID. | ||||
* @param token The ERC20 token address or zero address for native balance | ||||
* @param address The address we want to fetch the balance for | ||||
* @param blockTag The block tag (defaults to "latest"). | ||||
* @returns A Promise resolving to the token balance as a string. | ||||
*/ | ||||
public async queueBalanceCall( | ||||
chainId: number, | ||||
token: string, | ||||
address: string, | ||||
blockTag: providers.BlockTag = "latest" | ||||
): Promise<string> { | ||||
return new Promise((resolve, reject) => { | ||||
// Create active batch for this chain | ||||
if (!this.batchQueue[chainId]) { | ||||
this.batchQueue[chainId] = { | ||||
chainId, | ||||
blockTag, | ||||
aggregatedTokens: new Set([token]), | ||||
aggregatedAddresses: new Set([address]), | ||||
requests: [{ token, address, resolve, reject }], | ||||
}; | ||||
|
||||
this.batchQueue[chainId].timer = setTimeout(async () => { | ||||
const currentBatch = this.batchQueue[chainId]; | ||||
delete this.batchQueue[chainId]; | ||||
|
||||
const tokensArray = Array.from(currentBatch.aggregatedTokens); | ||||
const addressesArray = Array.from(currentBatch.aggregatedAddresses); | ||||
|
||||
try { | ||||
const result: MultiCallResult = await this.fetcher( | ||||
chainId, | ||||
addressesArray, | ||||
tokensArray, | ||||
currentBatch.blockTag | ||||
); | ||||
|
||||
currentBatch.requests.forEach(({ token, address, resolve }) => { | ||||
const balance = result.balances[address]?.[token] || "0"; | ||||
resolve(balance); | ||||
}); | ||||
} catch (error) { | ||||
currentBatch.requests.forEach(({ reject }) => reject(error)); | ||||
} | ||||
}, this.batchInterval); | ||||
} else { | ||||
// batch already exists for this interval, just add another | ||||
const existingBatch = this.batchQueue[chainId]; | ||||
existingBatch.aggregatedTokens.add(token); | ||||
existingBatch.aggregatedAddresses.add(address); | ||||
existingBatch.requests.push({ token, address, resolve, reject }); | ||||
} | ||||
}); | ||||
} | ||||
} | ||||
|
||||
// Do something with this singleton | ||||
export const balanceBatcher = new BalanceBatcher(getBatchBalanceViaMulticall3); | ||||
|
||||
/** | ||||
* @param chainId The blockchain chain ID. | ||||
* @param tokenAddress The ERC20 token address. | ||||
* @param walletAddress The wallet address whose balance will be fetched. | ||||
* @param blockTag Optional blockTag (defaults to "latest"). | ||||
* @returns An object containing the balance (a string), loading state, and any error. | ||||
*/ | ||||
export function useBalance({ | ||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. might need to add ability to resolve |
||||
chainId, | ||||
token, | ||||
address, | ||||
blockTag, | ||||
options, | ||||
}: { | ||||
chainId: number; | ||||
token: string; | ||||
address: string; | ||||
blockTag?: providers.BlockTag; | ||||
options?: Partial< | ||||
Omit<Parameters<typeof useQuery>[0], "queryKey" | "queryFn"> | ||||
>; | ||||
}) { | ||||
return useQuery({ | ||||
queryKey: [ | ||||
"balance", | ||||
chainId, | ||||
token, | ||||
address, | ||||
blockTag ?? "latest", | ||||
] as const, | ||||
queryFn: () => | ||||
balanceBatcher.queueBalanceCall( | ||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This looks interesting. I am just wondering if we could reach the same goal by using optimized tanstack/query options 🤔 |
||||
chainId, | ||||
token, | ||||
address, | ||||
blockTag ?? "latest" | ||||
), | ||||
...options, | ||||
}); | ||||
} |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,111 @@ | ||
import { BalanceBatcher, BalanceBatcherFetchFn } from "../balances"; | ||
|
||
// mock multicall balance fetcher | ||
const fakeGetBatchBalance: BalanceBatcherFetchFn = async ( | ||
chainId, | ||
addresses, | ||
tokens, | ||
blockTag | ||
) => { | ||
const balances: Record<string, Record<string, string>> = {}; | ||
for (const addr of addresses) { | ||
balances[addr] = {}; | ||
for (const token of tokens) { | ||
balances[addr][token] = `balance-${addr}-${token}`; | ||
} | ||
} | ||
return { blockNumber: blockTag ?? "latest", balances }; | ||
}; | ||
|
||
describe("BalanceBatcher", () => { | ||
let batcher: BalanceBatcher; | ||
let fetchSpy: jest.MockedFunction<BalanceBatcherFetchFn>; | ||
|
||
beforeEach(() => { | ||
fetchSpy = jest.fn( | ||
fakeGetBatchBalance | ||
) as jest.MockedFunction<BalanceBatcherFetchFn>; | ||
|
||
batcher = new BalanceBatcher(fetchSpy); | ||
jest.useFakeTimers(); | ||
jest.clearAllTimers(); | ||
}); | ||
|
||
afterEach(() => { | ||
jest.useRealTimers(); | ||
}); | ||
|
||
it("should batch multiple requests into one multicall", async () => { | ||
const chainId = 1; | ||
const tokenA = "0xTokenA"; | ||
const tokenB = "0xTokenB"; | ||
const address1 = "0xAddress1"; | ||
const address2 = "0xAddress2"; | ||
|
||
const promise1 = batcher.queueBalanceCall( | ||
chainId, | ||
tokenA, | ||
address1, | ||
"latest" | ||
); | ||
const promise2 = batcher.queueBalanceCall( | ||
chainId, | ||
tokenB, | ||
address1, | ||
"latest" | ||
); | ||
const promise3 = batcher.queueBalanceCall( | ||
chainId, | ||
tokenA, | ||
address2, | ||
"latest" | ||
); | ||
const promise4 = batcher.queueBalanceCall( | ||
chainId, | ||
tokenB, | ||
address2, | ||
"latest" | ||
); | ||
|
||
jest.advanceTimersByTime(batcher.batchInterval); | ||
|
||
const [res1, res2, res3, res4] = await Promise.all([ | ||
promise1, | ||
promise2, | ||
promise3, | ||
promise4, | ||
]); | ||
|
||
expect(res1).toBe(`balance-${address1}-${tokenA}`); | ||
expect(res2).toBe(`balance-${address1}-${tokenB}`); | ||
expect(res3).toBe(`balance-${address2}-${tokenA}`); | ||
expect(res4).toBe(`balance-${address2}-${tokenB}`); | ||
|
||
// Verify we're only making one RPC request | ||
expect(fetchSpy).toHaveBeenCalledTimes(1); | ||
expect(fetchSpy).toHaveBeenCalledWith( | ||
chainId, | ||
[address1, address2], | ||
[tokenA, tokenB], | ||
"latest" | ||
); | ||
}); | ||
|
||
it("should reject all requests if multicall fails", async () => { | ||
// Force a failed RPC request | ||
const failingGetBatchBalance = async () => { | ||
throw new Error("Multicall failed"); | ||
}; | ||
batcher = new BalanceBatcher(failingGetBatchBalance); | ||
|
||
const chainId = 1; | ||
const token = "0xTokenA"; | ||
const address = "0xAddress1"; | ||
|
||
const promise = batcher.queueBalanceCall(chainId, token, address, "latest"); | ||
|
||
jest.advanceTimersByTime(batcher.batchInterval); | ||
|
||
await expect(promise).rejects.toThrow("Multicall failed"); | ||
}); | ||
}); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@gsteenkamp89 Do you think we'll be supplying > 1 address in practice? I'd thought it would typically only be address of the connected wallet, though if we need to make more queries than that's even better.
(i.e. are we maybe also querying relayer balances this way?)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
yeah that's exactly right, we're repurposing the logic originally written for relayer balances.