While working on the CoinMarketCap clone, I ran into a limitation when trying to display data for the top 100 coins on the homepage.
Fetching data for more than around 30 coins resulted in a failed RPC call due to response size limits:
HttpRequestError: HTTP request failed.
Status: 500
URL: https://kaolin.holesky.golem-base.io/rpc
Request body: {"method":"golembase_queryEntities","params":["type=\"CoinHistoricalPrice\""]}
Details: {"code":-32020,"message":"backend response too large"}
Version: viem@2.31.4
I later found out the response size limit is 5 MB.
To address this, there needs to be support for pagination or batching so I can query the data in smaller chunks and combine it on the client side. This would make it possible to fetch all relevant data without hitting the size limit.
Without a solution to this problem, I basically know there's all the data I need in there, but I have no way to retrieve it.