Skip to content

Add support for pagination of data to avoid RPC errors due to responses being too large. #24

@cryptobench

Description

@cryptobench

While working on the CoinMarketCap clone, I ran into a limitation when trying to display data for the top 100 coins on the homepage.

Fetching data for more than around 30 coins resulted in a failed RPC call due to response size limits:

HttpRequestError: HTTP request failed.

Status: 500  
URL: https://kaolin.holesky.golem-base.io/rpc  
Request body: {"method":"golembase_queryEntities","params":["type=\"CoinHistoricalPrice\""]}

Details: {"code":-32020,"message":"backend response too large"}  
Version: viem@2.31.4

I later found out the response size limit is 5 MB.

To address this, there needs to be support for pagination or batching so I can query the data in smaller chunks and combine it on the client side. This would make it possible to fetch all relevant data without hitting the size limit.

Without a solution to this problem, I basically know there's all the data I need in there, but I have no way to retrieve it.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions