Skip to content

Commit 8811920

Browse files
0xrinegadeclaude
andcommitted
fix(research): Use cursor-based pagination for fetching ALL transfers
## Problem - MCP tool has 50 transfer default limit (not 1000) - `:limit` parameter wasn't being respected - `:offset` pagination not supported by MCP endpoint - Only fetching first 50 transfers regardless of pagination attempt ## Root Cause Analysis MCP server uses **cursor-based pagination**, not offset-based: - Response includes `nextPageSignature` field - Next page requires `beforeSignature` parameter with cursor from previous response - No support for numeric offsets ## Solution Changed from offset-based to cursor-based pagination: **Before (broken):** ```lisp (get_account_transfers {:address addr :limit 1000 :offset 1000}) ``` **After (working):** ```lisp ;; First page (get_account_transfers {:address addr :limit 1000}) ;; Get nextPageSignature from response ;; Next pages (get_account_transfers {:address addr :limit 1000 :beforeSignature cursor}) ;; Repeat until nextPageSignature is null ``` ## Implementation - Loop condition: `(while (or is_first_page next_cursor))` - Dynamically build params based on page (with/without beforeSignature) - Accumulate all pages: `(set! all_transfers (append all_transfers batch))` - Continue until `nextPageSignature` is null ## Expected Impact - Will now fetch ALL wallet transfers (not just first 50) - Handles wallets with 1000s of transfers via proper pagination - Aggregation will work on complete dataset 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]>
1 parent d7001fc commit 8811920

File tree

2 files changed

+18
-13
lines changed

2 files changed

+18
-13
lines changed

docs/book/README.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -72,9 +72,9 @@ Every chapter shows you **exactly** how these disasters could have been prevente
7272
| Chapter | Title | Words | Topics |
7373
|---------|-------|-------|--------|
7474
| [13](13_ai_sentiment_trading.md) | AI-Powered Sentiment Trading | 9,362 | NLP, multi-source verification, AP hack, Musk tweets |
75-
| [14](14_ml_prediction_trading.md) | Machine Learning for Price Prediction | 6,116 | Feature engineering, LSTM, overfitting prevention |
75+
| [14](14_ml_prediction_trading.md) | Machine Learning for Price Prediction | 5,443 | RIEF vs Medallion, prediction decay, replication crisis, walk-forward |
7676

77-
**Subtotal:** 15,478 words
77+
**Subtotal:** 14,805 words
7878

7979
### Part IV: DeFi & MEV Strategies
8080

src/commands/research.rs

Lines changed: 16 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -154,24 +154,29 @@ fn generate_wallet_analysis_script(wallet: &str) -> String {
154154
(do
155155
(define target "{}")
156156
157-
;; Fetch ALL transfers with pagination (1000 at a time)
157+
;; Fetch ALL transfers with cursor-based pagination
158158
(define all_transfers [])
159-
(define keep_fetching true)
160-
(define batch_num 0)
159+
(define next_cursor null)
160+
(define is_first_page true)
161161
162-
(while keep_fetching
162+
(while (or is_first_page next_cursor)
163163
(do
164-
(define resp (get_account_transfers {{:address target :limit 1000 :offset (* batch_num 1000)}}))
164+
;; Build request params based on whether we have a cursor
165+
(define params
166+
(if is_first_page
167+
{{:address target :limit 1000}}
168+
{{:address target :limit 1000 :beforeSignature next_cursor}}))
169+
170+
;; Fetch batch
171+
(define resp (get_account_transfers params))
165172
(define batch (get resp "data"))
166-
(define batch_size (length batch))
167173
168-
;; Append this batch to all_transfers
174+
;; Append to results
169175
(set! all_transfers (append all_transfers batch))
170176
171-
;; Stop if we got less than 1000 (last batch)
172-
(if (< batch_size 1000)
173-
(set! keep_fetching false)
174-
(set! batch_num (+ batch_num 1)))))
177+
;; Get next cursor for pagination
178+
(set! next_cursor (get resp "nextPageSignature"))
179+
(set! is_first_page false)))
175180
176181
;; Now aggregate by token mint
177182
(define by_mint (group-by all_transfers (lambda (tx) (get tx "mint"))))

0 commit comments

Comments
 (0)