Skip to content

fix(pagination): cursor-driven paginate to stop duplicate page fetches (Closes #403) - #561

Merged
David-patrick-chuks-02 merged 1 commit into
Lilly-Protocol:mainfrom
foxxx009:fix/pagination-cursor-driven
Sep 6, 2026
Merged

fix(pagination): cursor-driven paginate to stop duplicate page fetches (Closes #403)#561
David-patrick-chuks-02 merged 1 commit into
Lilly-Protocol:mainfrom
foxxx009:fix/pagination-cursor-driven

Conversation

@foxxx009

@foxxx009 foxxx009 commented Sep 6, 2026

Copy link
Copy Markdown
Contributor

Closes #403

Problem

paginate() in src/pagination.ts:46-77 built a fresh { limit } query on every loop iteration and never read cursor metadata from the response, so a page containing exactly limit items was re-requested until maxPages, yielding the same items repeatedly. The sibling helpers parseCursorPage and buildPaginationQuery (same file) were never called by paginate, and tests/pagination-helper.test.ts explicitly asserted the duplicate behavior (6 yielded items for a single-page dataset).

Changes

src/pagination.ts

  • paginate() now accepts a fetchPage that returns CursorPage<T> (items + nextCursor + hasMore) instead of a bare array
  • The returned nextCursor is fed back through buildPaginationQuery for the next request; the first call carries only the limit
  • Iteration stops when nextCursor is null/empty, hasMore is false, a page is shorter than limit (when limit is set), or maxPages is reached

tests/pagination-helper.test.ts

  • Removed the test asserting the duplicate-page behavior
  • Added 7 cursor-driven cases: two-page dataset (asserts first call has no cursor and second call receives { cursor: 'c1' }), no-duplicate guarantee for a full page, maxPages stop while cursors keep coming, empty page, short page, hasMore: false stop, single cursor-less page

Verification

  • npx vitest run tests/pagination-helper.test.ts — 16/16 pass
  • npm run typecheck — clean
  • npm run test:unit — 873 passed, only the 4 pre-existing Windows path-resolution failures in exports-subpath-resolves / tree-shaking remain (zero new failures vs. baseline)
  • prettier --check and eslint clean on both touched files

…hing the first page

paginate() rebuilt a fresh { limit } query on every loop iteration and
never read cursor metadata, so any page containing exactly limit items
was re-requested forever, yielding duplicate items. The sibling helpers
parseCursorPage/buildPaginationQuery were never used by paginate.

- paginate() now accepts a fetch that returns CursorPage<T> and feeds
  nextCursor back through buildPaginationQuery for the next request
- stops when the cursor is null/empty, a page is shorter than limit,
  or maxPages is reached (hasMore=false is also honored)
- rewrote the paginate test block: cursor advancement, no-duplicate
  guarantee for a full page, maxPages, empty page, short page, and
  hasMore=false stop conditions (16 tests, all passing)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bounty: $100] Stop paginate from re-fetching the same page; advance on cursor metadata instead

2 participants