Skip to content

[Bounty: $100] Stop paginate from re-fetching the same page; advance on cursor metadata instead #403

Description

@isholaquawiy-cmd

Context

paginate in src/pagination.ts builds a fresh { limit } query on every loop iteration and never reads cursor metadata, so when a page contains at least limit items the same first page is requested repeatedly and yielded more than once. The sibling helpers parseCursorPage and buildPaginationQuery defined in the same file are never called by paginate. tests/pagination-helper.test.ts currently encodes the duplicate behavior: paginate(fetchPage, { limit: 2, maxPages: 3 }) against a fetch that always returns [1, 2] asserts 6 yielded items.

Proposed Change

Rework paginate to be cursor-driven: accept a fetch that returns page metadata (items plus a next cursor, matching the existing CursorPage<T> shape), feed the returned cursor back through buildPaginationQuery, and stop when the cursor is null/empty, a page is shorter than limit, or maxPages is reached.

Acceptance Criteria

  • A two-page dataset (page 1 -> cursor c1, page 2 -> null cursor) is yielded once per item, with fetchPage called first with no cursor and then with { cursor: 'c1' }
  • A page whose length equals limit no longer triggers a repeat fetch of the same page (no duplicate items)
  • maxPages, empty-page, and cursor-less responses are covered by updated tests in tests/pagination-helper.test.ts

Suggested Label

bug

ETA: 24 hours

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions