fix: keep pagination iterators advancing past fully filtered pages

[vdusek]

TLDR

• Mirrors fix: Keep pagination iterators advancing past fully-filtered pages apify-client-python#964, which fixed the same bug in the Python client.
• The API-side root cause is described in apify/apify-core#27324.
• Closes: Pagination iterator mishandles server-side-filtered dataset pages #979

Description

The offset-based iterator stopped as soon as a page came back with no items, and advanced the offset by the number of items returned. Dataset filters (clean, skipEmpty, skipHidden) apply after offset and limit, so a page can scan up to limit rows while returning fewer, or none at all. listItems() used as an async iterable therefore ended early on a fully filtered page, and re-yielded rows an earlier page had already returned.

The iterator now paginates by the scanned count the API reports in x-apify-pagination-count, and by that number alone. items.length cannot stand in for it in either direction: filters leave it below the rows scanned and unwind can leave it above, so mixing the two would skip rows. PaginatedList.count stays items.length, since it is public and intentionally set that way; the scanned number travels with the page as a non-enumerable symbol property outside the public shape. Collection endpoints send no such header and fall back to items.length, so nothing about them changes.

The cursor iterators (listKeys(), listRequests(), paginateRequests()) are unchanged. The API never returns an empty page together with a cursor: the request queue listing hands out nextCursor only when it filled the whole page, and the key-value store listing derives nextExclusiveStartKey from the last key it returned.

Test plan

Regression tests for the fully filtered page, the duplicated offset, and unwind returning more items than rows scanned.

✍️ Drafted by Claude Code

[github-actions]

⚠️ There are broken links in the documentation.

See more at https://github.com/apify/apify-client-js/actions/runs/34473247015#summary-102857850927

[barjin]

Thank you @vdusek ,

I have a few findings ⬇️ , the first two are just notes (perhaps these changes are not necessary), the last one imo a bit more actionable.