Skip to content

feat: sync Rust client with Apify OpenAPI spec v2-2026-07-10T105921Z; add lazy pagination iterators - #17

Merged
Pijukatel merged 20 commits into
masterfrom
claude/busy-ride-6r538u
Jul 12, 2026
Merged

feat: sync Rust client with Apify OpenAPI spec v2-2026-07-10T105921Z; add lazy pagination iterators#17
Pijukatel merged 20 commits into
masterfrom
claude/busy-ride-6r538u

Conversation

@Pijukatel

@Pijukatel Pijukatel commented Jul 10, 2026 •
edited
Loading

Copy link
Copy Markdown
Collaborator

Syncs the Rust client to Apify OpenAPI spec v2-2026-07-10T105921Z and adds lazy pagination iterators across the client. Version bumped to 0.6.0.

Added

  • Lazy async pagination iterators on every collection client via a shared generic ListIterator<T> (exported at the crate root): iterate() on the actor, actor-version, env-var, build, run, dataset, key-value-store, request-queue, schedule, task, webhook, and webhook-dispatch collection clients, plus DatasetClient::iterate_items(). options.limit caps the total items yielded (matching the reference client); ListIterator::with_chunk_size sets the per-request page size.
  • KeyValueStoreClient::iterate_keys() returning a cursor-based KeyValueStoreKeysIterator (auto-paginates keys via exclusiveStartKey/nextExclusiveStartKey, mirroring the reference listKeys() async-iterable).
  • Re-exported StoreActorIterator at the crate root.

Changed

  • Bumped API_SPEC_VERSION to v2-2026-07-10T105921Z. The spec delta (added 401/402 error responses, relaxed field nullability) needs no functional change: errors are handled generically and response models are forward-compatible.
  • StoreCollectionClient::iterate now uses the shared ListIterator; store().iterate()'s options.limit changed from a per-page size to a total-item cap for consistency with the reference client (set page size via with_chunk_size). StoreActorIterator is now a type alias for ListIterator<ActorStoreListItem>.
  • Documented ApifyClient::set_status_message and BuildClient::get_openapi_definition.

Removed

  • KeyValueStoreClient::get_records and GetRecordsOptions — the GET /v2/key-value-stores/{storeId}/records endpoint is not implemented by the reference JS client, so it is out of scope (corrects an earlier scope violation).

Documentation

  • Documented all ActorStartOptions fields (added the previously undocumented restart_on_error, force_permission_level, webhooks); listed JSONL in the download_items format summary; made the run_store_actor example resilient to Store ranking shifts.

Test stability

  • Made the create-then-iterate collection integration tests tolerant of list-endpoint eventual consistency via a shared bounded retry helper (tests/common), fixing an intermittent iterate_key_value_stores failure. Test-only; no client runtime behaviour changed.

See CHANGELOG.md for the full [0.6.0] entry.

claude added 20 commits July 10, 2026 12:40
@claude
chore: sync Rust client with Apify OpenAPI spec v2-2026-07-10T105921Z
20016ec 
Bumps API_SPEC_VERSION to v2-2026-07-10T105921Z and crate version to 0.5.1.
The spec delta (added 401/402 error responses and relaxed field
nullability/optionality) requires no code change: errors are handled
generically and response models are forward-compatible.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019rkZYoebybdnv7MFH9Ua6E
@claude
feat: add lazy pagination iterators to all collection clients
2c5e045 
Adds a shared generic ListIterator<T> (src/clients/pagination.rs, exported
at the crate root) and an iterate() method on every collection client the
reference JS client iterates: actors, actor versions, env vars, builds,
runs, datasets, key-value stores, request queues, schedules, tasks,
webhooks, webhook dispatches, plus DatasetClient::iterate_items(). Store
iteration is refactored onto the shared iterator (StoreActorIterator kept
as a type alias). Termination is short-page based, robust to the
dataset-items endpoint reporting total=0.

Adds one item-iteration integration test per collection and corrects the
src/models.rs module doc. Minor version bump to 0.6.0.

Addresses review items in notes.md (iteration helpers, iteration tests,
module-doc accuracy).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019rkZYoebybdnv7MFH9Ua6E
@claude
test: add hermetic ListIterator unit tests; single-page env-var iterator
c8efe6e 
Addresses staff-review nits from notes.md:
- Add ListIterator::new_single_page for non-paginated endpoints (env-vars),
  removing the reliance on the API omitting a page limit.
- Reword pagination.rs module doc to drop the task/requirements reference.
- Add token-free unit tests covering all ListIterator termination branches
  (short-page, total=0, empty-page, reached_total early-stop, caller offset,
  single-page); tests exercise collect_all as their drain path.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019rkZYoebybdnv7MFH9Ua6E
@claude
docs: clarify iterate() limit semantics; drop process reference in ba…
84a9f3d 
…se.rs comment

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019rkZYoebybdnv7MFH9Ua6E
@claude
fix: align iterate() limit with reference (total cap) and stop short-…
28b49d1 
…page truncation

- ListIterator: limit is now a total-item cap; page size set via with_chunk_size,
  matching the reference _listPaginatedFromCallback (total-driven termination).
- iterate_items no longer truncates when skip_empty/clean/skip_hidden shorten a
  non-final page (short-page detection confined to the total==0 case).
- Corrected misleading limit-as-page-size docs (pagination.rs, dataset.rs, docs/README.md).
- Added hermetic unit tests for non-final short pages and total-cap behaviour.
- README: bump apify-client 0.5 -> 0.6 to match Cargo.toml.
- Iteration integration tests use with_chunk_size for page size (limit unset).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019rkZYoebybdnv7MFH9Ua6E
@claude
test: force multi-page fetching in iterate_store via with_chunk_size
4d84c41 
Migrate the last iteration test off limit-as-page-size (now a total cap) so its
seen>=12 break is reachable and it exercises page-crossing, matching siblings.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019rkZYoebybdnv7MFH9Ua6E
@claude
docs: correct iterate_items/pagination docs and add cap-truncation test
44b8f70 
Address 5 open review items (behaviour unchanged; reference-consistent):
- iterate_items docstring now describes real offset-advance behaviour
  (may re-yield filtered items), matching the reference JS client, instead
  of overpromising filters are honoured without truncation.
- Reword non_final_short_page test comment: it guards termination logic,
  not filter de-duplication.
- Add cap_truncates_page_that_exceeds_remaining_budget test for the
  received > remaining truncation branch.
- README timestamp prose: Option<chrono::DateTime<Utc>>.
- docs/README imports list: add ListIterator (a return type).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019rkZYoebybdnv7MFH9Ua6E
@claude
docs: clarify iterate() limit is total cap; harden missing-total head…
d61a7fe 
…er; extract iterate boilerplate

- Document on all 11 collection iterate() methods that options.limit is a
  total-item cap (page size via with_chunk_size), pointing to ListIterator.
- list_items reports total=0 (not count) when the pagination-total header is
  absent, so iterate_items no longer falsely terminates after page 1; add a
  hermetic header-less test in unit_http.rs.
- Remove orphaned historical narration from a pagination test comment.
- Extract the duplicated iterate() closure into a crate-internal list_iterator!
  macro used by the 10 uniform collections (run/dataset-items/env-var keep their
  explicit bodies).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019rkZYoebybdnv7MFH9Ua6E
@claude
docs: add iterate() total-cap note and list_iterator! macro to task_c…
0edc585 
…ollection

Completes the previous commit: task_collection was the 12th option-taking
collection and was missed by both the doc note (item 1) and the macro
extraction (item 4). All 11 uniform collections now use list_iterator!.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019rkZYoebybdnv7MFH9Ua6E
@claude
feat: add KVS iterate_keys, hermetic iterator tests, reconcile iterat…
df02875 
…or docs

Adds KeyValueStoreClient::iterate_keys with a cursor-based
KeyValueStoreKeysIterator (exclusiveStartKey/nextExclusiveStartKey), matching
the JS reference listKeys() async-iterable. Adds an integration test and
hermetic MockBackend coverage for the list_iterator! macro wiring,
RunCollectionClient::iterate, and the new key iterator. Reconciles the
StoreActorIterator/ListIterator docs, adds iterate()/iterate_items()/iterate_keys()
to the per-resource tables, documents with_chunk_size on the Store page, adds a
large-cap first-page doc note, and clarifies StoreListOptions.limit.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019rkZYoebybdnv7MFH9Ua6E
@claude
fix: normalize/clamp iterate_keys page limit, re-export cursor iterat…
d6809f4 
…ors, doc polish

Derives every iterate_keys page request limit from the remaining budget clamped
to a documented KEY_LIST_MAX_LIMIT (1000), so limit=0 means uncapped and a large
finite cap paginates instead of sending an out-of-range limit. Adds a defensive
cap-truncation guard, drops the unused collect_all, and re-exports
KeyValueStoreKeysIterator and RequestQueueRequestsIterator at the crate root.
Adds hermetic zero-limit and large-cap tests, a KeyValueStoreKey docs field
table, ListKeysOptions.limit dual-meaning callouts, and reworded CHANGELOG to
disclose the store().iterate() limit semantic change.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019rkZYoebybdnv7MFH9Ua6E
@claude
docs: make iterate_keys example self-contained and avoid Option debug…
fd51514 
… output

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019rkZYoebybdnv7MFH9Ua6E
@claude
fix: iterate_keys terminates on isTruncated; drop QueryParams from do…
bc4e40a 
…cs import list

- iterate_keys now leads termination with the model's is_truncated flag (cursor
  presence retained as fallback), avoiding a wasted final fetch and matching model
  semantics. Adds hermetic test iterate_keys_stops_on_is_truncated_even_with_trailing_cursor.
- Drop QueryParams from the docs crate-root import prose (no public API consumes a
  caller-built QueryParams); re-export left intact.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019rkZYoebybdnv7MFH9Ua6E
@claude
fix: remove out-of-scope KVS get_records endpoint
4996e06 
GET /v2/key-value-stores/{storeId}/records is not exposed by the reference
JS client, so per client_requirements it is out of scope. Remove get_records
and GetRecordsOptions (method, re-export, docs rows, integration test) and
record the removal in the changelog. Also correct the iterate_keys doc wording
to not claim it mirrors the reference generator's cursor loop.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019rkZYoebybdnv7MFH9Ua6E
@claude
docs: document set_status_message and get_openapi_definition; simplif…
d4d1221 
…y iterator cap

Address 3 open items from the independent full client review:
- Document ApifyClient::set_status_message in docs/README.md + root README.
- Add BuildClient::get_openapi_definition to the docs/builds.md method table.
- Drop dead .max(0) clamp in the offset and KVS iterators (rem is provably > 0).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019rkZYoebybdnv7MFH9Ua6E
@claude
docs: clarify iterate_items filter edge cases, trim comment, sync ite…
c1e3757 
…rate() docs

- Document the item-drop edge case (fully-filtered window -> empty page ->
  walk terminates) in iterate_items, alongside the existing duplicate case
- Trim the verbose pagination-total unwrap_or(0) rationale comment to 3 lines
- Add iterate() to the ActorVersion/ActorEnvVar collection tables in
  docs/actors.md to agree with docs/README.md and the code

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019rkZYoebybdnv7MFH9Ua6E
@claude
docs: complete ActorStartOptions fields, JSONL format, robust store e…
255235e 
…xample, clean Option output

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019rkZYoebybdnv7MFH9Ua6E
@claude
docs: link task start/call rows to ActorStartOptions field table
44fb51d 
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019rkZYoebybdnv7MFH9Ua6E
@claude
fix: retry create-then-iterate tests for collection list eventual con…
d0bebf2 
…sistency

The iterate_key_value_stores integration test flaked on CI: a just-created
key-value store was not yet reflected in the collection LIST endpoint when the
test scanned the iterator once. This is eventual consistency, not a pagination
or client bug.

Add a bounded, shared test helper `iter_contains_eventually` that rebuilds the
iterator and re-scans up to 5 times with a 500ms backoff (~2s budget), matching
on the first attempt when the entity is already present (no-op in the common
case). Use it for all 12 create-then-iterate collection assertions that share
this race. Test-only change; no client runtime behaviour is affected.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019rkZYoebybdnv7MFH9Ua6E
@claude
test: widen eventual-consistency retry budget and trim helper docstring
96e6032 
Address flake-fix re-review: raise iter_contains_eventually retry budget from
~2s (5 x 500ms) to ~15s (16 x 1s) so it has real headroom above the ~10s
propagation lag previously observed in this suite, instead of an unverified 2s
guess that could let the flake recur at lower frequency. Consistent accounts
still return on the first attempt with no sleeping. Also drop the concrete
incident-test name from the docstring and de-duplicate the budget prose.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019rkZYoebybdnv7MFH9Ua6E
@Pijukatel Pijukatel changed the title chore: sync Rust client with Apify OpenAPI spec v2-2026-07-10T105921Z; add lazy pagination iterators feat: sync Rust client with Apify OpenAPI spec v2-2026-07-10T105921Z; add lazy pagination iterators Jul 12, 2026
@Pijukatel
Pijukatel merged commit e28725d into master Jul 12, 2026
2 checks passed
@Pijukatel
Pijukatel deleted the claude/busy-ride-6r538u branch July 12, 2026 05:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@Pijukatel @apify-service-account @claude