Skip to content

feat: Support for Apify "multiple datasets" feature#571

Merged
janbuchar merged 7 commits intomasterfrom
multiple-storages
Mar 10, 2026
Merged

feat: Support for Apify "multiple datasets" feature#571
janbuchar merged 7 commits intomasterfrom
multiple-storages

Conversation

@janbuchar
Copy link
Contributor

  • closes Adapt to Apify's "multiple storages" functionality #552
  • on platform, it uses ACTOR_STORAGES_JSON
  • instead of changing idOrName to idOrNameOrAlias, a new way to open storages was added, so opening by alias is done like this: Actor.openDataset({alias: "xxxx"})
  • locally, this falls back to named storages and purges them when they are first accessed
  • non-default unnamed KVS or RQs are not currently supported by Apify, so we forbid using aliases for those on the type level

@janbuchar janbuchar added the t-tooling Issues with this label are in the ownership of the tooling team. label Mar 4, 2026
@github-actions github-actions bot added this to the 135th sprint - Tooling team milestone Mar 4, 2026
@github-actions github-actions bot added the tested Temporary label used only programatically for some analytics. label Mar 4, 2026
Copy link
Member

@barjin barjin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure I have all the context, here are some quick points I came up with reading this.

@janbuchar janbuchar requested review from Pijukatel and barjin March 6, 2026 15:05
@B4nan B4nan force-pushed the multiple-storages branch from ad8ed41 to 5b1a54f Compare March 6, 2026 22:55
Copy link
Member

@barjin barjin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm, thank you @janbuchar

Copy link
Member

@valekjo valekjo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, thanks!

I have some questions about how actor.json should / should not be respected locally, but that is rather something for v4 - the local behavior as implemented is close enough to the platform

@janbuchar
Copy link
Contributor Author

@valekjo for the sake of completeness - apify/apify-sdk-python#827 - alias="default" should now work the same in both SDK flavors

@janbuchar janbuchar merged commit 0809bd3 into master Mar 10, 2026
18 checks passed
@janbuchar janbuchar deleted the multiple-storages branch March 10, 2026 16:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

t-tooling Issues with this label are in the ownership of the tooling team. tested Temporary label used only programatically for some analytics.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Adapt to Apify's "multiple storages" functionality

5 participants