Skip to content

Export hitting rate limit with very few entries exported #1146

@hartmagj

Description

@hartmagj

My team and I are trying to migrate the content from one contentful environment to another. Running our migration script on a subset of the content works just fine, but when we run our script on a larger set of contentful entries, we receive rate limit errors that look like this (though not necessarily in this order):
Oh no! Some errors occurred! Error [ContentfulMultiError]: Errors occured
11:20:68 - Rate limit error occurred. Waiting for 1624 ms before retrying...
11:20:45 - Connection error occurred. Waiting for 1978 ms before retrying...

After these errors only one or two entries actually get migrated, the rest of the changes seem to be ignored.

For reference this is a relevant piece of the migration script:

const exportOptions = expOpts;
exportOptions.queryEntries = ['sys.id=0000'];
exportOptions.queryAssets = [sys.id=${id}];
contentfulExport(exportOptions)
.then((result) => {
contentfulImport({
content: result,
...impOpts,
});
})
.catch((err) => {
console.log('Oh no! Some errors occurred!', err);
});

This is being run on node 12.19.1 and has been tested on node 16 as well with no improvement

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions