Skip to content

Children speed improvements#20

Open
manuelapilongo wants to merge 14 commits into
singer-io:masterfrom
manuelapilongo:children-speed-improvements
Open

Children speed improvements#20
manuelapilongo wants to merge 14 commits into
singer-io:masterfrom
manuelapilongo:children-speed-improvements

Conversation

@manuelapilongo

Copy link
Copy Markdown

Description of change

Suggested changes to improve sync speed on subsequent runs.
On each run, the amount of contacts field values or contact field automations synchronised it's always same and bookmarks aren't used. With this change the amount synchronised is based on how many contacts were updated since previous run.

Manual QA steps

  • Executed successful run with and saved state
  • Updated a sample set of contacts custom fields
  • Verified that fields for only the sample set of contacts were synchronised.

Risks

Rollback steps

  • revert this branch

@manuelapilongo

Copy link
Copy Markdown
Author

Hi all, in our setup, on each run we synchronise more that 200K contacts with more than 20 custom field each. Each run took more than 24 hours. With these changes we achieved improvements in speed on an average day were not all the contacts( we have) updated at least a value. I'll be pleased to hear your opinion regarding these changes and alternative ways to improve the overall speed. Thank you in advance.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants