Skip to content

payout_transactions syncs too much duplicate data #63

@idris

Description

@idris

I am using Stitch and replicating Stripe every 30 minutes. As you can see in the screenshot below, for the last 12 hours, the payout_transactions table has replicated 8,701 rows every 30 minutes.

My guess is that it's selecting the latest payout (because it's >= the created) of the latest replicated one), and then querying Stripe for all balance_transactions related to that payout. In my opinion, it should either (a) use > for payouts, instead of >= or (b) keep the latest payout.id in addition to the created date, and if we've already replicated that id, don't bother fetching the balance_transactions to fill in the payout_transactions sub-stream.

The impact of the current behavior is (1) waste of requests to Stripe and (2) a lot of unnecessary additional cost on Stitch.

Screen Shot 2020-04-20 at 10 15 44 AM

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions