-
Notifications
You must be signed in to change notification settings - Fork 5k
fix(source-stripe): Add date filtering for checkout_sessions full refresh #72363
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
fix(source-stripe): Add date filtering for checkout_sessions full refresh #72363
Conversation
…resh The checkout_sessions stream was using entity_restricted_stream which does not include start_time_option and end_time_option parameters. This caused the full refresh stream to fetch ALL checkout sessions regardless of the configured start_date. Changed to use entity_stream which properly filters by created[gte] and created[lte] parameters, respecting the user's start_date config. Fixes: airbytehq/oncall#11043 Co-Authored-By: [email protected] <[email protected]>
🤖 Devin AI EngineerI'll be helping with this pull request! Here's what you should know: ✅ I will automatically:
Note: I can only respond to comments from users who have write access to this repository. ⚙️ Control Options:
|
👋 Greetings, Airbyte Team Member!Here are some helpful tips and reminders for your convenience. 💡 Show Tips and TricksPR Slash CommandsAirbyte Maintainers (that's you!) can execute the following slash commands on your PR:
📚 Show Repo GuidanceHelpful Resources
|
|
|
Fix Validation SummaryConnector: Validation ThesisSuccess Thesis: The Failure Thesis: The stream will continue to fetch ALL checkout sessions regardless of Test Plan: Test on a connection with Pre-flight Checks
Testing Results
Note: Connection IDs and detailed logs are recorded in the linked private issue. VerdictNO DECISION - MORE DATA NEEDED - Live connection testing was blocked.
Next Steps
Session: https://app.devin.ai/sessions/0cb4d1d980df4bf2a6c7679decbdc084 |
|
/bump-version
|
|
Deploy preview for airbyte-docs ready! ✅ Preview Built with commit 9797c96. |
|
/ai-review
Reviewing PR for connector safety and quality. Phase D adds MCP-based validation verification.
|
|
AI PR Review (Phase D) - Review in progress... Evaluating 6 gates for connector safety and quality. Will post full gate report shortly. Session: https://app.devin.ai/sessions/5a79183540284df8b1d708ad595b736d |
AI PR Review Report (Phase D)Connector(s): Auto-Approve EligibilityEligible: No Review Action TakenNO ACTION (NOT ELIGIBLE) - All gates pass but PR is not eligible for auto-approval. No PR review submitted. Human review required.
Gate Summary
Overall AssessmentALL GATES PASS (NOT ELIGIBLE) - All gates pass but PR is not eligible for auto-approval (contains functional code changes). No automated review submitted. Human review required. Gate Evaluation DetailsGate 1: CI/Tests
Gate 2: Security
Gate 3: Safety/Reversibility
Gate 4: Operational Risk
Gate 5: Pre-Release Validation
Gate 6: PR Quality
Evidence Consulted
|
What
The
checkout_sessionsstream's full refresh mode was not respecting thestart_dateconfiguration, causing it to fetch ALL checkout sessions from the Stripe account regardless of the configured start date. This resulted in extremely long sync times for accounts with large checkout session histories.Fixes: https://github.com/airbytehq/oncall/issues/11043
How
Changed the
checkout_sessionsstream'sfull_refresh_streamto useentity_streaminstead ofentity_restricted_stream.The key difference:
entity_restricted_streamusesentity_single_slice_cursorwhich lacksstart_time_optionandend_time_optionparametersentity_streamusesentity_with_slice_cursorwhich includes these options, addingcreated[gte]andcreated[lte]parameters to API requestsThe Stripe API's List all Checkout Sessions endpoint supports the
createdfilter parameter, so this change is compatible with the API.Review guide
airbyte-integrations/connectors/source-stripe/manifest.yaml- Single line change fromentity_restricted_streamtoentity_streamKey verification points:
checkout_sessionswas usingentity_restricted_stream(e.g., API limitations that no longer exist)entity_restricted_stream(accounts, credit_notes, early_fraud_warnings, external_account_cards, external_account_bank_accounts) might have the same issueUser Impact
Users with large Stripe accounts will see significantly faster initial syncs for the
checkout_sessionsstream, as the connector will now properly filter by the configuredstart_dateinstead of fetching all historical data.Can this PR be safely reverted and rolled back?
Link to Devin run: https://app.devin.ai/sessions/167a186f54c04edcb2918e527a779a0c
Requested by: @agarctfi