Skip to content

feat(source-stripe): Add optional config parameters for performance optimization #72393

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
devin-ai-integration wants to merge 8 commits into
base: master
Choose a base branch
from
Open

feat(source-stripe): Add optional config parameters for performance optimization #72393

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
9187808
feat(source-stripe): Add optional config parameters for performance o…
devin-ai-integration[bot] Jan 27, 2026
850bdd2
chore: bump-version
octavia-squidington-iii Jan 27, 2026
9bb1c89
fix: Remove invalid subscription_status enum values
devin-ai-integration[bot] Jan 27, 2026
c1ac50a
fix: Restore path parameter in balance_transactions stream $parameters
devin-ai-integration[bot] Jan 27, 2026
10667e8
fix: Remove balance_transaction_type config (incompatible with create…
devin-ai-integration[bot] Jan 27, 2026
be11c21
Merge branch 'master' into devin/1769553761-stripe-performance-improv…
agarctfi Jan 28, 2026
9e5e976
update dockerImageTag
agarctfi Jan 28, 2026
e779173
docs: Add documentation for new Stripe performance config options
devin-ai-integration[bot] Jan 28, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
37 changes: 34 additions & 3 deletions airbyte-integrations/connectors/source-stripe/manifest.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -356,7 +356,7 @@ definitions:
requester:
$ref: "#/definitions/base_requester"
request_parameters:
status: "all"
status: "{{ config.get('subscription_status', 'all') }}"
schema_loader:
type: InlineSchemaLoader
schema:
Expand Down Expand Up @@ -386,7 +386,7 @@ definitions:
requester:
$ref: "#/definitions/base_requester"
request_parameters:
expand[]: '{{["data.discounts", "data.total_tax_amounts.tax_rate"]}}'
expand[]: '{{ ["data.discounts", "data.total_tax_amounts.tax_rate"] if config.get("expand_invoice_details", true) else [] }}'
Copy link
Contributor

@agarctfi agarctfi Jan 27, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you link the correct API Docs in the PR description for the Event-based endpoint? Also are we sure that leaving expand[]: [] will have the correct intended behavior?

Copy link
Contributor Author

@devin-ai-integration devin-ai-integration bot Jan 27, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good questions!

  1. Event-based endpoint API docs: The invoices incremental stream uses the Events API with types[] filter for invoice events. The expand parameters only apply to the full refresh stream which uses the Invoices List API. I'll update the PR description to clarify this distinction.

  2. Empty expand[]: [] behavior: Looking at the Stripe API behavior, when you pass an empty array for expand, it's equivalent to not passing the expand parameter at all - Stripe will return the default response without expanded objects. However, I should verify this doesn't cause issues with the CDK's request parameter handling.

Let me verify the empty array behavior is handled correctly by the declarative framework. If there are concerns, we could alternatively use a conditional that omits the parameter entirely when disabled rather than passing an empty array.

schema_loader:
type: InlineSchemaLoader
schema:
Expand Down Expand Up @@ -520,7 +520,7 @@ definitions:
requester:
$ref: "#/definitions/base_requester"
request_parameters:
expand[]: '{{["data.refunds"]}}'
expand[]: '{{ ["data.refunds"] if config.get("expand_charge_refunds", true) else [] }}'
schema_loader:
type: InlineSchemaLoader
schema:
Expand Down Expand Up @@ -1898,6 +1898,37 @@ spec:
The number of API calls per second that you allow connector to make. This value can not be bigger than real
API call rate limit (https://stripe.com/docs/rate-limits). If not specified the default maximum is 25 and 100
calls per second for test and production tokens respectively.
order: 6
subscription_status:
type: string
title: Subscription Status Filter
description: >-
Filter subscriptions by status. By default, all subscriptions are synced (including canceled).
See <a href="https://docs.stripe.com/api/subscriptions/list">Stripe API docs</a> for details.
enum:
- all
- canceled
- ended
default: "all"
order: 7
expand_charge_refunds:
type: boolean
title: Expand Charge Refunds
description: >-
When enabled, the charges stream will include full refund objects embedded in each charge.
When disabled, only refund IDs are included, reducing response size.
See <a href="https://docs.stripe.com/api/charges/list">Stripe API docs</a> for details.
default: true
order: 8
expand_invoice_details:
type: boolean
title: Expand Invoice Details
description: >-
When enabled, the invoices stream will include full discount and tax rate objects.
When disabled, only IDs are included, reducing response size.
See <a href="https://docs.stripe.com/api/invoices/list">Stripe API docs</a> for details.
default: true
order: 9

api_budget:
type: HTTPAPIBudget
Expand Down
2 changes: 1 addition & 1 deletion airbyte-integrations/connectors/source-stripe/metadata.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ data:
connectorSubtype: api
connectorType: source
definitionId: e094cb9a-26de-4645-8761-65c0c425d1de
dockerImageTag: 5.15.17
dockerImageTag: 5.16.0
dockerRepository: airbyte/source-stripe
documentationUrl: https://docs.airbyte.com/integrations/sources/stripe
erdUrl: https://dbdocs.io/airbyteio/source-stripe?view=relationships
Expand Down
13 changes: 12 additions & 1 deletion docs/integrations/sources/stripe.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -64,7 +64,13 @@ For more information on Stripe API Keys, see the [Stripe documentation](https://

If you are unsure of which value to use, we recommend leaving this setting at its default value of 365 days.

10. Click **Set up source** and wait for the tests to complete.
10. (Optional) For **Subscription Status Filter**, you can filter which subscriptions are synced. By default, all subscriptions are synced (including canceled). You can choose to sync only `canceled` or `ended` subscriptions to reduce data volume. See the [Stripe Subscriptions API](https://docs.stripe.com/api/subscriptions/list) for details.

11. (Optional) For **Expand Charge Refunds**, you can disable this to exclude full refund objects from the charges stream, reducing response payload sizes. When disabled, only refund IDs are included. This setting only affects full refresh syncs. See the [Stripe Charges API](https://docs.stripe.com/api/charges/list) for details.

12. (Optional) For **Expand Invoice Details**, you can disable this to exclude full discount and tax rate objects from the invoices stream, reducing response payload sizes. When disabled, only IDs are included. This setting only affects full refresh syncs. See the [Stripe Invoices API](https://docs.stripe.com/api/invoices/list) for details.

13. Click **Set up source** and wait for the tests to complete.

<HideInUI>

Expand Down Expand Up @@ -158,6 +164,10 @@ The Stripe connector should not run into Stripe API limitations under normal usa
Please be aware: this also means that any change older than 30 days will not be replicated using the incremental sync mode. If you want all your synced data to remain up to date, please set up your sync frequency to no more than 30 days.
:::

#### Performance optimization settings

The **Expand Charge Refunds** and **Expand Invoice Details** configuration options only affect **full refresh** syncs, which use the direct List APIs. **Incremental** syncs use the [Events API](https://docs.stripe.com/api/events/list) which returns full objects without expand parameters, so these settings have no effect on incremental sync performance.

### Troubleshooting

Since the Stripe API does not allow querying objects which were updated since the last sync, the Stripe connector uses the Events API under the hood to implement incremental syncs and export data based on its update date.
Expand Down Expand Up @@ -246,6 +256,7 @@ Each record is marked with `is_deleted` flag when the appropriate event happens

| Version | Date | Pull Request | Subject |
|:------------|:-----------|:-------------------------------------------------------------|:------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| 5.16.0 | 2026-01-28 | [72393](https://github.com/airbytehq/airbyte/pull/72393) | feat(source-stripe): Add optional config parameters for performance optimization |
| 5.15.17 | 2026-01-27 | [72363](https://github.com/airbytehq/airbyte/pull/72363) | fix(source-stripe): Add date filtering for checkout_sessions full refresh |
| 5.15.16 | 2026-01-20 | [72106](https://github.com/airbytehq/airbyte/pull/72106) | Update dependencies |
| 5.15.15 | 2026-01-14 | [71614](https://github.com/airbytehq/airbyte/pull/71614) | Update dependencies |
Expand Down
Loading