stream_partition in AddFields resolves to same value across all partitions

[Andreas Skielboe (askielboe)]

I have a declarative connector (Connector Builder, CDK 7.17.1) with three streams:

1. auth — fetches per-tenant credentials from an external secrets API, outputs records each containing a pre-computed Basic Auth header
2. parent — uses SubstreamPartitionRouter with auth as parent. partition_field: auth_header + request_option injects the Authorization header per tenant. Works correctly — each tenant's data is fetched with the right credentials.
3. child — substream of parent. Needs each parent record's auth_header to authenticate detail requests for the correct tenant.

To pass auth_header from parent to child, I use AddFields on the parent stream to stamp each record:

  - type: AddedFieldDefinition
    path:
      - auth_header
    value: "{{ stream_partition.extra_fields['auth_header'] }}"

The child then reads it via parent_key: auth_header.

Problem: The AddFields Jinja expression resolves to the same partition's value for every record across all partitions. All parent records end up with one tenant's auth header, so the child sends all requests with wrong credentials.

I've tried stream_partition.<partition_field>, stream_partition.extra_fields['...'], renaming fields to avoid collisions, and concurrency_level: 1 — all produce the same result. Meanwhile, request_option and URL templates correctly resolve per-partition.

Is there a known issue with stream_partition inside AddFields in the concurrent code path? Or a different way to propagate partition data onto output records?

[airbyte-support-bot]

Hi Andreas Skielboe (@askielboe), thanks for the very detailed write-up — the reproduction matrix (partition_field vs. extra_fields, renamed fields, concurrency_level: 1) and the contrast you drew between request_option/URL templates working per-partition while AddFields does not is exactly the kind of detail that helps us look into this quickly.

We're not able to confirm a root cause from the description alone, so we've escalated this to our team for investigation: airbytehq/oncall#12159. They'll follow up with any findings or follow-up questions there or back here.

A few things that may be useful while the team takes a look:

Possible workaround to unblock yourself

Since request_option and the URL templater do appear to resolve stream_partition per partition correctly on your parent stream, you can sometimes sidestep AddFields entirely by routing the value through the response itself — for example, by using a ResponseToRecord / record_selector extractor field that pulls the parent's auth context out of the request, or by using a parent_key on the child stream that points directly at a field already present in the parent record (rather than one synthesized via AddFields).

If your auth_header is deterministic from a parent-record field that already exists in the response (e.g. a tenant ID), you can compute the header in the child's authenticator using {{ stream_partition['tenant_id'] | ... }} and skip the AddFields propagation step altogether.

These are workarounds while we investigate, not a confirmation that the behavior you saw is incorrect.

Information that would help us investigate

If you're able to share, the following would speed up the investigation on the linked oncall issue:

• The relevant slice of your manifest YAML — specifically the auth, parent, and child stream definitions, the SubstreamPartitionRouter configuration, and the AddFields block. Please mask or remove any tokens, API keys, passwords, or other secrets before sharing.
• A minimal manifest that reproduces the same behavior with two fake "tenants" (no real credentials) would be ideal if you have time.
• Connector Builder logs from a run where the issue happens (with secrets masked).

---

Need more help? Join the Airbyte Community Slack for peer support, or if you're a Cloud customer, open a support ticket referencing this discussion.

---

Devin session