Skip to content

fix(declarative): Support nested $ref patterns in dynamic stream templates (do not merge) #708

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 1 commit into
base: main
Choose a base branch
from
Draft

fix(declarative): Support nested $ref patterns in dynamic stream templates (do not merge) #708

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
42 changes: 42 additions & 0 deletions airbyte_cdk/sources/declarative/declarative_component_schema.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1508,6 +1508,13 @@ definitions:
- "$ref": "#/definitions/SimpleRetriever"
- "$ref": "#/definitions/AsyncRetriever"
- "$ref": "#/definitions/CustomRetriever"
- type: object
properties:
$ref:
type: string
pattern: "^#/definitions/"
Copy link

Copilot AI Aug 11, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The regex pattern ^#/definitions/ only validates the prefix but doesn't prevent potentially malicious or malformed references like #/definitions/../../../sensitive/path. Consider using a more restrictive pattern that validates the complete reference structure.

Suggested change
pattern: "^#/definitions/"
pattern: "^#/definitions/[A-Za-z0-9_-]+$"

Copilot uses AI. Check for mistakes.

required: ["$ref"]
additionalProperties: true
Copy link

Copilot AI Aug 11, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Setting additionalProperties: true allows any additional properties beyond $ref, which could lead to unexpected behavior or security issues. Consider restricting this to only allow known safe properties or set to false if additional properties aren't necessary.

Suggested change
additionalProperties: true
additionalProperties: false

Copilot uses AI. Check for mistakes.

incremental_sync:
title: Incremental Sync
description: Component used to fetch data incrementally based on a time field in the data.
Expand Down Expand Up @@ -2414,6 +2421,13 @@ definitions:
- "$ref": "#/definitions/SimpleRetriever"
- "$ref": "#/definitions/AsyncRetriever"
- "$ref": "#/definitions/CustomRetriever"
- type: object
properties:
$ref:
type: string
pattern: "^#/definitions/"
required: ["$ref"]
additionalProperties: true
schema_filter:
title: Schema Filter
description: Responsible for filtering fields to be added to json schema.
Expand Down Expand Up @@ -3214,6 +3228,13 @@ definitions:
anyOf:
- "$ref": "#/definitions/DeclarativeStream"
- "$ref": "#/definitions/StateDelegatingStream"
- type: object
properties:
$ref:
type: string
pattern: "^#/definitions/"
required: ["$ref"]
additionalProperties: true
parent_key:
title: Parent Key
description: The primary key of records from the parent stream that will be used during the retrieval of records for the current substream. This parent identifier field is typically a characteristic of the child records being extracted from the source API.
Expand Down Expand Up @@ -3654,6 +3675,13 @@ definitions:
anyOf:
- "$ref": "#/definitions/HttpRequester"
- "$ref": "#/definitions/CustomRequester"
- type: object
properties:
$ref:
type: string
pattern: "^#/definitions/"
required: ["$ref"]
additionalProperties: true
decoder:
title: HTTP Response Format
description: Component decoding the response so records can be extracted.
Expand Down Expand Up @@ -4202,6 +4230,13 @@ definitions:
- "$ref": "#/definitions/SimpleRetriever"
- "$ref": "#/definitions/AsyncRetriever"
- "$ref": "#/definitions/CustomRetriever"
- type: object
properties:
$ref:
type: string
pattern: "^#/definitions/"
required: ["$ref"]
additionalProperties: true
components_mapping:
type: array
items:
Expand Down Expand Up @@ -4336,6 +4371,13 @@ definitions:
anyOf:
- "$ref": "#/definitions/DeclarativeStream"
- "$ref": "#/definitions/StateDelegatingStream"
- type: object
properties:
$ref:
type: string
pattern: "^#/definitions/"
required: ["$ref"]
additionalProperties: true
components_resolver:
title: Components Resolver
description: Component resolve and populates stream templates with components values.
Expand Down
Loading