Skip to content

Conversation

jpefaur
Copy link
Contributor

@jpefaur jpefaur commented Oct 10, 2025

What

The existing CDK Connector Compatibility Test workflow currently tests all connectors using the bulk CDK. However, we only want to test certified connectors. This PR modifies the workflow to run tests exclusively on certified connectors using the CDK.

How

Instead of getting the list of connectors to test from the diff generated by running a CDK upgrade task, we will get the list of certified connectors by reusing the script used in the auto bump workflow to get this information.

How did I test this?

Tested it in this PR. More specifically, here's the PR check run https://github.com/airbytehq/airbyte/actions/runs/18410594109/job/52461620803?pr=67605

Note: it looks like the connectors we are testing didn't change. I'd guess it is possible that in the future we will have some none certified connectors using the cdk so the change in this PR still makes sense.

Can this PR be safely reverted and rolled back?

  • YES 💚
  • NO ❌

Copy link
Contributor

👋 Greetings, Airbyte Team Member!

Here are some helpful tips and reminders for your convenience.

Helpful Resources

PR Slash Commands

Airbyte Maintainers (that's you!) can execute the following slash commands on your PR:

  • /format-fix - Fixes most formatting issues.
  • /bump-version - Bumps connector versions.
    • You can specify a custom changelog by passing changelog. Example: /bump-version changelog="My cool update"
    • Leaving the changelog arg blank will auto-populate the changelog from the PR title.
  • /run-cat-tests - Runs legacy CAT tests (Connector Acceptance Tests)
  • /build-connector-images - Builds and publishes a pre-release docker image for the modified connector(s).
  • JVM connectors:
    • /update-connector-cdk-version connector=<CONNECTOR_NAME> - Updates the specified connector to the latest CDK version.
      Example: /update-connector-cdk-version connector=destination-bigquery
    • /bump-bulk-cdk-version type=patch changelog='foo' - Bump the Bulk CDK's version. type can be major/minor/patch.
  • Python connectors:
    • /poe connector source-example lock - Run the Poe lock task on the source-example connector, committing the results back to the branch.
    • /poe source example lock - Alias for /poe connector source-example lock.
    • /poe source example use-cdk-branch my/branch - Pin the source-example CDK reference to the branch name specified.
    • /poe source example use-cdk-latest - Update the source-example CDK dependency to the latest available version.

📝 Edit this welcome message.

- name: List certified connectors
id: list-connectors
run: |
json_array=$(tools/bin/bulk-cdk-auto-upgrade/list-connectors-to-upgrade.sh)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

changed where the script lives since it is now being used not only in the auto-upgrade context

uses: actions/checkout@08eba0b27e820071cde6df949e0beb9ba4906955 # v4.3.0
with:
submodules: true # Needed for airbyte-enterprise connectors (no-op otherwise)
fetch-depth: 0
Copy link
Contributor Author

@jpefaur jpefaur Oct 10, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we need this now since we are not using a git diff to know which connectors changed

- name: Generate Connector Matrix
id: generate-matrix
run: echo "connectors_matrix=$(./poe-tasks/get-modified-connectors.sh --json --local-cdk)" | tee -a $GITHUB_OUTPUT
run: echo "connectors_matrix=$(tools/bin/bulk-cdk/get-certified-connectors.sh)" | tee -a $GITHUB_OUTPUT
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

reuse auto bump script

@jpefaur jpefaur changed the title Jose/run cdk compatibility on certified only ci: run cdk compatibility test only on certified connectors Oct 10, 2025
@jpefaur jpefaur marked this pull request as ready for review October 10, 2025 16:28
@jpefaur jpefaur requested review from davinchia and edgao October 10, 2025 16:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants