Skip to content

Comments

feat(destination-teradata): Implemented Airbyte V2 Typing and Deduping feature#60292

Closed
devin-ai-integration[bot] wants to merge 42 commits intomasterfrom
devin/1747297489-teradata-typing-deduping
Closed

feat(destination-teradata): Implemented Airbyte V2 Typing and Deduping feature#60292
devin-ai-integration[bot] wants to merge 42 commits intomasterfrom
devin/1747297489-teradata-typing-deduping

Conversation

@devin-ai-integration
Copy link
Contributor

@devin-ai-integration devin-ai-integration bot commented May 15, 2025

UPDATE. Closing this PR in favor of the below. Now that BYO secrets are in place for the fork, iteration will continue there on that original PR. 🎉

Copy of:

Implemented Airbyte V2 Typing and Deduping feature to Teradata destination connector

What

  • Implemented Typing and Deduping feature to Teradata destination connector
  • Added integration tests for typing deduping classes
  • Disabled V1 integration tests after DV2 migration. Re-enable with fixtures updated to DV2.
  • The integration tests requires three config files under secrets folder
  • Documentation updated with V2 and added Migration guide from V1 to V2

How

  • By extending Typing and Deduping feature

User Impact

Please refer docs/integrations/destinations/teradata-migrations.md

This PR is a copy of PR #56985 created to allow CI to run with proper permissions and secrets.

Requested by: Aaron ("AJ") Steers (aj@airbyte.io)

Link to Devin run: https://app.devin.ai/sessions/ba53e9a8588d4b96a5a01f240e3e6d8e

…ation connector

Typing and Deduping Implementation
Integration tests
documentation
@devin-ai-integration
Copy link
Contributor Author

Devin is currently unreachable - the session may have died.

@github-actions
Copy link
Contributor

github-actions bot commented May 15, 2025

destination-teradata Connector Test Results

  8 files    8 suites   4h 30m 31s ⏱️
151 tests  75 ✅ 75 💤 1 ❌
187 runs  111 ✅ 75 💤 1 ❌

For more details on these failures, see this check.

Results for commit 28b33cb.

♻️ This comment has been updated with latest results.

@vercel
Copy link

vercel bot commented May 15, 2025

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Comments Updated (UTC)
airbyte-docs ❌ Failed (Inspect) May 29, 2025 6:36am

Copy link
Contributor

@marcosmarxm Marcos Marx (marcosmarxm) left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🚢

@github-project-automation github-project-automation bot moved this from Backlog to Ready to Ship in 🧑‍🏭 Community Pull Requests May 15, 2025
@sc250072
Copy link
Contributor

Aaron ("AJ") Steers (@aaronsteers) Could you please review the workflow below? The Teradata integration tests are currently failing.

https://github.com/airbytehq/airbyte/actions/runs/15046371928/job/42289912895?pr=60292#step:12:716

@davinchia
Copy link
Contributor

Satish Chinthanippu (@sc250072) the above failures is implementation-related and should be able to be reproduce locally by running the integrationTestJava task on the connector. Can you ask your devs to double check this passes their local tests?

@sc250072
Copy link
Contributor

Satish Chinthanippu (@sc250072) the above failures is implementation-related and should be able to be reproduce locally by running the integrationTestJava task on the connector. Can you ask your devs to double check this passes their local tests?

Davin Chia (@davinchia) We raised the PR after the Unit and Integration tests succeeded. Today, I also ran our repository’s GitHub workflow, and the tests passed successfully.
Here the link of workflow.

https://github.com/Teradata/airbyte/actions/runs/15111480303/job/42471778813
image

It seems there's an issue with the secrets configuration files. I've attached the three required config files needed to create the necessary Teradata instances for running integration tests. These files need to be populated with valid tokens and passwords.

raw_override_typing_config.json
typing_config.json
disable_typing_config.json

@sc250072
Copy link
Contributor

Aaron ("AJ") Steers (@aaronsteers)
Could you please check the issue

@aaronsteers
Copy link
Collaborator

Aaron ("AJ") Steers (aaronsteers) commented May 28, 2025

Marcos Marx (@marcosmarxm) and Satish Chinthanippu (@sc250072) - I've updated the secrets (including prev password and token values) in GSM. I only updated the 3 secrets passed above (disable_typing_config, typing_config, and raw_override_typing_config. I did not modify the other secrets values in the screenshot below.

image

Job running here: https://github.com/airbytehq/airbyte/actions/runs/15046371928/job/43072695820

Update: Job failed: https://github.com/airbytehq/airbyte/runs/43072997723

Update: this looks a new error:

TeradataTypingDedupingTest > initializationError FAILED
    java.util.concurrent.ExecutionException: io.airbyte.integrations.destination.teradata.envclient.exception.Error4xxException: ***"message":"Environment limit of 2 reached."***

@github-actions
Copy link
Contributor

github-actions bot commented May 28, 2025

Note

Detected that there are differences in the Gradle dependencies.

@aaronsteers Aaron ("AJ") Steers (aaronsteers) changed the title dupe: feat(destination-teradata): Implemented Airbyte V2 Typing and Deduping feature feat(destination-teradata): Implemented Airbyte V2 Typing and Deduping feature May 28, 2025
@aaronsteers
Copy link
Collaborator

Aaron ("AJ") Steers (aaronsteers) commented Jun 3, 2025

/run-connector-tests

Connector CI Tests Started

These tests will leverage Airbyte's integration test credentials.

Check job output.
Connector CI Tests job completed. See logs for details.

@aaronsteers
Copy link
Collaborator

Aaron ("AJ") Steers (aaronsteers) commented Jun 4, 2025

/run-connector-tests

Connector CI Tests Started

These tests will leverage Airbyte's integration test credentials.

Check job output.
Connector CI Tests job completed. See logs for details.

@aaronsteers
Copy link
Collaborator

Above is using new security token for the configs. ☝️

Includes updates for disable_typing_config, raw_override_typing_config and typing_config scenarios.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants