Skip to content

Conversation

mwbayley
Copy link
Contributor

@mwbayley mwbayley commented Sep 8, 2025

Add field type mapping and associated tests based on the testing framework to the postgres source on the bulk CDK.

Can this PR be safely reverted and rolled back?

  • YES πŸ’š
  • NO ❌

Copy link
Contributor

github-actions bot commented Sep 8, 2025

πŸ‘‹ Greetings, Airbyte Team Member!

Here are some helpful tips and reminders for your convenience.

Helpful Resources

PR Slash Commands

Airbyte Maintainers (that's you!) can execute the following slash commands on your PR:

  • /format-fix - Fixes most formatting issues.
  • /bump-version - Bumps connector versions.
    • You can specify a custom changelog by passing changelog. Example: /bump-version changelog="My cool update"
    • Leaving the changelog arg blank will auto-populate the changelog from the PR title.
  • /run-cat-tests - Runs legacy CAT tests (Connector Acceptance Tests)
  • /build-connector-images - Builds and publishes a pre-release docker image for the modified connector(s).
  • /poe connector source-example lock - Run the Poe lock task on the source-example connector, committing the results back to the branch.
  • /poe source example lock - Alias for /poe connector source-example lock.
  • /poe source example use-cdk-branch my/branch - Pin the source-example CDK reference to the branch name specified.
  • /poe source example use-cdk-latest - Update the source-example CDK dependency to the latest available version.

πŸ“ Edit this welcome message.

Copy link
Contributor

github-actions bot commented Sep 8, 2025

destination-clickhouse Connector Test Results

0 tests   0 βœ…β€ƒβ€ƒ0s ⏱️
0 suites  0 πŸ’€
0 files    0 ❌

Results for commit a255a19.

♻️ This comment has been updated with latest results.

Copy link
Contributor

github-actions bot commented Sep 8, 2025

destination-s3 Connector Test Results

0 tests   0 βœ…β€ƒβ€ƒ0s ⏱️
0 suites  0 πŸ’€
0 files    0 ❌

Results for commit a255a19.

♻️ This comment has been updated with latest results.

Copy link
Contributor

github-actions bot commented Sep 8, 2025

destination-customer-io Connector Test Results

0 tests   0 βœ…β€ƒβ€ƒ0s ⏱️
0 suites  0 πŸ’€
0 files    0 ❌

Results for commit a255a19.

♻️ This comment has been updated with latest results.

Copy link
Contributor

github-actions bot commented Sep 8, 2025

destination-dev-null Connector Test Results

0 tests   0 βœ…β€ƒβ€ƒ0s ⏱️
0 suites  0 πŸ’€
0 files    0 ❌

Results for commit a255a19.

♻️ This comment has been updated with latest results.

Copy link
Contributor

github-actions bot commented Sep 8, 2025

destination-mssql Connector Test Results

0 tests   0 βœ…β€ƒβ€ƒ0s ⏱️
0 suites  0 πŸ’€
0 files    0 ❌

Results for commit a255a19.

♻️ This comment has been updated with latest results.

Copy link
Contributor

github-actions bot commented Sep 8, 2025

source-mysql Connector Test Results

0 tests   0 βœ…β€ƒβ€ƒ0s ⏱️
0 suites  0 πŸ’€
0 files    0 ❌

Results for commit a255a19.

♻️ This comment has been updated with latest results.

Copy link
Contributor

github-actions bot commented Sep 8, 2025

source-postgres Connector Test Results

0 tests   0 βœ…β€ƒβ€ƒ0s ⏱️
0 suites  0 πŸ’€
0 files    0 ❌

Results for commit a255a19.

♻️ This comment has been updated with latest results.

@mwbayley mwbayley force-pushed the mbayley/source-postgres/field-type-tests branch from 40cf841 to de8cc06 Compare September 11, 2025 22:47
@mwbayley mwbayley marked this pull request as ready for review September 13, 2025 01:03
@mwbayley mwbayley requested review from a team as code owners September 13, 2025 01:03
}
}

class InfJsonEncoder<T>(private val base: JsonEncoder<T>) : JsonEncoder<InfWrapper<T>> {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since we're using the type of JsonEncoder as a hint for protobuf encoding, this may create a difficulty because the inner type is erased.
Maybe reified can help here.
I'm noting this as a difficulty to myself.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

inline and reified are new to me. Good idea.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just played around with this a bit. Maybe we can pair on it in person later this week.

JDBCType.CHAR,
JDBCType.VARCHAR,
// JDBCType.TINYTEXT,
// JDBCType.TEXT,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

all these don't exist?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, forgot to clean this up. Commit incoming.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

wait, I have this removed locally and the sha on the tip commit matches.... something weird is going on

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, the commented ones are all in red here.... duh. Yes, these JDBC types are not present in column metadata for Postgres.

object PostgresSource {
@JvmStatic
fun main(args: Array<String>) {
// TODO: Remove before merging
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oh 🀦

@mwbayley mwbayley merged this pull request into source-postgres/bulk-cdk Sep 16, 2025
30 of 50 checks passed
@mwbayley mwbayley deleted the mbayley/source-postgres/field-type-tests branch September 16, 2025 00:06
mwbayley added a commit that referenced this pull request Sep 16, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants