Expose BigQuery schema autodetect in Java SDK #25632
Conversation
|
Assigning reviewers. If you would like to opt out of this review, comment R: @robertwb for label java. Available commands:
The PR bot will only process comments in the main thread (not review comments). |
|
Run Java_GCP_IO_Direct PreCommit |
There was a problem hiding this comment.
Thanks for the contribution. I see this is a long standing issue and this sounds a useful change. Will look into the original JIRA ticket and Python implementation to get more contexts. CC: @reuvenlax @ahmedabu98 for BigQuery SchemaUpdate related changes.
For now I just have one comment. In accordance with the BigQuery option and Python, maybe consider naming this option schemaAutoDetect, BigQueryIO.Write.withSchemaAutoDetect?
| @@ -381,10 +381,6 @@ private JobStatus runLoadJob(JobReference jobRef, JobConfigurationLoad load) | |||
| CreateDisposition createDisposition = CreateDisposition.valueOf(load.getCreateDisposition()); | |||
|
|
|||
| Table existingTable = datasetService.getTable(destination); | |||
| if (schema == null) { | |||
There was a problem hiding this comment.
is it possible to preserve this check?
|
Assigning new set of reviewers because Pr has gone too long without review. If you would like to opt out of this review, comment R: @kennknowles for label java. Available commands:
|
|
I'm not sure I understand this PR. The schema is required if CREATE_IF_NEEDED is set, in which case the schema is needed to create the table. We don't currently require a schema for existing tables. How does autodetect schema work if the table doesn't exist yet? |
|
Reminder, please take a look at this pr: @kennknowles @johnjcasey |
|
Assigning new set of reviewers because Pr has gone too long without review. If you would like to opt out of this review, comment R: @lukecwik for label java. Available commands:
|
|
Assigning new set of reviewers because Pr has gone too long without review. If you would like to opt out of this review, comment R: @kennknowles for label java. Available commands:
|
|
Reminder, please take a look at this pr: @kennknowles @chamikaramj |
|
Assigning new set of reviewers because Pr has gone too long without review. If you would like to opt out of this review, comment R: @Abacn for label java. Available commands:
|
|
Assigning new set of reviewers because Pr has gone too long without review. If you would like to opt out of this review, comment R: @robertwb for label java. Available commands:
|
|
waiting on author |
|
This pull request has been marked as stale due to 60 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull request requires a review, please simply write any comment. If closed, you can revive the PR at any time and @mention a reviewer or discuss it on the [email protected] list. Thank you for your contributions. |
|
This pull request has been closed due to lack of activity. If you think that is incorrect, or the pull request requires review, you can revive the PR at any time. |
Fixes #20515
Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:
addresses #123), if applicable. This will automatically add a link to the pull request in the issue. If you would like the issue to automatically close on merging the pull request, commentfixes #<ISSUE NUMBER>instead.CHANGES.mdwith noteworthy changes.See the Contributor Guide for more tips on how to make review process smoother.
To check the build health, please visit https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md
GitHub Actions Tests Status (on master branch)
See CI.md for more information about GitHub Actions CI.