Dataflow Templates 2022-09-13-00_RC00

Release Week of 2022-09-12

New Templates

• JDBC to BigQuery Flex template added. Same functionality as the existing classic template, but the new template also supports BigQuery Storage Write API.

Improvements

• [BigQueryToParquet Template] Now supports row restrictions of the BigQuery Storage Read API.
• [DataStreamToSpanner Template] Better handling of incoming change events based on the schema mappings in the session file.
• Prevent WindowedFilenamePolicy from changing bucket names. WindowedFileNamePolicy replaces date patterns in output directory for dynamically changing the output location based on the window end time. This leads to errors when bucket name contains a date pattern. This change makes sure that the bucket name is always unchanged.

Bug Fixes

• Temporary workarounds for SpannerIO: LocalSpannerIO and LocalReadSpannerSchema classes now cast JSONB records as VARCHAR.

Minor changes

• Removed the explicit Bigtable client version from the pom.xml files to use the transitive version from Beam. This is to keep the client library up-to-date and match the version expected by Beam.
• Updated maven-dependency-plugin version.

Contributors

@pranavbhandari24
@oleg-semenov
@Deep1998
@shubhamswe
@bvolpato

Full Changelog: 2022-09-05-00_RC00...2022-09-13-00_RC00