v1.4.1

What's Changed

• Clean unused csv in SF Lite by @sayuzbas-google in #888
• Revert an optimization in GcsBytesource (and some automatic formatting changes) by @tonyxiaowei in #903
• Feat: b/400932733 Add date range for cloudera manager tasks by @zaldis in #889
• [b/420369667] fix NPE for Oozie end date by @vladislav-sidorovich in #910

Full Changelog: v1.4.0...v1.4.1

v1.4.0

What's Changed

• [b/418155474] Drop the --database flag from snowflake-lite by @misolt in #893
• [b/404190134] add permissions-migration by @kajgol in #890
• [b/404190134] add permissions-migration to the main build by @kajgol in #902

Full Changelog: v1.3.1...v1.4.0

v1.3.0

What's Changed

• Update README.md by @tonyxiaowei in #879
• [b/413037987] Add more fields to Yarn applications response by @vladislav-sidorovich in #880

Full Changelog: v1.2.2...v1.3.0

v1.2.2

What's Changed

• [OPS] Add some fun to a developer's life by @vladislav-sidorovich in #864
• [OPS] Use immutable list in TaskGroup by @vladislav-sidorovich in #865
• gradle google profile by @kaxuna in #842
• Bump commons-io:commons-io from 2.18.0 to 2.19.0 by @dependabot in #867
• Bump org.slf4j:jcl-over-slf4j from 2.0.16 to 2.0.17 by @dependabot in #868
• Bump com.google.cloud:google-cloud-storage from 2.49.0 to 2.50.0 by @dependabot in #869
• Allow the tool to reuse existing progress resuming from a crash. by @tonyxiaowei in #863
• Allow overriding snowflake authenticator parameter by @kpiotrowski in #874

Full Changelog: v1.2.1...v1.2.2

Gcsync private review

The binary for the gcsync tool for syncing files into GCS from your local file system.

Note: This tool is still in development. Use it at your own risk.

Refer to https://github.com/google/dwh-migration-tools/blob/main/dbsync/README.md for more details on how to use the tool.

v1.2.1

What's Changed

• [OPS] add source code to distribution by @vladislav-sidorovich in #862

Full Changelog: v1.2.0...v1.2.1

v1.1.1

What's Changed

• Bump software.amazon.awssdk:s3 from 2.27.2 to 2.31.6 by @dependabot in #810
• Bump com.google.cloud:google-cloud-run from 0.57.0 to 0.60.0 by @dependabot in #809
• Bump jackson from 2.18.2 to 2.18.3 by @dependabot in #788
• Bump org.apache.avro:avro from 1.11.3 to 1.12.0 by @dependabot in #755
• [OPS] add plugin to log test execution time into console by @vladislav-sidorovich in #811
• [Snowflake Connector] Add test to verify JDBC tasks produce expected SQL. by @paolomorandini in #807
• [b/390590500] Airflow dates params by @vladislav-sidorovich in #766
• Fix LOG -> logger typo by @sten1ee in #817
• Revert "Bump org.apache.avro:avro from 1.11.3 to 1.12.0" by @misolt in #815
• Ops/deps config by @vladislav-sidorovich in #818
• Revert "[b/394312229] Dump Snowflake external table metadata (#746)" by @misolt in #813

Full Changelog: v1.1.0...v1.1.1

v1.1.0

What's Changed

• [b/385126043] close http client by @vladislav-sidorovich in #683
• remove ignore section for dependabot by @vladislav-sidorovich in #684
• Remove DummyHandle, don't extend AbstractHandle in JdbcHandle by @misolt in #682
• [b/387249832] Move Redshift URL utilities to a Util class by @misolt in #681
• Reduce dependabot configurations by @vladislav-sidorovich in #685
• Update woodstox-core library to version 5.4.0 by @kaxuna in #694
• [b/377651080] rewrite Cloudera services task test to wiremock by @vladislav-sidorovich in #678
• [b/377651080] rewrite cluster test with wiremock by @vladislav-sidorovich in #677
• [fix] fix code style by @vladislav-sidorovich in #695
• Bump org.apache.avro:avro from 1.11.4 to 1.12.0 by @dependabot in #690
• Bump org.xerial:sqlite-jdbc from 3.47.0.0 to 3.48.0.0 by @dependabot in #692
• [OPS] Update dependabot config by @vladislav-sidorovich in #700
• Bump net.snowflake:snowflake-jdbc from 3.20.0 to 3.21.0 by @dependabot in #705
• Bump jackson from 2.18.1 to 2.18.2 by @dependabot in #702
• [b/385081594]: Add more http tests for Cloudera tasks by @zaldis in #688
• [b/391318611] Align Cloudera API by @zaldis in #708
• [381225149] upgraded jetty-server to 12.0.12 by @kaxuna in #709
• Chore: Add exception cause into TimeSeriesException by @zaldis in #711
• 381225149 removed jetty from hive module by @kaxuna in #710
• Bump com.amazon.redshift:redshift-jdbc42 from 2.1.0.30 to 2.1.0.32 by @dependabot in #704
• Bump commons-io:commons-io from 2.17.0 to 2.18.0 by @dependabot in #706
• Bump org.postgresql:postgresql from 42.7.4 to 42.7.5 by @dependabot in #716
• Bump ch.qos.logback:logback-classic from 1.3.14 to 1.3.15 by @dependabot in #714
• Bump com.google.cloud:google-cloud-nio from 0.127.26 to 0.127.30 by @dependabot in #715
• Bump org.apache.commons:commons-csv from 1.12.0 to 1.13.0 by @dependabot in #717
• Bump com.github.ben-manes.versions from 0.51.0 to 0.52.0 by @dependabot in #718
• [b/392580006] fix: Missing views in SF Lite extraction by @misolt in #712
• Update code owners by @vladislav-sidorovich in #697
• Bump net.snowflake:snowflake-jdbc from 3.21.0 to 3.22.0 by @dependabot in #721
• Bump ch.qos.reload4j:reload4j from 1.2.25 to 1.2.26 by @dependabot in #720
• return ignored dependencies to dependabot config by @vladislav-sidorovich in #724
• Bump org.apache.httpcomponents.client5:httpclient5 from 5.3.1 to 5.4.2 by @dependabot in #725
• Bump joda-time:joda-time from 2.13.0 to 2.13.1 by @dependabot in #726
• Bump org.checkerframework:checker-qual from 3.48.2 to 3.49.0 by @dependabot in #727
• [ops] Schedule dependabot weekly by @vladislav-sidorovich in #728
• [b/394256261] [Dumper] Publish test coverage by @tonyxiaowei in #729
• [b/360552594] Remove redundant logger frameworks and use single version of logback by @vladislav-sidorovich in #730
• [b/360548709] Update hadoop version for hive312 runtime by @vladislav-sidorovich in #731
• Feat [b/382469906]: Add a task to dump Cloudera YARN applications by @zaldis in #713
• Bump org.xerial:sqlite-jdbc from 3.48.0.0 to 3.49.0.0 by @dependabot in #732
• Remove unused files by @vladislav-sidorovich in #734
• [b/394561079] better error message for ranger connector by @lszydelko-google in #733
• Fix: Filter by YARN application type which contains space characters by @zaldis in #735
• Bump com.google.cloud:google-cloud-bigquery from 2.42.0 to 2.48.0 by @dependabot in #739
• Bump org.yaml:snakeyaml from 2.3 to 2.4 by @dependabot in #742
• [b/384669047] Add new connector to extract Redshift Serverless metrics by @kpiotrowski in #743
• Add missing license links by @kpiotrowski in #744
• DBSync Initial Implemetation by @samdgupi in #737
• Fix: Collect cluster name with YARN application types by @zaldis in #747
• [b/390590500] Airflow connector by @vladislav-sidorovich in #745
• [b394751887] add compilerworks_metadata.yaml to Ranger output zip by @lszydelko-google in #736
• [b/398815530] Add start and end dates to snowflake-lite output by @misolt in #748
• Add gitignore files in dbsync by @misolt in #756
• [OPS] fix code style by @vladislav-sidorovich in #757
• [b/384669047] Add missing timestamp to time series file by @kpiotrowski in #759
• [OPS] simplify git ignore files by @vladislav-sidorovich in #758
• [OPS] Update README.md by @vladislav-sidorovich in #760
• Add sizeIfKnown for GCS source and also fix some other issues by @samdgupi in #750
• [b/377253441] Add warehouse utilization info to snowflake-lite by @misolt in #761
• [b/390590500] url param added to Airflow connector by @vladislav-sidorovich in #762
• [b/390590500] add additional table to airflow connector by @vladislav-sidorovich in #763
• Update source file headers by @misolt in #764
• [b/399850453] Don't use the Snowflake format from SPI by @misolt in #767
• Add compile section to README.md by @vladislav-sidorovich in #769
• [b/390590500] use info log level for not found drivers for Airflow connector by @vladislav-sidorovich in #765
• Add licenseReport task to dbsync by @kpiotrowski in #770
• Add sdkman recommendation README.md by @vladislav-sidorovich in #772
• Typo fix README.md by @vladislav-sidorovich in #773
• Add task to build the license report for the whole dwh-migration-tools repository by @kpiotrowski in #774
• Feat: Dump Oozie applications by default by @zaldis in #775
• [b/400931733] always add trailing slash by @vladislav-sidorovich in #776
• [b/377253441] Add extraction of WAREHOUSE_EVENTS_HISTORY by @misolt in #777
• [b/400931410] Add missing license links for dbsync dependencies by @kpiotrowski in #779
• Update for Java 8 CONTRIBUTING.md by @vladislav-sidorovich in #778
• Bump com.google.cloud:google-cloud-storage from 2.41.0 to 2.49.0 by @dependabot in #768
• [b/394577286] Replace the warehouse events task with a lite version by @misolt in #783
• Bump it.unimi.dsi:fastutil from 8.5.13 to 8.5.15 by @dependabot in #752
• [b/402436948] Create a lite version of WarehouseMeteringHistory by @misolt in #784
• [OPS] Mark some Cloudera tasks opti...

v1.0.66

What's Changed

• Bump aws-java-sdk from 1.12.773 to 1.12.778 by @dependabot in #648
• Adding new columns for segment-stats.sql by @sayuzbas-google in #646
• Update dependabot config for spring framework by @vladislav-sidorovich in #649
• Update spring to the latest version by @vladislav-sidorovich in #650
• [b/367261072] Cloudera API integration - main data by @vladislav-sidorovich in #621
• Enable Kerberos keytab auth for HdfsConnector by @sten1ee in #635
• [improve] add unit test for services endpoint in Cloudera manager API by @vladislav-sidorovich in #651
• [b/371118648] When using DBMS_METADATA.GET_XML exclude system-generat… by @ikhavkin-google in #574
• [b/375191371] Add an option to specify the uri scheme for ranger and … by @kajgol in #652
• Kerberos auth for HdfsConnector: Applying mods suggested by Radek by @sten1ee in #654
• [b/375191371] Fix flaky tls test by @kajgol in #662
• [b/381361432] hdfs connector may use kinit-cached credentials for auth by @sten1ee in #661
• [b/377657866] Remove ClouderaMetadataConnector from the code by @vladislav-sidorovich in #664
• [b/381400224] Rename ClouderaMetadata connector to HadoopMetadata by @rocgranada in #665
• [b360554196] remove log4j implementation by @vladislav-sidorovich in #669
• [/b376629469] Cloudera timeseries for CPU usage by @zaldis in #668
• Adding 2 new columns for SegmentStatistics table by @sayuzbas-google in #671
• [b/376629469] Upgrade avro library to version 1.11.4 by @kaxuna in #667
• [b/376629469] Add a task to fetch RAM usage per host by @zaldis in #672
• Rename classes -Dto -> -DTO by @zaldis in #674
• [b/380000872] Dump hosts components by @vladislav-sidorovich in #675
• [b/380000872] Add hostId to jsonl by @vladislav-sidorovich in #676
• [b/356461225] Add progress during HDFS dumping by @sten1ee in #673
• Remove writeRecordsCsv, refactor the usage metrics task for Redshift by @misolt in #660
• [b/382467945] Move parts of JdbcHandle logic to a util class by @misolt in #679

New Contributors

• @kajgol made their first contribution in #652
• @zaldis made their first contribution in #668
• @kaxuna made their first contribution in #667

Full Changelog: v1.0.65...v1.0.66

v1.0.65

What's Changed

• Bump net.snowflake:snowflake-jdbc from 3.19.0 to 3.19.1 by @dependabot in #624
• Bump nl.littlerobots.version-catalog-update from 0.8.4 to 0.8.5 by @dependabot in #623
• Class design cleanup for Snowflake Logs by @misolt in #590
• [improve] Configure dependabot by @vladislav-sidorovich in #629
• Bump com.google.cloud:google-cloud-nio from 0.127.25 to 0.127.26 by @dependabot in #630
• Reduce code duplication between metadata and lite connectors, allow non-enum connector properties by @misolt in #602
• Remove another connector's configuration from snowflake-lite by @misolt in #634
• [b/377253441] Add a task to get aggregate events in snowflake-lite by @misolt in #633
• [b/376638949] Don't use INFORMATION_SCHEMA in snowflake-lite by @misolt in #636
• Bump jackson from 2.18.0 to 2.18.1 by @dependabot in #637
• Bump net.snowflake:snowflake-jdbc from 3.19.1 to 3.20.0 by @dependabot in #641
• Bump org.xerial:sqlite-jdbc from 3.46.1.3 to 3.47.0.0 by @dependabot in #639
• [improve] configure dependabot for hikari and JDK 8 by @vladislav-sidorovich in #642
• Bump org.checkerframework:checker-qual from 3.48.1 to 3.48.2 by @dependabot in #645

Full Changelog: v1.0.64...v1.0.65