Releases: google/dwh-migration-tools
Releases · google/dwh-migration-tools
v1.1.0
What's Changed
- [b/385126043] close http client by @vladislav-sidorovich in #683
- remove ignore section for dependabot by @vladislav-sidorovich in #684
- Remove DummyHandle, don't extend AbstractHandle in JdbcHandle by @misolt in #682
- [b/387249832] Move Redshift URL utilities to a Util class by @misolt in #681
- Reduce dependabot configurations by @vladislav-sidorovich in #685
- Update woodstox-core library to version 5.4.0 by @kaxuna in #694
- [b/377651080] rewrite Cloudera services task test to wiremock by @vladislav-sidorovich in #678
- [b/377651080] rewrite cluster test with wiremock by @vladislav-sidorovich in #677
- [fix] fix code style by @vladislav-sidorovich in #695
- Bump org.apache.avro:avro from 1.11.4 to 1.12.0 by @dependabot in #690
- Bump org.xerial:sqlite-jdbc from 3.47.0.0 to 3.48.0.0 by @dependabot in #692
- [OPS] Update dependabot config by @vladislav-sidorovich in #700
- Bump net.snowflake:snowflake-jdbc from 3.20.0 to 3.21.0 by @dependabot in #705
- Bump jackson from 2.18.1 to 2.18.2 by @dependabot in #702
- [b/385081594]: Add more http tests for Cloudera tasks by @zaldis in #688
- [b/391318611] Align Cloudera API by @zaldis in #708
- [381225149] upgraded jetty-server to 12.0.12 by @kaxuna in #709
- Chore: Add exception cause into TimeSeriesException by @zaldis in #711
- 381225149 removed jetty from hive module by @kaxuna in #710
- Bump com.amazon.redshift:redshift-jdbc42 from 2.1.0.30 to 2.1.0.32 by @dependabot in #704
- Bump commons-io:commons-io from 2.17.0 to 2.18.0 by @dependabot in #706
- Bump org.postgresql:postgresql from 42.7.4 to 42.7.5 by @dependabot in #716
- Bump ch.qos.logback:logback-classic from 1.3.14 to 1.3.15 by @dependabot in #714
- Bump com.google.cloud:google-cloud-nio from 0.127.26 to 0.127.30 by @dependabot in #715
- Bump org.apache.commons:commons-csv from 1.12.0 to 1.13.0 by @dependabot in #717
- Bump com.github.ben-manes.versions from 0.51.0 to 0.52.0 by @dependabot in #718
- [b/392580006] fix: Missing views in SF Lite extraction by @misolt in #712
- Update code owners by @vladislav-sidorovich in #697
- Bump net.snowflake:snowflake-jdbc from 3.21.0 to 3.22.0 by @dependabot in #721
- Bump ch.qos.reload4j:reload4j from 1.2.25 to 1.2.26 by @dependabot in #720
- return ignored dependencies to dependabot config by @vladislav-sidorovich in #724
- Bump org.apache.httpcomponents.client5:httpclient5 from 5.3.1 to 5.4.2 by @dependabot in #725
- Bump joda-time:joda-time from 2.13.0 to 2.13.1 by @dependabot in #726
- Bump org.checkerframework:checker-qual from 3.48.2 to 3.49.0 by @dependabot in #727
- [ops] Schedule dependabot weekly by @vladislav-sidorovich in #728
- [b/394256261] [Dumper] Publish test coverage by @tonyxiaowei in #729
- [b/360552594] Remove redundant logger frameworks and use single version of logback by @vladislav-sidorovich in #730
- [b/360548709] Update hadoop version for hive312 runtime by @vladislav-sidorovich in #731
- Feat [b/382469906]: Add a task to dump Cloudera YARN applications by @zaldis in #713
- Bump org.xerial:sqlite-jdbc from 3.48.0.0 to 3.49.0.0 by @dependabot in #732
- Remove unused files by @vladislav-sidorovich in #734
- [b/394561079] better error message for ranger connector by @lszydelko-google in #733
- Fix: Filter by YARN application type which contains space characters by @zaldis in #735
- Bump com.google.cloud:google-cloud-bigquery from 2.42.0 to 2.48.0 by @dependabot in #739
- Bump org.yaml:snakeyaml from 2.3 to 2.4 by @dependabot in #742
- [b/384669047] Add new connector to extract Redshift Serverless metrics by @kpiotrowski in #743
- Add missing license links by @kpiotrowski in #744
- DBSync Initial Implemetation by @samdgupi in #737
- Fix: Collect cluster name with YARN application types by @zaldis in #747
- [b/390590500] Airflow connector by @vladislav-sidorovich in #745
- [b394751887] add compilerworks_metadata.yaml to Ranger output zip by @lszydelko-google in #736
- [b/398815530] Add start and end dates to snowflake-lite output by @misolt in #748
- Add gitignore files in dbsync by @misolt in #756
- [OPS] fix code style by @vladislav-sidorovich in #757
- [b/384669047] Add missing timestamp to time series file by @kpiotrowski in #759
- [OPS] simplify git ignore files by @vladislav-sidorovich in #758
- [OPS] Update README.md by @vladislav-sidorovich in #760
- Add sizeIfKnown for GCS source and also fix some other issues by @samdgupi in #750
- [b/377253441] Add warehouse utilization info to snowflake-lite by @misolt in #761
- [b/390590500] url param added to Airflow connector by @vladislav-sidorovich in #762
- [b/390590500] add additional table to airflow connector by @vladislav-sidorovich in #763
- Update source file headers by @misolt in #764
- [b/399850453] Don't use the Snowflake format from SPI by @misolt in #767
- Add compile section to README.md by @vladislav-sidorovich in #769
- [b/390590500] use info log level for not found drivers for Airflow connector by @vladislav-sidorovich in #765
- Add licenseReport task to dbsync by @kpiotrowski in #770
- Add sdkman recommendation README.md by @vladislav-sidorovich in #772
- Typo fix README.md by @vladislav-sidorovich in #773
- Add task to build the license report for the whole dwh-migration-tools repository by @kpiotrowski in #774
- Feat: Dump Oozie applications by default by @zaldis in #775
- [b/400931733] always add trailing slash by @vladislav-sidorovich in #776
- [b/377253441] Add extraction of WAREHOUSE_EVENTS_HISTORY by @misolt in #777
- [b/400931410] Add missing license links for dbsync dependencies by @kpiotrowski in #779
- Update for Java 8 CONTRIBUTING.md by @vladislav-sidorovich in #778
- Bump com.google.cloud:google-cloud-storage from 2.41.0 to 2.49.0 by @dependabot in #768
- [b/394577286] Replace the warehouse events task with a lite version by @misolt in #783
- Bump it.unimi.dsi:fastutil from 8.5.13 to 8.5.15 by @dependabot in #752
- [b/402436948] Create a lite version of WarehouseMeteringHistory by @misolt in #784
- [OPS] Mark some Cloudera tasks opti...
v1.0.66
What's Changed
- Bump aws-java-sdk from 1.12.773 to 1.12.778 by @dependabot in #648
- Adding new columns for segment-stats.sql by @sayuzbas-google in #646
- Update dependabot config for spring framework by @vladislav-sidorovich in #649
- Update spring to the latest version by @vladislav-sidorovich in #650
- [b/367261072] Cloudera API integration - main data by @vladislav-sidorovich in #621
- Enable Kerberos keytab auth for HdfsConnector by @sten1ee in #635
- [improve] add unit test for services endpoint in Cloudera manager API by @vladislav-sidorovich in #651
- [b/371118648] When using DBMS_METADATA.GET_XML exclude system-generat… by @ikhavkin-google in #574
- [b/375191371] Add an option to specify the uri scheme for ranger and … by @kajgol in #652
- Kerberos auth for HdfsConnector: Applying mods suggested by Radek by @sten1ee in #654
- [b/375191371] Fix flaky tls test by @kajgol in #662
- [b/381361432] hdfs connector may use kinit-cached credentials for auth by @sten1ee in #661
- [b/377657866] Remove ClouderaMetadataConnector from the code by @vladislav-sidorovich in #664
- [b/381400224] Rename ClouderaMetadata connector to HadoopMetadata by @rocgranada in #665
- [b360554196] remove log4j implementation by @vladislav-sidorovich in #669
- [/b376629469] Cloudera timeseries for CPU usage by @zaldis in #668
- Adding 2 new columns for SegmentStatistics table by @sayuzbas-google in #671
- [b/376629469] Upgrade avro library to version 1.11.4 by @kaxuna in #667
- [b/376629469] Add a task to fetch RAM usage per host by @zaldis in #672
- Rename classes -Dto -> -DTO by @zaldis in #674
- [b/380000872] Dump hosts components by @vladislav-sidorovich in #675
- [b/380000872] Add hostId to jsonl by @vladislav-sidorovich in #676
- [b/356461225] Add progress during HDFS dumping by @sten1ee in #673
- Remove writeRecordsCsv, refactor the usage metrics task for Redshift by @misolt in #660
- [b/382467945] Move parts of JdbcHandle logic to a util class by @misolt in #679
New Contributors
- @kajgol made their first contribution in #652
- @zaldis made their first contribution in #668
- @kaxuna made their first contribution in #667
Full Changelog: v1.0.65...v1.0.66
v1.0.65
What's Changed
- Bump net.snowflake:snowflake-jdbc from 3.19.0 to 3.19.1 by @dependabot in #624
- Bump nl.littlerobots.version-catalog-update from 0.8.4 to 0.8.5 by @dependabot in #623
- Class design cleanup for Snowflake Logs by @misolt in #590
- [improve] Configure dependabot by @vladislav-sidorovich in #629
- Bump com.google.cloud:google-cloud-nio from 0.127.25 to 0.127.26 by @dependabot in #630
- Reduce code duplication between metadata and lite connectors, allow non-enum connector properties by @misolt in #602
- Remove another connector's configuration from snowflake-lite by @misolt in #634
- [b/377253441] Add a task to get aggregate events in snowflake-lite by @misolt in #633
- [b/376638949] Don't use INFORMATION_SCHEMA in snowflake-lite by @misolt in #636
- Bump jackson from 2.18.0 to 2.18.1 by @dependabot in #637
- Bump net.snowflake:snowflake-jdbc from 3.19.1 to 3.20.0 by @dependabot in #641
- Bump org.xerial:sqlite-jdbc from 3.46.1.3 to 3.47.0.0 by @dependabot in #639
- [improve] configure dependabot for hikari and JDK 8 by @vladislav-sidorovich in #642
- Bump org.checkerframework:checker-qual from 3.48.1 to 3.48.2 by @dependabot in #645
Full Changelog: v1.0.64...v1.0.65
v1.0.64
What's Changed
- Bump joda-time:joda-time from 2.12.7 to 2.13.0 by @dependabot in #597
- Bump org.checkerframework:checker-qual from 3.48.0 to 3.48.1 by @dependabot in #601
- Bump org.postgresql:postgresql from 42.7.3 to 42.7.4 by @dependabot in #599
- Fix bugs in ZonedIntervalIterableTest by @misolt in #598
- [b/370703701] Add more descriptive connection error message for Oracle by @kpiotrowski in #609
- Bump net.snowflake:snowflake-jdbc from 3.18.0 to 3.19.0 by @dependabot in #613
- [b/375140671] Create a new connector for SF TCO Lite by @misolt in #620
- [b/368222012] Report AccessControlException(s) in HdfsExtractionTask separately by @sten1ee in #619
- fix compile warnings by @vladislav-sidorovich in #622
- [b/374026408] Fix Hive extraction for Iceberg raw tables by @rocgranada in #614
- [b/368220129] HdfsContentSummaryTask should not fail hard on AccessControlException(s) by @sten1ee in #615
New Contributors
- @rocgranada made their first contribution in #614
Full Changelog: v1.0.63...v1.0.64
v1.0.63
What's Changed
- Remove redundant arguments from AutoService in Snowflake by @misolt in #576
- Bump org.checkerframework:checker-qual from 3.46.0 to 3.48.0 by @dependabot in #569
- Bump org.apache.commons:commons-csv from 1.11.0 to 1.12.0 by @dependabot in #578
- Output first and last query log entries for Teradata with a new task by @sayuzbas-google in #571
- Bump com.diffplug.spotless:spotless-plugin-gradle from 6.4.1 to 6.25.0 by @dependabot in #568
- Rollback spotless plugin to 6.4.1 because of java 8 support by @vladislav-sidorovich in #584
- fix spotless codestyle by @vladislav-sidorovich in #585
- Clean up class design for Snowflake Metadata by @misolt in #586
- Bump jackson from 2.17.2 to 2.18.0 by @dependabot in #588
- Bump org.apache.commons:commons-compress from 1.27.0 to 1.27.1 by @dependabot in #589
- Bump com.google.cloud:google-cloud-nio from 0.127.21 to 0.127.25 by @dependabot in #587
- [issue 114] improve error messages for JDBC driver by @vladislav-sidorovich in #579
- Bump ch.qos.reload4j:reload4j from 1.2.22 to 1.2.25 by @dependabot in #591
- Bump aws-java-sdk from 1.12.767 to 1.12.773 by @dependabot in #593
- [b/373568138] Change timestamp format used in zip entry files to work on windows by @kpiotrowski in #594
- Bump org.xerial:sqlite-jdbc from 3.46.0.1 to 3.46.1.3 by @dependabot in #596
New Contributors
- @sayuzbas-google made their first contribution in #571
Full Changelog: v1.0.62...v1.0.63
v1.0.62
What's Changed
- More precise .gitignore rules by @misolt in #575
- [b/368572924] Fix: missing data of public synonym objects by @misolt in #558
Full Changelog: v1.0.61...v1.0.62
v1.0.61
What's Changed
- [b/359499526] Dump raw Ranger API entries to output jsonl files. by @aleofreddi in #518
- [b/364818924] Remove all 'owner' filters from stats SQLs by @misolt in #553
- Provide a more descriptive error message when driver JAR does not exist by @misolt in #555
- Extract Cloudera metaconnector in-dump paths to SPI. by @damian-compilerworks in #554
- [b/355196176] Implement fail fast strategy for Teradata logs tables by @vladislav-sidorovich in #557
- [b/370428253] Rename 'Dbid' to 'DbId' by @misolt in #563
- Replace boolean flag with enum by @vladislav-sidorovich in #562
- [b/369075732] Add Teradata tables validation for additional command line options by @vladislav-sidorovich in #561
- Bump org.yaml:snakeyaml from 2.2 to 2.3 by @dependabot in #551
- update gradle lockfiles to fix snakeyaml by @vladislav-sidorovich in #564
- Update compliance information with license links by @misolt in #570
- Move license overrides to a util class by @misolt in #572
New Contributors
- @vladislav-sidorovich made their first contribution in #557
Full Changelog: v1.0.60...v1.0.61
v1.0.60
What's Changed
- Remove dwh-migration-client by @dawidxc in #521
- [b/361574176] Extract additional software packages by @dawidxc in #523
- [b/361566488] Extract user local libraries by @dawidxc in #525
- [b/353201454] Extract HDFS report by @sten1ee in #524
- Group single-line scripts in a single text file by @dawidxc in #526
- [b/356467392] Remove MIGRATION_METADATA flag by @dawidxc in #530
- [b/361568886] Extract hive properties by @dawidxc in #527
- [b/361572606] Fix the command for extracting spark version by @dawidxc in #528
- [b/353664125] Extract statuses of services by @dawidxc in #529
- [b/353201454] Ignore slow nodes extraction failure by @dawidxc in #535
- [b/362196485] Add db-links, db-link-sources SQLs by @misolt in #539
- [b/362205662] Extend Oracle source-conn-latest sql with additional columns by @kpiotrowski in #537
- [b/360840839] Extract Hadoop native libraries by @dawidxc in #532
- [b/361577312] Extract hardware, storage and HDFS details by @dawidxc in #533
- [b/357517011] Extract filenames from local filesystem by @dawidxc in #531
- [b/352490642] Add cloudera metaconnector by @dawidxc in #538
- Rename hadoop connector to cloudera-metadata by @dawidxc in #542
- Rename hdfs-permissions connector to hdfs by @dawidxc in #543
- [b/352490642] Add HDFS to Cloudera metaconnector by @dawidxc in #544
- [b/352490642] Add child connectors to metadata yaml by @dawidxc in #545
- [b/361371764] Add Java version requirement to README by @dawidxc in #547
- Make HDFS tasks optional by @dawidxc in #548
- Cleanup loggers in HDFS connector by @dawidxc in #549
- [b/364035944] Prevent a NullPointerException where a result's toString has returned null by @misolt in #552
- ParallelTaskGroup fixes by @misolt in #541
Full Changelog: v1.0.59...v1.0.60
v1.0.59
What's Changed
- [b/350521386] Add correct Oracle column name for ALL_SEQUENCES.SEQUENCE_OWNER by @ikhavkin-google in #508
- [b/359866106] Add SQLs to discover Oracle exadata and system configuration by @kpiotrowski in #520
- [b/361012673] Extract historical metrics from Oracle cdb_hist_seg_stat view by @kpiotrowski in #522
- [b/342096412] Detect Java version in the main script by @dawidxc in #434
Full Changelog: v1.0.58...v1.0.59
v1.0.58
What's Changed
- [b/353960111] Extract additional software versions by @dawidxc in #512
- [b/353527405] Add gradle version lock files by @Hackerpilot in #503
- [b/357128377] Update Java dependencies via ./gradlew versionCatalogApplyUpdates by @ajwelch4 in #504
Full Changelog: v1.0.57...v1.0.58