Skip to content

v2.3.0 dbt_fivetran_log
Compare
Choose a tag to compare
@fivetran-data-model-bot fivetran-data-model-bot released this 14 Aug 20:34
· 1 commit to main since this release
v2.3.0
348ac40
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

PR #164 includes the following updates:

Schema Data Changes

3 total changes • 3 possible breaking changes

To prevent potential errors from naming and materialization updates, a --full-refresh is required after upgrading.

Data Models Change Type Old New Notes
fivetran_platform__audit_table Model materialization (only for Databricks SQL Warehouse runtimes) table incremental Added incremental model support for Databricks SQL Warehouses using the merge strategy. Previously, incremental support was limited to Databricks All Purpose Clusters via the insert_overwrite strategy. This update extends incremental functionality to SQL Warehouses, enabling more efficient model builds.
fivetran_platform__audit_table New column write_to_table_start_day Changed the column partitioned on from sync_start_day to the new write_to_table_start_day. The previous column could contain null values, which are not ideal for partitioning and may lead to unexpected behavior in incremental models.
fivetran_platform__audit_table Deleted column sync_start_day No longer in use given the above.

Bug Fixes

  • Updated fivetran_platform__mar_table_history to include consumption records do not have an associated active connection and/or destination.
    • As a result this table may now contain additional records that were previously excluded.
    • For more details, see the corresponding DECISIONLOG entry.

dbt Fusion Compatibility Updates

  • Updated package to maintain compatibility with dbt-core versions both before and after v1.10.6, which introduced a breaking change to multi-argument test syntax (e.g., unique_combination_of_columns).
  • Temporarily removed unsupported tests to avoid errors and ensure smoother upgrades across different dbt-core versions. These tests will be reintroduced once a safe migration path is available.
    • Removed all dbt_utils.unique_combination_of_columns tests.
    • Moved loaded_at_field: _fivetran_synced under the config: block in src_fivetran_log.yml.

Under the Hood

  • Updated the is_incremental_compatible() macro to include Databricks SQL Warehouses.
  • Introduced a new macro, is_databricks_all_purpose_cluster(), to distinguish between Databricks All Purpose Clusters and SQL Warehouses.
  • Updated conditions in .github/workflows/auto-release.yml.
  • Added .github/workflows/generate-docs.yml.

Full Changelog: v2.2.2...v2.3.0

Assets 2
Loading