Skip to content

dbt 1.9 migration#296

Merged
99Lys merged 10 commits into
dremio:mainfrom
99Lys:DX-99958-dbt-1.9-migration
Jul 4, 2025
Merged

dbt 1.9 migration#296
99Lys merged 10 commits into
dremio:mainfrom
99Lys:DX-99958-dbt-1.9-migration

Conversation

@99Lys
Copy link
Copy Markdown
Contributor

@99Lys 99Lys commented May 28, 2025

Summary

Updating dbt-dremio to match dbt-core v1.9.0.

Description

Following Adapter Maintainers: Upgrading to dbt-core v1.9.0:

  • Snapshot improvements
    • Snapshots: Enable setting a datetime value for dbt_valid_to for current records instead of NULL
    • Snapshots: Enable hard_deletes config to add a metadata column if a record has been deleted (automatically inherited from dbt-adapters)
    • Snapshots: Allow customizing names of metadata fields
    • Snapshots: Enable unique_key as a list (automatically inherited from dbt-adapters)
  • Microbatch incremental strategy → optional, so not included in this PR

Test Results

  • Fixed tests/hooks/test_run_hooks.py
  • Added TestSnapshotDbtValidToCurrentDremio, TestSnapshotColumnNamesDremio, TestSnapshotColumnNamesFromDbtProjectDremio and TestSnapshotInvalidColumnNamesDremio tests for the snapshot improvements

Changelog

  • Added a summary of what this PR accomplishes to CHANGELOG.md

@99Lys 99Lys force-pushed the DX-99958-dbt-1.9-migration branch from 3ba2f3a to 6b9ae60 Compare May 30, 2025 14:24
@99Lys 99Lys force-pushed the DX-99958-dbt-1.9-migration branch from bfa5c79 to 74a629c Compare June 18, 2025 12:11
@99Lys 99Lys closed this Jun 18, 2025
@99Lys 99Lys force-pushed the DX-99958-dbt-1.9-migration branch from 74a629c to 1e3c6c9 Compare June 18, 2025 12:16
@99Lys 99Lys reopened this Jun 18, 2025
@99Lys 99Lys force-pushed the DX-99958-dbt-1.9-migration branch from 3f16685 to d2d4ef5 Compare June 27, 2025 16:28
Comment thread tests/functional/adapter/empty/test_empty.py
Comment thread dbt/include/dremio/macros/builtins/builtins.sql
@99Lys 99Lys marked this pull request as ready for review June 27, 2025 16:51
Comment thread dbt/include/dremio/macros/adapters/snapshot.sql
Comment thread dbt/include/dremio/macros/adapters/snapshot.sql
@howareyouman
Copy link
Copy Markdown

howareyouman commented Jul 1, 2025

@99Lys could you please add the changelog message about the migration? Thanks!

Copy link
Copy Markdown

@howareyouman howareyouman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you please add the test for the new behavior of snapshot macro?
The rest is great.

Comment thread dbt/include/dremio/macros/adapters/snapshot.sql
@99Lys 99Lys force-pushed the DX-99958-dbt-1.9-migration branch from 9080518 to 512c36a Compare July 1, 2025 17:20
Comment thread tests/functional/adapter/materialization/test_snapshots.py Outdated
howareyouman
howareyouman previously approved these changes Jul 3, 2025
Copy link
Copy Markdown

@howareyouman howareyouman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for your PR!
It would be awesome if we could clean TestSnapshotCheckColsDremio and TestSnapshotTimestampDremio test classes.

Comment thread tests/functional/adapter/materialization/test_snapshots.py Outdated
@99Lys 99Lys force-pushed the DX-99958-dbt-1.9-migration branch from a871537 to ce99ea6 Compare July 3, 2025 16:53
@99Lys 99Lys requested a review from howareyouman July 3, 2025 17:02
Copy link
Copy Markdown

@howareyouman howareyouman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm ok having duplication of tests from dbt-adapters, but at the same time, if it's possible to patch the problem there, we should do it.

Comment thread tests/functional/adapter/materialization/test_snapshots.py
@99Lys 99Lys merged commit b2d7513 into dremio:main Jul 4, 2025
19 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

3 participants