Skip to content

Releases: sdebruyn/dbt-fabric

v1.12.1

20 May 15:23
b672504

Choose a tag to compare

Patch release with several FabricSpark and Fabric DW improvements, mostly package-compatibility and reliability fixes. No breaking changes.

User-facing changes

New features

  • FabricSpark: clustered_by → Delta liquid clustering (#281) — the existing clustered_by model config is now wired to Delta liquid clustering on Lakehouse tables.
  • Fabric DW: compare_and_classify_query_results support (#289) — enables the corresponding dbt-audit-helper macro on Fabric DW.
  • Fabric DW: configurable lock timeout (#274, #278) — schema-lock waits are now capped via SET LOCK_TIMEOUT, with a configurable value (default 30s) to fail fast instead of hanging on contended DDL.

Fixes

  • Fabric DW: release Sch-S locks after synapsesql writes (#272) — Python model writes now await JVM GC so the underlying JDBC sessions release their Sch-S locks on the target table, eliminating multi-minute LCK_M_SCH_M waits on follow-up DDL.
  • dbt-date: get_fiscal_periods now dispatched correctly (#277) — closes #243.

Cleanups (user-visible)

  • Remove stale dbt-utils overrides for test_not_null_where / test_unique_where — these would have raised NoSuchMacroError against current dbt-utils since the upstream defaults were removed (#286).
  • Remove redundant relationships_where override — it now matches the upstream default verbatim (#288).

Dependencies

  • Lower requests floor to >=2.32.5 for broader resolver compatibility (#293).
  • Merged upstream microsoft/dbt-fabric v1.9.10 + v1.10.0 (review notes in commit; nothing needed porting).

Documentation

Internal / dev improvements

Not user-facing — listed for completeness:

  • CI: always run python_model tests (#279)
  • FabricSpark integration test speedup via HC session pooling (#268)
  • Test fixture: close HC Livy sessions before drop_test_schema (#276)
  • Comment cleanup in not_empty_string override (#287)

v1.12.0

18 May 07:53
bf45c64

Choose a tag to compare

Highlights

  • dbt Core 1.12 support (compatibility)
  • FabricSpark (Lakehouse) adapter matured significantly — now a first-class adapter alongside Fabric DW (lakehouse guide)
  • Microsoft Purview integration for syncing dbt metadata to your data catalog (docs)
  • dbt-external-tables support via OPENROWSET (docs)
  • Documentation site rebuilt on Zensical with extensive new content (dbt-fabric.debruyn.dev)

FabricSpark (Lakehouse) adapter

See the Lakehouse guide and FabricSpark configuration.

  • New materializations: incremental, snapshot, clone, materialized_view (lake views), and proper view support
  • Default materialization changed to view (matches dbt-spark conventions)
  • Cross-workspace 4-part naming
  • persist_docs support
  • High-concurrency Livy sessions for parallel statement execution
  • Configurable Fabric API base URLs for non-production tenants (gov clouds, sovereign clouds)
  • Fixed source freshness failures on Lakehouse (ported from upstream)
  • Fixed snapshots failing with character varying type error
  • Fixed TestMetadataWithEmptyFlag parse-time crash
  • Fixed escape_single_quotes behavior
  • Fixed schema filter push-down in list_relations_without_caching (performance)
  • Handle [SCHEMA_NOT_FOUND] gracefully in relation listing
  • Force JVM GC after synapsesql writes to release JDBC schema locks

Fabric Data Warehouse

  • Snapshots now use a single MERGE statement instead of UPDATE + INSERT (atomic, simpler) — see warehouse snapshots
  • apply_label() decoupled from statement termination
  • CLUSTER BY support on tables — see cluster by
  • New statistics config option — see statistics
  • Approximate row count statistics in catalog generation — see catalog stats
  • Fixed T-SQL identifier quoting to use brackets ([ ])
  • Fixed rows_affected reporting -1 for table materializations
  • Automatic retry for transient snapshot isolation errors on metadata queries
  • Singleton pollution fix in TestWarehouseSnapshots

Authentication

See the authentication guide for the full walkthrough.

  • New workload_identity method for federated credentials (CI/CD-friendly, OIDC)
  • New token_credential method (ported from upstream)
  • Authentication doc significantly expanded with end-to-end setup walkthroughs

Community package compatibility

Integration tests and macro overrides added for both Fabric and FabricSpark across these packages — see the packages index for required dispatch config and per-package notes:

Documentation

Docs site migrated from mkdocs-material to Zensical: dbt-fabric.debruyn.dev. New and significantly expanded pages:

Dependencies

  • Bumped dbt-core to 1.11.10
  • Bumped dbt-adapters to 1.23.0
  • Bumped mssql-python to 1.6.0
  • Security fixes via dependency bumps

Tooling and CI

  • Devcontainer for cloud development environments (Codespaces, etc.)
  • DW and DE integration tests split into separate workflows
  • On-demand DE tests via /test-de PR comments
  • On-demand DW tests via /test-dw PR comments
  • Extensive unit test coverage added for adapter internals (Credentials, Adapter, Column, Relation, ConnectionManager, ApiClient, LivySession, FabricSparkCursor)
  • Improved FabricSparkCursor PEP 249 compliance

v1.11.2

22 Feb 16:06

Choose a tag to compare

  • Make sure Python models reuse existing Spark sessions and add concurrency tests

v1.11.1

22 Feb 12:53

Choose a tag to compare

Add support for scalar functions (new feature in dbt Core 1.11)

v1.11.0

21 Feb 20:42

Choose a tag to compare

  • Add support for dbt Core 1.11 (Functions not supported yet)
  • Fix clone support: now deletes existing clones when trying to create a new one
  • Incremental models now throw an error when trying to use a strategy that doesn't exist
  • Added more integration tests

v1.10.4

21 Feb 15:29

Choose a tag to compare

  • Add support for array_concat macro
  • Add support for array_append macro
  • Rework incremental materialization logic with more resilient backup strategy and avoid creating views
  • Add support for incremental Python models

v1.10.3

20 Feb 17:49

Choose a tag to compare

What's Changed

  • Removed unnamed column warning for date spines
  • Fixed validate_sql dbt_utils macro

Full Changelog: v1.10.2...v1.10.3

v1.10.2

17 Feb 14:43

Choose a tag to compare

What's New

Warehouse Snapshots

Create point-in-time snapshots of your Fabric Data Warehouse directly from dbt using the new create_or_update_fabric_warehouse_snapshot macro. Unlike Microsoft's upstream implementation — which hijacks Python runtime components and bypasses dbt's lifecycle — this adapter gives you a macro you call explicitly from on-run-start, on-run-end, post-hook, or any other Jinja context.

on-run-end:
  - "{{ create_or_update_fabric_warehouse_snapshot('daily_snapshot', 'Snapshot after dbt run') }}"

You get full control: multiple snapshots per run, dynamic names, custom timing — all using standard dbt configuration.

Python Models

Run PySpark transformations against your Fabric Data Warehouse using Python models. Add your Workspace and Lakehouse to profiles.yml and write a model(dbt, spark) function that returns a PySpark DataFrame. The adapter handles Livy session management, code submission, and reading/writing via Fabric's native synapsesql connector.

# profiles.yml
dev:
  type: fabric
  workspace: My Workspace
  database: my_data_warehouse
  schema: dbt
  lakehouse: My Lakehouse
  authentication: CLI

This feature is exclusive to dbt-fabric-samdebruyn.

Documentation

New feature guides have been added to the documentation site:

  • Authentication — Covers Azure CLI, DefaultAzureCredential, service principal, environment variables, and Fabric Notebook authentication with ready-to-use profiles.yml examples
  • Python Models — How to write, configure, and troubleshoot Python models on Fabric
  • Warehouse Snapshots — Using the snapshot macro with hooks, dynamic names, and a comparison with Microsoft's approach