Add Microsoft Fabric Data Warehouse support (+ SQL Server and Synapse)#576
Conversation
Fabric uses T-SQL with BIT booleans and has several SQL dialect differences from PostgreSQL/Snowflake/BigQuery. This commit makes the package cross-platform compatible with Fabric DW by: - Adding fabric__recursive_dag (loop-based, no recursive CTEs) - Adding fabric__get_dbtreplace_directory_pattern (no regexp_replace) - Adding fabric__type_string_dpe (varchar(8000)) - Adding quote_identifier/bool_literal dispatch macros - Replacing boolean expressions in SELECT with CASE WHEN - Replacing bare booleans in WHERE with explicit comparisons - Replacing GROUP BY ordinals with column names - Replacing || with dbt.concat() - Replacing 'where false' with 'where 1=0' - Replacing cast(True/False as ...) with cast(1/0 as ...) - Adding 'fabric' to target.type conditionals - Guarding ORDER BY in CTEs (invalid in T-SQL without TOP) Relates to dbt-labs#229 Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The column aliases `as database` and `as schema` in stg_nodes.sql and stg_sources.sql cause T-SQL syntax errors since these are reserved keywords. Use quote_identifier dispatch macro for the aliases too. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
is_primary_test_relationship was still used as a bare boolean in WHERE. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Generic dispatch macros belong in their own files (like type_string.sql), not in fabric_shims.sql which is only for Fabric-specific overrides. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Adapter-specific overrides of the same macro belong in one file. fabric__escape_single_quotes joins spark__escape_single_quotes. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
File names should reflect the macro concept, not the adapter. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
There was a problem hiding this comment.
Pull request overview
Adds support for Microsoft Fabric Data Warehouse (fabric adapter). Because Fabric uses T-SQL semantics (BIT booleans, no ordinal GROUP BY in CTEs, no || concat, no false literal, no regexp_replace, ORDER BY not allowed in CTEs without TOP/OFFSET, reserved-word handling for database/schema), the PR rewrites SQL patterns across staging and mart models to be Fabric-compatible while remaining ANSI for other adapters, and introduces several Fabric-specific dispatch macros.
Changes:
- New Fabric-specific macros:
fabric__recursive_dag,fabric__get_dbtreplace_directory_pattern,fabric__type_string_dpe,fabric__is_not_empty_string, plus new dispatch macrosquote_identifierandbool_literal. - Cross-adapter SQL rewrites: explicit boolean comparisons (
= cast(1/0 as boolean)),dbt.concat()instead of||,where 1=0instead ofwhere false, column-nameGROUP BY, andquote_identifier('database'/'schema')for reserved words. - Wiring
fabricintodbt_project.ymlmaterialization conditionals,listaggordering lists, and CTE-levelORDER BYguards.
Reviewed changes
Copilot reviewed 54 out of 54 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
supported_adapters.env |
Adds fabric to supported adapters list. |
dbt_project.yml |
Adds fabric to materialization-type conditionals and max_depth_dag defaults. |
macros/cross_db_shim/quote_identifier.sql |
New dispatch macro to bracket reserved column names on Fabric. |
macros/cross_db_shim/bool_literal.sql |
New dispatch macro to render Python booleans as 1/0 for Fabric. |
macros/cross_db_shim/escape_single_quotes.sql |
Consolidated single-quote escapers (moved Spark + added Fabric). |
macros/cross_db_shim/spark_shims.sql |
Removed; logic moved to escape_single_quotes.sql. |
macros/cross_db_shim/type_string.sql |
Adds fabric__type_string_dpe returning varchar(8000). |
macros/is_not_empty_string.sql |
Adds Fabric implementation returning 1/0. |
macros/get_directory_pattern.sql |
Adds Fabric branch using charindex/left (only handles /). |
macros/recursive_dag.sql |
Adds loop-based fabric__recursive_dag (no recursive CTEs). |
macros/unpack/get_*_values.sql |
Wraps Python booleans through bool_literal for Fabric compatibility. |
models/marts/core/int_all_graph_resources.sql |
Uses quote_identifier for database/schema, concat for prefix, Fabric branch for file_name. |
models/marts/core/int_direct_relationships.sql |
Rewrites boolean expression as CASE. |
models/marts/dag/fct_*.sql |
Replaces not <bool> with explicit comparisons, ordinals with column names, guards order by in CTEs for Fabric. |
models/marts/dag/fct_duplicate_sources.sql |
Uses quote_identifier for database/schema references. |
models/marts/dag/fct_direct_join_to_source.sql |
Moves order by out of CTE to outer select. |
models/marts/dag/fct_model_fanout.sql, fct_source_fanout.sql, fct_multiple_sources_joined.sql |
Guards CTE order by for Fabric, adds Fabric to listagg ordered list. |
models/marts/documentation/*.sql |
Replaces not is_* predicates with explicit boolean comparisons. |
models/marts/governance/*.sql |
Same boolean-comparison rewrites. |
models/marts/performance/*.sql |
Same boolean-comparison rewrites. |
models/marts/structure/fct_model_directories.sql, fct_source_directories.sql |
Replaces ` |
models/marts/structure/fct_model_naming_conventions.sql, fct_test_directories.sql |
Boolean comparison rewrites + Fabric listagg handling. |
models/marts/tests/*.sql |
Boolean rewrites; rewrites cast(sum(...)>=1 as boolean) as CASE for Fabric. |
models/staging/graph/**/*.sql |
Replaces true/false literals, uses quote_identifier, where 1=0. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
The Fabric-specific charindex/left/right logic only handled forward slashes. Windows users running dbt against Fabric would get incorrect directory_path and file_name values. Now dispatches on is_os_mac_or_linux like the non-Fabric branches already do. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Documents the package, dispatch config, project_vars (max_depth_dag), known dbt-utils dependency, and the integration_tests subdirectory caveat. Notes Lakehouse compatibility (integration-tested) and links to upstream PR dbt-labs/dbt-project-evaluator#576 for Fabric DW. Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
Thanks @sdebruyn 🙌 I had this one on the backburner for the same reason that you found working on it. This requires a lot of changes 😄 . I will try to give it a proper look in the next few days. |
…tax error filter_exceptions() emits a WHERE clause; ORDER BY must come after WHERE.
…attern to dispatch - Add Microsoft Fabric Data Warehouse and Fabric Spark to supported adapter lists in README and docs/index.md - Update Additional setup headings to include Fabric - Update Limitations section and max_depth_dag docs to include Fabric (no recursive CTE support, uses loop-based approach, defaults to 9) - Refactor get_dbtreplace_directory_pattern from if target.type guard to adapter.dispatch pattern, consistent with recursive_dag.sql
|
Hi @sdebruyn ! I just did some small changes to the PR. |
|
Yes, LGTM! |
Summary
Adds support for Microsoft Fabric Data Warehouse (adapter type:
fabric). Tested with dbt-fabric.Fabric uses T-SQL with BIT for booleans, which requires several SQL pattern changes. Most changes are cross-platform improvements that benefit all adapters:
column = cast(1 as {{ dbt.type_boolean() }})), boolean expressions in SELECT with CASE WHEN, andtrue/falseliterals withcast(1/0 as {{ dbt.type_boolean() }}))GROUP BY 1, 2with column names (Fabric DW does not support ordinal positions in CTEs)||with{{ dbt.concat() }}(T-SQL uses+, not||)where false: Replaced withwhere 1=0(T-SQL has nofalsekeyword)Fabric-specific additions:
fabric__recursive_dag— loop-based approach since Fabric DW does not support recursive CTEs (same pattern as BigQuery/Spark/Trino)fabric__get_dbtreplace_directory_pattern— usescharindex/leftsince T-SQL has noregexp_replacefabric__type_string_dpe— returnsvarchar(8000)quote_identifierdispatch macro — wraps reserved-word column names (database,schema) in[brackets]on Fabricbool_literaldispatch macro — converts Python booleans to1/0in INSERT statementsfabric__is_not_empty_string— returns1/0instead oftrue/falsefabricto alltarget.typeconditional lists (dbt_project.yml,listaggorder clauses, materialization configs){% if target.type not in ['fabric'] %}(invalid in T-SQL without TOP/OFFSET)Closes #229
Closes #553
Test plan
🤖 Generated with Claude Code