Skip to content

Commit 8c84320

Browse files
committed
Support dbt 1.5+
Support dbt 1.5+ and plan to switch to using dbt-core contraints (see note in readme) Note: dbt 1.5+ supports primary_key and foreign_key [constraints](https://docs.getdbt.com/reference/resource-properties/constraints). However, as of dbt-core v1.5.2, constraints require [contracts](https://docs.getdbt.com/reference/resource-configs/contract) to be enabled. The catch-22 is that contracts then require ALL model columns to define data_type, but data_type is database-specific, so you lose the incredible database agnostic features of dbt. For this reason, we use the [`dbt_constraints`](https://github.com/Snowflake-Labs/dbt_constraints) package to enable cross-database support for primary and foreign keys. The issue will be tracked [here](#11).
1 parent dd46da5 commit 8c84320

File tree

5 files changed

+32
-13
lines changed

5 files changed

+32
-13
lines changed

README.md

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -252,6 +252,12 @@ models:
252252
field: customer_key
253253
```
254254

255+
Another way for tools to derive joins from dbt models is by defining primary and foreign keys. While many columnar databases (e.g. Snowflake) do not enfore referential integrity (primary and foreign keys), they can still be defined so that tools can pick them up. For traditional relational databases, it can be important to define primary and foreign keys for performance reasons as well.
256+
257+
> Important: to use primary and foreign key constraints, set the dbt_project.yml variable "dbt_constraints_enabled: true"
258+
259+
> Note: dbt 1.5+ supports primary_key and foreign_key [constraints](https://docs.getdbt.com/reference/resource-properties/constraints). However, as of dbt-core v1.5.2, constraints require [contracts](https://docs.getdbt.com/reference/resource-configs/contract) to be enabled. The catch-22 is that contracts then require ALL model columns to define data_type, but data_type is database-specific, so you lose the incredible database agnostic features of dbt. For this reason, we use the [`dbt_constraints`](https://github.com/Snowflake-Labs/dbt_constraints) package to enable cross-database support for primary and foreign keys. The issue will be tracked [here](https://github.com/flexanalytics/dbt-business-intelligence/issues/11).
260+
255261
### Data Freshness
256262

257263
BI tools should use dbt's built in run/job update timestamps and [source freshness](https://docs.getdbt.com/reference/resource-properties/freshness) to display data freshness information to end users.
@@ -379,6 +385,9 @@ Here is a list of concepts that were covered in this repo.
379385
* [`dbt-date`](https://github.com/calogica/dbt-date#get_date_dimensionstart_date-end_date)
380386
* [`dbt-utils`](https://github.com/dbt-labs/dbt-utils)
381387
* [`dbt_constraints`](https://github.com/Snowflake-Labs/dbt_constraints)
388+
* [`metrics`](https://github.com/dbt-labs/metrics)
389+
390+
> **WARNING**: dbt_metrics is going to be deprecated in dbt-core 1.6 in July 2023 as part of the migration to MetricFlow. This package will continue to work with dbt-core 1.5 but a 1.6 version will not be released. If you have any questions, please join us in the #dbt-core-metrics in the dbt Community Slack.
382391

383392
* **Jinja & Macros** - using [Jinja & Macros](https://docs.getdbt.com/docs/building-a-dbt-project/jinja-macros) to create re-usable code
384393

macros/bigquery__create_constraints.sql

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -1,34 +1,34 @@
11
{# BigQuery specific implementation to create a primary key #}
2-
{%- macro bigquery__create_primary_key(table_relation, column_names, verify_permissions, quote_columns=false) -%}
2+
{%- macro bigquery__create_primary_key(table_relation, column_names, verify_permissions, quote_columns=false, constraint_name=none, lookup_cache=none) -%}
33
{#- TODO -#}
44
{%- endmacro -%}
55

66

77

88
{# BigQuery specific implementation to create a unique key #}
9-
{%- macro bigquery__create_unique_key(table_relation, column_names, verify_permissions, quote_columns=false) -%}
9+
{%- macro bigquery__create_unique_key(table_relation, column_names, verify_permissions, quote_columns=false, constraint_name=none, lookup_cache=none) -%}
1010
{#- TODO -#}
1111
{%- endmacro -%}
1212

1313

1414

1515
{# BigQuery specific implementation to create a foreign key #}
16-
{%- macro bigquery__create_foreign_key(pk_table_relation, pk_column_names, fk_table_relation, fk_column_names, verify_permissions, quote_columns=true) -%}
16+
{%- macro bigquery__create_foreign_key(pk_table_relation, pk_column_names, fk_table_relation, fk_column_names, verify_permissions, quote_columns=true, constraint_name=none, lookup_cache=none) -%}
1717
{#- TODO -#}
1818
{%- endmacro -%}
1919

2020

2121

2222
{#- This macro is used in create macros to avoid duplicate PK/UK constraints
2323
and to skip FK where no PK/UK constraint exists on the parent table -#}
24-
{%- macro bigquery__unique_constraint_exists(table_relation, column_names) -%}
24+
{%- macro bigquery__unique_constraint_exists(table_relation, column_names, lookup_cache=none) -%}
2525
{#- TODO -#}
2626
{%- endmacro -%}
2727

2828

2929

3030
{#- This macro is used in create macros to avoid duplicate FK constraints -#}
31-
{%- macro bigquery__foreign_key_exists(table_relation, column_names) -%}
31+
{%- macro bigquery__foreign_key_exists(table_relation, column_names, lookup_cache=none) -%}
3232
{#- TODO -#}
3333
{%- endmacro -%}
3434

@@ -51,6 +51,6 @@
5151
{#- TODO -#}
5252
{% endmacro %}
5353

54-
{% macro bigquery__create_not_null() -%}
54+
{% macro bigquery__create_not_null(table_relation, column_names, verify_permissions, quote_columns=false, lookup_cache=none) -%}
5555
{#- TODO -#}
5656
{% endmacro %}

macros/sqlserver__create_constraints.sql

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
{# SQL Server specific implementation to create a primary key #}
2-
{%- macro sqlserver__create_primary_key(table_relation, column_names, verify_permissions, quote_columns=false) -%}
2+
{%- macro sqlserver__create_primary_key(table_relation, column_names, verify_permissions, quote_columns=false, constraint_name=none, lookup_cache=none) -%}
33
{%- set constraint_name = (table_relation.identifier ~ "_" ~ column_names|join('_') ~ "_PK") | upper -%}
44

55
{%- if constraint_name|length > 63 %}
@@ -36,7 +36,7 @@
3636

3737

3838
{# SQL Server specific implementation to create a unique key #}
39-
{%- macro sqlserver__create_unique_key(table_relation, column_names, verify_permissions, quote_columns=false) -%}
39+
{%- macro sqlserver__create_unique_key(table_relation, column_names, verify_permissions, quote_columns=false, constraint_name=none, lookup_cache=none) -%}
4040
{%- set constraint_name = (table_relation.identifier ~ "_" ~ column_names|join('_') ~ "_UK") | upper -%}
4141

4242
{%- if constraint_name|length > 63 %}
@@ -71,7 +71,7 @@
7171
{%- endmacro -%}
7272

7373
{# SQL Server specific implementation to create a not null constraint #}
74-
{%- macro sqlserver__create_not_null(table_relation, column_names, verify_permissions, quote_columns=false) -%}
74+
{%- macro sqlserver__create_not_null(table_relation, column_names, verify_permissions, quote_columns=false, lookup_cache=none) -%}
7575
{%- set columns_list = dbt_constraints.get_quoted_column_list(column_names, quote_columns) -%}
7676

7777
{%- if dbt_constraints.have_ownership_priv(table_relation, verify_permissions) -%}
@@ -93,7 +93,7 @@
9393
{%- endmacro -%}
9494

9595
{# SQL Server specific implementation to create a foreign key #}
96-
{%- macro sqlserver__create_foreign_key(pk_table_relation, pk_column_names, fk_table_relation, fk_column_names, verify_permissions, quote_columns=true) -%}
96+
{%- macro sqlserver__create_foreign_key(pk_table_relation, pk_column_names, fk_table_relation, fk_column_names, verify_permissions, quote_columns=true, constraint_name=none, lookup_cache=none) -%}
9797
{%- set constraint_name = (fk_table_relation.identifier ~ "_" ~ fk_column_names|join('_') ~ "_FK") | upper -%}
9898

9999
{%- if constraint_name|length > 63 %}
@@ -136,7 +136,7 @@
136136

137137
{#- This macro is used in create macros to avoid duplicate PK/UK constraints
138138
and to skip FK where no PK/UK constraint exists on the parent table -#}
139-
{%- macro sqlserver__unique_constraint_exists(table_relation, column_names) -%}
139+
{%- macro sqlserver__unique_constraint_exists(table_relation, column_names, lookup_cache=none) -%}
140140
{%- set lookup_query -%}
141141
select c.oid as constraint_name
142142
, upper(col.attname) as column_name
@@ -170,7 +170,7 @@
170170

171171

172172
{#- This macro is used in create macros to avoid duplicate FK constraints -#}
173-
{%- macro sqlserver__foreign_key_exists(table_relation, column_names) -%}
173+
{%- macro sqlserver__foreign_key_exists(table_relation, column_names, lookup_cache=none) -%}
174174
{%- set lookup_query -%}
175175
select c.oid as fk_name
176176
, upper(col.attname) as fk_column_name

models/schema.yml

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,12 @@ models:
1010
- order_key
1111
- product_key
1212
- date_key
13+
- dbt_constraints.primary_key:
14+
column_names:
15+
- customer_key
16+
- order_key
17+
- product_key
18+
- date_key
1319
meta:
1420
label: Sales Fact
1521
display_index: 1
@@ -85,6 +91,7 @@ models:
8591
hidden: true
8692
tests:
8793
- unique
94+
- dbt_constraints.primary_key
8895
- name: customer_code
8996
meta:
9097
label: Customer Id
@@ -131,6 +138,7 @@ models:
131138
hidden: true
132139
tests:
133140
- unique
141+
- dbt_constraints.primary_key
134142
- name: order_number
135143
meta:
136144
label: Order Number
@@ -160,6 +168,7 @@ models:
160168
hidden: true
161169
tests:
162170
- unique
171+
- dbt_constraints.primary_key
163172
- name: product_code
164173
meta:
165174
label: Product Code
@@ -201,6 +210,7 @@ models:
201210
hidden: true
202211
tests:
203212
- unique
213+
- dbt_constraints.primary_key
204214
- name: date_day
205215
meta:
206216
label: Order Date

packages.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@ packages:
22
- package: calogica/dbt_date
33
version: 0.6.2
44
- package: dbt-labs/metrics
5-
version: 1.4.1
5+
version: 1.5.0
66
- package: Snowflake-Labs/dbt_constraints
77
version: 0.6.0
88
- package: dbt-msft/tsql_utils

0 commit comments

Comments
 (0)