Is this a new bug?
Which packages are affected?
Current Behavior
When enable_truthy_nulls_equals_macro is enabled (which becomes the default in 1.12 via #1877), bigquery__equals emits IS NOT DISTINCT FROM. This clause is reused inside the MERGE ON predicate of the default get_merge_sql macro whenever unique_key is a string.
For an incremental model that has partition_by set together with require_partition_filter=True, BigQuery's partition-pruning analyzer no longer recognizes the auxiliary (<partition_field> is null or <partition_field> is not null) predicate (added by predicate_for_avoid_require_partition_filter) as a valid partition filter once the unique-key match uses IS NOT DISTINCT FROM. The MERGE then fails at runtime.
Expected Behavior
Incremental models that use merge strategy with a string unique_key, partition_by, and require_partition_filter=True should run successfully when enable_truthy_nulls_equals_macro is enabled, just as they do when the flag is disabled.
Steps To Reproduce
-
Configure a BigQuery target.
-
Enable the behavior flag in dbt_project.yml:
flags:
enable_truthy_nulls_equals_macro: true
-
Define an incremental model with a string unique_key, partition_by, and require_partition_filter=True. Example (models/incremental_merge_time_with_require_partition.sql):
{{
config(
materialized="incremental",
unique_key="id",
cluster_by="id",
require_partition_filter=true,
partition_by={
"field": "date_time",
"data_type": "dateTime"
}
)
}}
with data as (
{% if not is_incremental() %}
select 1 as id, cast('2020-01-01' as datetime) as date_time union all
select 2 as id, cast('2020-01-01' as datetime) as date_time
{% else %}
select 1 as id, cast('2020-01-02' as datetime) as date_time union all
select 3 as id, cast('2020-01-02' as datetime) as date_time
{% endif %}
)
select * from data
-
Run dbt run once to materialize the model (full refresh).
-
Run dbt run again to trigger the incremental MERGE.
The second run fails.
Relevant log output
Database Error in model incremental_merge_time_with_require_partition (models/incremental_merge_time_with_require_partition.sql)
Cannot query over table 'dbt-test-env.<schema>.incremental_merge_time_with_require_partition' without a filter over column(s) 'date_time' that can be used for partition elimination
compiled code at target/run/test/models/incremental_merge_time_with_require_partition.sql
Generated MERGE ON clause (problematic):
merge into `<target>` as DBT_INTERNAL_DEST
using (...) as DBT_INTERNAL_SOURCE
on (DBT_INTERNAL_SOURCE.id IS NOT DISTINCT FROM DBT_INTERNAL_DEST.id)
and (`DBT_INTERNAL_DEST`.`date_time` is null
or `DBT_INTERNAL_DEST`.`date_time` is not null)
CI run that surfaced this: https://github.com/dbt-labs/dbt-adapters/actions/runs/24562327293/job/72021560451 (PR #1877, which flips the flag default to `true` for 1.12).
Environment
- OS: macOS 26.2 (also reproduces on Ubuntu 22.04 in CI)
- Python: 3.12.4 (CI uses 3.10)
- dbt-adapters: 1.22.10
- dbt-bigquery: 1.11.0rc3
Additional Context
The other adapters that emit IS NOT DISTINCT FROM from their equals macro (postgres, snowflake, athena) are unaffected — only BigQuery's MERGE ON / partition-pruning interaction breaks.
Is this a new bug?
Which packages are affected?
Current Behavior
When
enable_truthy_nulls_equals_macrois enabled (which becomes the default in 1.12 via #1877),bigquery__equalsemitsIS NOT DISTINCT FROM. This clause is reused inside the MERGE ON predicate of the defaultget_merge_sqlmacro wheneverunique_keyis a string.For an incremental model that has
partition_byset together withrequire_partition_filter=True, BigQuery's partition-pruning analyzer no longer recognizes the auxiliary(<partition_field> is null or <partition_field> is not null)predicate (added bypredicate_for_avoid_require_partition_filter) as a valid partition filter once the unique-key match usesIS NOT DISTINCT FROM. The MERGE then fails at runtime.Expected Behavior
Incremental models that use
mergestrategy with a stringunique_key,partition_by, andrequire_partition_filter=Trueshould run successfully whenenable_truthy_nulls_equals_macrois enabled, just as they do when the flag is disabled.Steps To Reproduce
Configure a BigQuery target.
Enable the behavior flag in
dbt_project.yml:Define an incremental model with a string
unique_key,partition_by, andrequire_partition_filter=True. Example (models/incremental_merge_time_with_require_partition.sql):{{ config( materialized="incremental", unique_key="id", cluster_by="id", require_partition_filter=true, partition_by={ "field": "date_time", "data_type": "dateTime" } ) }} with data as ( {% if not is_incremental() %} select 1 as id, cast('2020-01-01' as datetime) as date_time union all select 2 as id, cast('2020-01-01' as datetime) as date_time {% else %} select 1 as id, cast('2020-01-02' as datetime) as date_time union all select 3 as id, cast('2020-01-02' as datetime) as date_time {% endif %} ) select * from dataRun
dbt runonce to materialize the model (full refresh).Run
dbt runagain to trigger the incremental MERGE.The second run fails.
Relevant log output
Environment
Additional Context
The other adapters that emit
IS NOT DISTINCT FROMfrom theirequalsmacro (postgres,snowflake,athena) are unaffected — only BigQuery's MERGE ON / partition-pruning interaction breaks.