Skip to content

Commit 38f84fb

Browse files
committed
fix(bigquery): qualify column refs in test_unique/test_not_null to avoid row-struct resolution
When a column shares its name with the model, BigQuery's identifier resolution returns the row STRUCT rather than the column value for unqualified references. The default `bigquery__test_unique` and the inherited `default__test_not_null` therefore evaluated WHERE/GROUP BY against the row struct, masking duplicates and nulls. Alias the source relation as `dbt_test__source` and qualify the column references in `bigquery__test_unique`, and add a `bigquery__test_not_null` override that does the same. Adds a regression test that creates a single-column model whose column name matches the model name and asserts both generic tests detect failures. Resolves dbt-core#11067.
1 parent afd64a3 commit 38f84fb

4 files changed

Lines changed: 93 additions & 3 deletions

File tree

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
kind: Fixes
2+
body: Alias the source relation in `bigquery__test_unique` and add a `bigquery__test_not_null`
3+
override so that columns sharing a name with their model resolve to the column
4+
value instead of BigQuery's row STRUCT, fixing dbt-core#11067.
5+
time: 2026-04-27T08:38:10.000000+00:00
6+
custom:
7+
Author: "1fanwang"
8+
Issue: "11067"

dbt-bigquery/src/dbt/include/bigquery/macros/adapters.sql

Lines changed: 18 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -209,13 +209,16 @@
209209
{% endmacro %}
210210

211211

212+
{# When a column has the same name as its table, BigQuery resolves an unqualified
213+
column reference to the row STRUCT instead of the column value. Aliasing the
214+
relation and qualifying the column reference forces resolution to the column. #}
212215
{% macro bigquery__test_unique(model, column_name) %}
213216

214217
with dbt_test__target as (
215218

216-
select {{ column_name }} as unique_field
217-
from {{ model }}
218-
where {{ column_name }} is not null
219+
select dbt_test__source.{{ column_name }} as unique_field
220+
from {{ model }} dbt_test__source
221+
where dbt_test__source.{{ column_name }} is not null
219222

220223
)
221224

@@ -229,6 +232,18 @@ having count(*) > 1
229232

230233
{% endmacro %}
231234

235+
{# Same row-vs-column resolution issue as bigquery__test_unique: alias the
236+
relation and qualify the column reference. #}
237+
{% macro bigquery__test_not_null(model, column_name) %}
238+
239+
{% set column_list = '*' if should_store_failures() else 'dbt_test__source.' ~ column_name %}
240+
241+
select {{ column_list }}
242+
from {{ model }} dbt_test__source
243+
where dbt_test__source.{{ column_name }} is null
244+
245+
{% endmacro %}
246+
232247
{% macro bigquery__upload_file(local_file_path, database, table_schema, table_name) %}
233248

234249
{{ log("kwargs: " ~ kwargs) }}

dbt-bigquery/tests/functional/adapter/generic_tests/__init__.py

Whitespace-only changes.
Lines changed: 67 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,67 @@
1+
"""
2+
Regression test for dbt-core#11067.
3+
4+
When a column has the same name as its table, BigQuery resolves an unqualified
5+
column reference to the row STRUCT (containing all of the row's columns) rather
6+
than to the column value. The default `unique` and `not_null` generic tests
7+
emit unqualified column references, which means the resulting SQL filters and
8+
groups against the row struct instead of the column. After the fix in
9+
`bigquery__test_unique` and `bigquery__test_not_null`, the source relation is
10+
aliased and column references are qualified, so the tests evaluate against the
11+
column as expected.
12+
"""
13+
14+
import pytest
15+
16+
from dbt.tests.util import run_dbt
17+
18+
19+
_MODEL_SQL = """
20+
select 1 as orders union all
21+
select 1 as orders union all
22+
select 2 as orders union all
23+
select cast(null as int64) as orders
24+
""".lstrip()
25+
26+
27+
_SCHEMA_YML = """
28+
version: 2
29+
models:
30+
- name: orders
31+
columns:
32+
- name: orders
33+
data_tests:
34+
- unique
35+
- not_null
36+
""".lstrip()
37+
38+
39+
class TestBigQueryGenericTestsColumnTableNameCollision:
40+
@pytest.fixture(scope="class")
41+
def models(self):
42+
return {
43+
"orders.sql": _MODEL_SQL,
44+
"schema.yml": _SCHEMA_YML,
45+
}
46+
47+
def test_unique_and_not_null_detect_failures(self, project):
48+
run_dbt(["run"])
49+
results = run_dbt(["test"], expect_pass=False)
50+
51+
assert len(results) == 2
52+
53+
statuses = {r.node.name: (r.status, r.failures) for r in results}
54+
55+
unique_status, unique_failures = statuses["unique_orders_orders"]
56+
assert unique_status == "fail", (
57+
f"unique test passed unexpectedly (status={unique_status}); "
58+
"with the bug, the row-struct grouping can mask duplicates"
59+
)
60+
assert unique_failures >= 1
61+
62+
not_null_status, not_null_failures = statuses["not_null_orders_orders"]
63+
assert not_null_status == "fail", (
64+
f"not_null test passed unexpectedly (status={not_null_status}); "
65+
"with the bug, the row-struct nullness check masks the null column value"
66+
)
67+
assert not_null_failures >= 1

0 commit comments

Comments
 (0)