Skip to content

CHANGELOG.md

Latest commit

 

History

History
291 lines (200 loc) · 21.5 KB

CHANGELOG.md

File metadata and controls

291 lines (200 loc) · 21.5 KB
Raw

Changelog

  • This file provides a full account of all changes to dbt-bigquery
  • The format is based on Keep a Changelog, adheres to Semantic Versioning, and is generated by Changie
  • Changes are listed under the (pre-)release in which they first appear
  • Subsequent releases include changes from previous releases
  • "Breaking changes" listed under a version may require action from end users or external maintainers when upgrading to that version
  • Do not edit this file directly. This file is auto-generated using changie
  • For details on how to document a change, see the contributing guide

dbt-bigquery 1.12.0-b1 - June 15, 2026

Features

  • Add reservation config option to the BigQuery adapter, enabling jobs to be routed to a specific BigQuery reservation at the target (profiles.yml), project (+reservation), or model level ({{ config(reservation=...) }}) (#1228)

Fixes

  • Fixed a bug where a BigQuery job that fails with an internal error causes dbt to poll getQueryResults until the full job_execution_timeout_seconds elapses (potentially hours) instead of failing fast. The polling retry now short-circuits immediately when the underlying job has reached a terminal failed state, and the retry budget is controlled by job_retry_deadline_seconds (default 10 min) rather than job_execution_timeout_seconds. (#1956)
  • Fixed a regression where transient getQueryResults rate-limit errors (rateLimitExceeded) surfaced as hard job failures during bursts of concurrent jobs. The polling retry no longer caps attempts at job_retries; retryable errors now back off (initial 5s) until job_retry_deadline_seconds, while still short-circuiting immediately when the underlying job is in a terminal failed state. Rate-limit errors back off without an extra jobs.get reload to avoid adding pressure to the rate limit being waited out. (#1957)

Under the Hood

  • Stabilize flaky BigQuery Python-model integration tests by retrying transient Dataproc CPU-quota failures, and shard the suite across parallel CI runners with pytest-split to keep it fast (#)

Dependencies

  • Add upper bound <2.0 on dbt-core to prevent pip from resolving to dbt-core 2.0 (Fusion engine), which is incompatible with the python adapter (#2013)

Contributors

dbt-bigquery 1.11.2-b1 - May 25, 2026

dbt-bigquery 1.11.1-b1 - May 22, 2026

Breaking Changes

  • Enable bigquery_use_standard_sql_for_partitions by default, switching partition metadata queries from Legacy SQL ($__PARTITIONS_SUMMARY__) to Standard SQL (INFORMATION_SCHEMA.PARTITIONS) ahead of BigQuery's Legacy SQL deprecation on June 1, 2026. This may require IAM permission updates (bigquery.tables.get + bigquery.tables.list instead of bigquery.tables.getData) for users with custom roles, and custom macros that access columns beyond partition_id from get_partitions_metadata() will need updating. Users can revert to Legacy SQL by setting flags.bigquery_use_standard_sql_for_partitions: false in dbt_project.yml. (#1725)

Features

  • Add BigQuery relation naming macros to support concurrent dbt execution without relation name collisions (#1359)
  • Configure notebook execution job security labels to support cross-project runtime templates (#1594)
  • Make job_execution_timeout_seconds configurable at the model level (#575)
  • Support PyPI packages for Python UDFs - when packages are specified in function config, they are templated into OPTIONS(packages = [...]) for scalar Python UDFs (#1737)
  • Add behavior flag bigquery_use_standard_sql_for_partitions to migrate get_partitions_metadata from Legacy SQL to Standard SQL using INFORMATION_SCHEMA.PARTITIONS (#1725)
  • Added support for JS UDFs (#1859)
  • Added support for --empty in dbt seed (#1865)
  • Add bridge_v2_catalog support via _v2_to_v1_type and Capability.CatalogsV2 (#1920)

Fixes

  • Fix query retries on transient BigQuery errors (503, 500, rate limits) by wrapping query execution with retry logic and adding retry parameter to result fetching (#941)
  • Add testing for grant preservation during clone operations (#1503)
  • Ensure BigQuery query result polling enforces the configured job execution timeout plus a short buffer to avoid indefinite hangs when connections drop mid-query. (#1500)
  • Lazy-load aiplatform and nbformat imports to fix dbt parse performance regression introduced in 1.9.2 by BigFrames support. (#1604)
  • Fix incorrect source freshness for wildcard table names (e.g. events_*). Metadata-based freshness checks now raise a clear error for wildcard tables. (#536)
  • Fix BigFrames notebook execution with OAuth service account credentials by detecting identity from credential object properties (#1231)
  • Fix insert_overwrite with integer range partitions generating oversized replacement arrays and leaving stale data by normalizing field values to partition boundaries. (#605)
  • Make BigQuery job link logging user-configurable via job_link_info_level_log profile setting. Defaults to debug level; set job_link_info_level_log: true in profiles.yml to log at info level (#544)
  • equals macro now uses IS NOT DISTINCT FROM clause for comparison (#1663)
  • When persist_docs.columns is enabled, dbt now warns about columns defined in YAML that don't exist in the database via the validate_doc_columns helper, with STRUCT-aware column name matching. (#1690)
  • Maintains declared order of cluster_by in generated query (#586)
  • Fix expiration_timestamp rendered as unquoted Python datetime in materialized view OPTIONS clause, causing BigQuery syntax error; now formatted as TIMESTAMP literal (#1209)
  • Fix nested STRUCT fields misassigning values when YAML order differs (#1530)
  • Enable discovering pre-existing functions by updating list_relations_without_caching to include function relations (#1488)
  • Fixed a bug where incremental models fail when using flag enable_truthy_nulls_equals_macro (#1902)
  • Skip nonexistent schemas like information_schema when building catalog relations (#1005)
  • Struct aware column list creation in persist_docs now retains case sensitivity for case sensitive validation (#NA)

Under the Hood

  • Add bigquery_noop_alter_relation_comment behavior flag to avoid an unnecessary table description update call (#1548)

Dependencies

  • Bump dbt-adapters floor to 1.24.1 for catalogs v2 and JS UDF support (#1964)

Contributors

dbt-bigquery 1.11.0-rc3 - December 17, 2025

Features

  • Support BigQuery STRUCT schema change (#599)

Contributors

dbt-bigquery 1.11.0-rc2 - December 09, 2025

Features

  • 🐛 ⛄ Support batched source freshness in bigquery using TABLE_STORAGE, behind behavior change flag. (#561)
  • Add support for py3.13 (#1475)

dbt-bigquery 1.11.0-rc1 - November 19, 2025

Features

  • Add support for python UDFs (#1410)
  • Add support for aggregate sql functions (#1432)

Dependencies

  • Bump minimum dbt-adapters to 1.19.0 (#1421)

dbt-bigquery 1.11.0-b2 - November 04, 2025

Fixes

  • Fix BigFrames python model + oauth + notebook template by adding a Request to creds.refresh calll (#1379)

Under the Hood

  • Update bigquery job handling to invoke job.cancel on timeout (#1355)
  • Test that bigquery does nothing with volatility (#1345)
  • Test handling of unknown function macro lookup (#1344)

Dependencies

  • Drop support for python 3.9 (#1412)

dbt-bigquery 1.11.0-b1 - October 07, 2025

Features

  • Add support for BigQuery resource tags via resource_tags configuration option (#565, #577)
  • Support the updating of incremental model description after initial creation (#585)
  • Add enable_change_history option for models on BigQuery. (#1206)
  • Add support for scalar functions (#1290)

Fixes

  • BigFrames notebook execution when using a service account with oauth (#1231)
  • Ensure we have exchanged a refresh-token for a token prior to detemining notebook user in BigFrames (#1219)
  • Fix references to function properties in function macros (#1365)

Under the Hood

  • Validate notebooks authorization request has succeeded (#1307)
  • Emit SQLQueryStatus during query execution (#1327)

Contributors

dbt-bigquery 1.10.1 - July 29, 2025

Features

  • Support impersonate service account in bigframes mode (#1202)

Fixes

  • Add support for copy_partitions when used with static partitions. (#540)
  • Track notebook job status to avoid LRO in bigframes mode (#1175)

Contributors

dbt-bigquery 1.10.0-rc2 - June 24, 2025

Features

  • Add user agent in bigframes mode (#1152)

Fixes

  • Add create_notebook_client (#977)
  • Use execute wrapper instead of raw execute to add the query comment as query header (#1090)

Under the Hood

  • Issue drop statement before cloning (#1077)

Dependencies

  • update the lower bound for dbt-adapters version in dbt-bigquery to >=1.16 (#1148)
  • add lower bound for google-auth in dbt-bigquery (#1149)

Contributors

dbt-bigquery 1.10.0-rc1 - June 04, 2025

Breaking Changes

  • rename 'catalog' model config value to 'catalog_name' (#1134)

Features

Fixes

  • Set partial ordering mode as default in dbt bigframes (#1035)
  • Set 1 hour as default timeout in bigframes mode (#1070)
  • Set job_retry to None to resolve Google Cloud warning (##1088)

Under the Hood

  • populate file_format from top level integration config field (#1129)
  • Rename and separate out info schema and biglake catalog integrations (#1129)

Dependencies

  • bump dbt-core requirement to 1.10.0rc0 (#1142)

Contributors

dbt-bigquery 1.10.0-b1 - April 28, 2025

Features

  • Allow copy_partitions in microbatch (#1414)
  • Add authentication method for Workload Identity Federation. (#593)
  • Add support for creating BigQuery labels from meta configuration (#889)
  • Support bigframes in dbt bigquery python model (#921)
  • Support timeout in bigframes mode of dbt python model (#949)
  • Support custom packages in bigframes mode (#978)
  • Support oauth secrets credential method in bigframes mode (#1026)

Fixes

  • Fix issue where dbt-bigquery was not retrying in certain retryable scenarios, e.g. 503's (#682)
  • Cast event_time to a timestamp prior to comparing against microbatch start/end time (#1422)
  • Fix issue where rate limit errors on table service calls are not retried (#1423)
  • Fix retry scenarios so that dbt always retries when BigQuery recommends a retry (#263)
  • Fix silently ignoring materialized view authorization (#1471)
  • Format the outputs of notebook execution (#943)
  • Create a new default runtime template (#950)
  • use compute_region instead of dataproc_region (#957)
  • Fix the credential method issue (#970)
  • Raise an error when notebook execution failed (#976)
  • datetime.datetime.utcnow() is deprecated as of Python 3.12 (#980)
  • Make location optional in bigframes (#1001)
  • Quick fix of session in table.sql (#1003)
  • set job_timeout_ms in job config so BQ cancels queries on timeout (#1004)

Under the Hood

  • Move from setup.py to pyproject.toml and to hatch as a dev tool (#1407)
  • replace DbtRuntime exception with DbtDatabase exception in dbt-bigquery connection exception handling (#798)
  • Added annotation for recording. (#761)

Dependencies

  • Permits newer versions of google-cloud-storage than 2.4.x to be used, defaults to latest (#931)

Contributors

Previous Releases

For information on prior major and minor releases, see their changelogs: