Skip to content

[BUG] [DBRX] persist_docs.columns=true with struct column causes error CREATE_VIEW_COLUMN_ARITY_MISMATCH #1299

@gdatascience-acorns

Description

@gdatascience-acorns

Describe the bug
When persist_docs.columns = true is configured (either at project or model level), dbt Fusion fails to create views containing struct columns in Databricks with a CREATE_VIEW_COLUMN_ARITY_MISMATCH.NOT_ENOUGH_DATA_COLUMNS error. The error occurs because Fusion expands struct fields in the view column definition (e.g., my_struct/field1, my_struct/field2) but the underlying query returns the struct as a single column (my_struct).

What version of dbt Fusion is this bug in? (find out by running dbt --version)
dbt-fusion 2.0.0-preview.110

Is this a discrepancy between the dbt Fusion Engine and dbt Core? Check one.

  • YES
  • NO

To Reproduce

{{
  config(
    materialized='view',
    persist_docs={"columns": true}
  )
}}

SELECT
  'test_id' AS id,
  STRUCT('value1' AS field1, 'value2' AS field2) AS my_struct

Expected behavior
Views with struct columns should be created successfully when persist_docs.columns = true, just as they are in dbt Core. The struct should remain as a single column in the view definition rather than being expanded into individual fields.

Alternatively, if struct expansion is intentional behavior, Fusion should handle the column arity correctly to match the expanded definition.

Screenshots
Shows a build when persist_docs.columns = true followed by a build when persist_docs.columns = false
Image

Operating System and CPU Type (please complete the following information):

  • Mac
  • ARM

Metadata

Metadata

Assignees

No one assigned

    Type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions