Skip to content

[Bug] base_location set as default not supported for Iceberg SNOWFLAKE_MANAGED #1911

Description

@CommonCrisis

Is this a new bug?

  • I believe this is a new bug
  • I have searched the existing issues, and I could not find an existing issue for this bug

Which packages are affected?

  • dbt-adapters
  • dbt-tests-adapter
  • dbt-athena
  • dbt-athena-community
  • dbt-bigquery
  • dbt-postgres
  • dbt-redshift
  • dbt-snowflake
  • dbt-spark

Current Behavior

Hi,

I set up a catalogs.yml in my dbt Snowflake project:

catalogs:
- name: catalog_horizon
  active_write_integration: snowflake_write_integration
  write_integrations:
  - name: snowflake_write_integration
    external_volume: SNOWFLAKE_MANAGED
    table_format: iceberg
    catalog_type: built_in
    adapter_properties:
      change_tracking: true

I want to use the preview feature here: https://docs.snowflake.com/en/user-guide/tables-iceberg-internal-storage
In the past I used an external volume on AWS but I don't like paying for the egress costs so I assume using the storage managed by Snowflake is ideal.

The problem is that this function by default returns a base_location which is not supported for external_volume set to SNOWFLAKE_MANAGED. Ideally the function checks if external_volume.lower() = "snowflake_managed" and if True does not set the base_locationat all.

    def get_iceberg_ddl_options(self, config: RelationConfig) -> str:
        # If the base_location_root config is supplied, overwrite the default value ("_dbt/")
        base_location: str = (
            f"{config.get('base_location_root', '_dbt')}/{self.schema}/{self.name}"  # type:ignore
        )

        if subpath := config.get("base_location_subpath"):  # type:ignore
            base_location += f"/{subpath}"

        external_volume = config.get("external_volume")  # type:ignore
        iceberg_ddl_predicates: str = f"""
        catalog = 'snowflake'
        base_location = '{base_location}'
        """
        if external_volume := config.get("external_volume"):  # type:ignore
            iceberg_ddl_predicates += f"\nexternal_volume = '{external_volume}'"
        return textwrap.indent(textwrap.dedent(iceberg_ddl_predicates), " " * 10)

Error Message: 004571 (0A000): SQL Compilation Error: BASE_LOCATION property is not supported for Iceberg tables using Snowflake Managed Storage

If I copy paste the SQL query created by dbt and remove the base_location it works perfectly fine.

Expected Behavior

If external_volume: SNOWFLAKE_MANAGED do not set BASE_LOCATION in the compiled model.

Steps To Reproduce

  1. Create a catalogs.yml as described above
  2. Create a dbt model with catalog_name = 'catalog_horizon', in the config
  3. Build model
  4. Copy paste the created sql and remove the base_location = '_dbt/... and execute manually in Snowflake
  5. This should be executed without any problems

Relevant log output

Environment

- OS: Windows 11
- Python: 3.13
- dbt-adapters: 1.22.9
- dbt-snowflake: 1.11.3

Additional Context

No response

Metadata

Metadata

Assignees

No one assigned

    Labels

    triage:productIn Product's queuetype:bugSomething isn't working as documented

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions