Skip to content

Improve Table Materialization to Minimize Downtime #265

@gjvanvuuren

Description

@gjvanvuuren

As addressed in dbt-fabric issue 189, the fabric adapter's table materialization was changed to:

  1. load the data into a temp table and then
  2. to swap the temp table with the destination table.

This prevents the destination table from being offline, possibly for longer periods, while the data loads.

The dbt-synapse adapter (v1.8.1) still drops the target table and creates a new table during table materialization. This is creating errors, because the target table disappears for minutes, causing other queries and reports to fail.

Would it be possible to merge the above update from dbt-fabric into dbt-synapse?

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions