Skip to content

Add support for Spark in Azure Synapse Analytics #266

@sdurry

Description

@sdurry

Add native support to the adapter to send workloads to Synapse Spark Pools, in addition to already supported dedicated SQL Pools.

We've heard numerous requests from dbt Cloud users and interested teams that they would like to use dbt with Synpase, not only with dedicated SQL pools, but also Spark Pools, for them to choose the right option depending on the need. It may be cost optimisation, or architectural decisions - but flexibility is greatly desired.

Similar work has been done for dbt-fabric already (See microsoft/dbt-fabricspark). Yet, ideally this could be unified under a single adapter to provide a simpler experience to developers.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions