Skip to content

Livy connection support for Spark SQL models #991

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

mukeshkumarkulmi
Copy link

@mukeshkumarkulmi mukeshkumarkulmi commented Apr 11, 2025

resolves #483

Problem

Implemented Livy method to connect Livy server to submit Spark SQL through dbt-spark adapter and read results.

Solution

Use below in profile.yml

project:
target: dev
outputs:
 dev:
  type: spark_livy
  method: livy
  schema: schema
  host: Livy_server_url
  user: username
  password: password

@mukeshkumarkulmi mukeshkumarkulmi requested a review from a team as a code owner April 11, 2025 09:57
@cla-bot cla-bot bot added the cla:yes The PR author has signed the CLA label Apr 11, 2025
Copy link
Contributor

Thank you for your pull request! We could not find a changelog entry for this change in the dbt-spark package. For details on how to document a change, see the Contributing Guide.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cla:yes The PR author has signed the CLA
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant