Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions docs/utilities.rst
Original file line number Diff line number Diff line change
Expand Up @@ -69,3 +69,9 @@ Below are the required and optional arguments utilizing Google Cloud Storage as
- No
- Defaults is 60 minutes.

.. _dbt:
=============
dbt Utilities
=============
.. autoclass:: parsons.utilities.dbt
:inherited-members:
327 changes: 0 additions & 327 deletions parsons/utilities/dbt.py

This file was deleted.

60 changes: 60 additions & 0 deletions parsons/utilities/dbt/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,60 @@
"""Utility for running and logging output from dbt commands

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These docs are good, but ideally we want to be able to surface them on the docs website. Maybe add a hook into the autodocs on this page? https://move-coop.github.io/parsons/html/stable/utilities.html#

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thiiink 8d56b8a should do this

Enable this utility by installing parsons with a dbt extra:
`pip install parsons[dbt-redshift]`
or `pip install parsons[dbt-postgres]`
or `pip install parsons[dbt-snowflake]`
or `pip install parsons[dbt-bigquery]`

To run dbt commands, you will need to have a dbt project directory
somewhere on the local filesystem.

The dbt command will inherit environment variables from the python
process shell, so if your dbt profiles.yml file uses environment
variables, ensure those are set in python or the parent shell before
running this dbt utility.

Logging is handled separately from the dbt run itself. The
dbtRunner.run method returns a dbtCommandResult object which can be
passed to a child class of dbtLogger for logging to stdout, slack,
etc.

Parsons provides a few example dbtLogger child classes, but for
best results, design your own!

Example usage:
```
from parsons.utilities.dbt import (
run_dbt_commands,
dbtLoggerSlack,
dbtLoggerPython
)

run_dbt_commands(
commands=['run', 'test'],
dbt_project_directory='/home/ubuntu/code/dbt_project/',
loggers=[
dbtLoggerPython,
dbtLoggerSlack(slack_webhook=os.environ['SLACK_WEBHOOK'])
]
)

```

"""

from parsons.utilities.dbt.dbt import run_dbt_commands
from parsons.utilities.dbt.logging import (
dbtLoggerMarkdown,
dbtLoggerPython,
dbtLoggerSlack,
dbtLoggerStdout,
)

__all__ = [
"run_dbt_commands",
"dbtLoggerMarkdown",
"dbtLoggerSlack",
"dbtLoggerStdout",
"dbtLoggerPython",
]
Loading
Loading