Skip to content

Elementary =~0.18.0 does not seem to fully support microbatches in dbt v1.9 #1894

Open
@sarah-lier

Description

@sarah-lier

Describe the bug
dbt v1.9 has introduced a new feature for incremental models called microbatches which may generate a dbt status called partial success, if the run was only partially successful.

When we implemented the microbatch strategy for one of our models on Saturday (5th of April), the new dbt status caused elementary to fall over (s. first screenshot). Since the latest elementary patch release on Sunday or Monday, the error message output has changed but it seemed to refer to the same issue (s. second screenshot).

It looks like elementary 0.18.0 does not seem to fully support microbatches in dbt v1.9.

To Reproduce
Steps to reproduce the behavior:

  1. Create an incremental dbt model with microbatch strategy.
  2. Run the model but it interrupt it halfway through to cause a partial success status.
  3. Run elementary.

Expected behavior
We expect elementary to run successfully and handle the new dbt status that comes with microbatches.

Image Image

Screenshots
If applicable, add screenshots to help explain your problem.

Environment (please complete the following information):

  • Elementary CLI (edr) version: ~=0.18.0
  • Elementary dbt package version: [">=0.18.0", "<0.19.0"]
  • dbt version you're using: 1.9.3
  • Data warehouse: Databricks
  • Infrastructure details: Prod environment

Additional context
Add any other context about the problem here.

Would you be willing to contribute a fix for this issue?
Let us know if you're willing to contribute so that we'll be able to provide guidance.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions