feat(athena): add disable_batch_fallback config option - #3
Open
dtaniwaki wants to merge 3 commits into
Open
Conversation
…ITIONS behavior When batch_fallback=False, TOO_MANY_OPEN_PARTITIONS errors are raised instead of silently falling back to batch mode. Default is True (preserving existing behavior).
Align with other bool configs by using False as the default value. disable_batch_fallback=True raises TOO_MANY_OPEN_PARTITIONS instead of silently falling back to batch mode.
dtaniwaki
added a commit
that referenced
this pull request
Apr 9, 2026
PR dbt-labs#1830/dbt-labs#1832 のテストが fork #3 で追加された disable_batch_fallback 引数を MockAdapter.run_query_with_partitions_limit_catching に含んでいなかった。
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
disable_batch_fallbackconfig option (defaultFalse) to dbt-athenaTrue,TOO_MANY_OPEN_PARTITIONSerrors are raised instead of silently falling back to batch insert modeMotivation
Currently, dbt-athena automatically falls back to batch mode when Athena returns
TOO_MANY_OPEN_PARTITIONSon partitioned tables. This is convenient but can mask issues silently. The newdisable_batch_fallbackoption gives users explicit control to fail fast when this error occurs.Usage
Note:
TOO_MANY_OPEN_PARTITIONSonly occurs on partitioned tables. This config has no effect on non-partitioned tables.Test plan
run_query_with_partitions_limit_catchingwithdisable_batch_fallback=TrueandFalse