fix: handle zero values for max_retries, initial_backoff, and max_backoff in OpenSearch index_documents - #3432
Open
hsusul wants to merge 1 commit into
Open
Conversation
…koff in OpenSearch index_documents
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Fixes an issue in
awswrangler.opensearch.index_documentswhere passing0for retry and backoff configuration arguments (max_retries=0,initial_backoff=0,max_backoff=0) resulted in incorrect default fallback assignment and validation bypass.Root Cause
if use_threads and any([max_retries, initial_backoff, max_backoff]):evaluatedany([0, None, None])asFalsebecause0is falsey in Python. As a result, passingmax_retries=0whenuse_threads=Truefailed to triggerexceptions.InvalidArgumentCombination.use_threads=False,bulk_kwargs["max_retries"] = 5 if not max_retries else max_retriesevaluatednot 0asTrue, overridingmax_retries=0to5. Similarly,initial_backoff=0was overridden to2andmax_backoff=0was overridden to600.Solution
use_threadsvalidation check toany(x is not None for x in (max_retries, initial_backoff, max_backoff))to properly check for explicit argument presence.bulk_kwargsassignments to checkx is Noneinstead ofnot x, ensuring explicit zero values (0) are preserved.Validation Results
poetry run pytest tests/unit/test_opensearch.py -k "test_index_documents_zero_retries"passed cleanly.poetry run pytest tests/unit/test_moto.pypassed cleanly (46 passed).poetry run ruff checkandpoetry run ruff format --checkpassed cleanly.AWS Integration Tests Not Run
tests/unit/test_opensearch.pyrequiring live AWS OpenSearch domain infrastructure were not run as they require live AWS credentials and CloudFormation stack resources.By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.