-
Notifications
You must be signed in to change notification settings - Fork 136
feat(csharp): Add retry-after behavior for 503 responses in Spark ADBC driver #2664
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for the change! I've left feedback, some more optional than others.
Can we hold off submitting this until a decision is made on #2672? |
6906845
to
0b4526b
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for the change! Please make sure the linter doesn't complain about whitespace issues and that the retry flag is properly handled.
There are some merge conflicts, so this will probably need to be rebased. |
Update DatabricksConnection.cs Update RetryHttpHandlerTest.cs Add retry after handling in ADBC Spark driver fix pre commit check failures fix build error address PR comments fix linter lint Update DatabricksConnection.cs address comments
9acda68
to
b89d9be
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks!
…C driver (apache#2664) ## Description This PR implements retry-after behavior for the Spark ADBC driver when receiving 503 responses with Retry-After headers. This is particularly useful for Databricks clusters that may return 503 responses when a cluster is starting up or experiencing temporary unavailability. ## Changes - Added new configuration parameters: - `adbc.spark.temporarily_unavailable_retry` (default: 1 - enabled) - `adbc.spark.temporarily_unavailable_retry_timeout` (default: 900 seconds) - Created a `RetryHttpHandler` class that wraps the existing `HttpClientHandler` to handle 503 responses - Modified `SparkHttpConnection` to use the new retry handler - Added comprehensive unit tests for the retry behavior ## Implementation Details When a 503 response with a Retry-After header is received: 1. The handler will wait for the number of seconds specified in the header 2. It will then retry the request 3. If another 503 response is received, it will continue retrying 4. If the total retry time exceeds the configured timeout, it will fail with an appropriate error message ## Testing Added unit tests to verify: - Retry behavior for 503 responses with Retry-After headers - Timeout behavior when retry time exceeds the configured limit - Handling of invalid or missing Retry-After headers - Disabling retry behavior via configuration - Parameter validationv
Description
This PR implements retry-after behavior for the Spark ADBC driver when receiving 503 responses with Retry-After headers. This is particularly useful for Databricks clusters that may return 503 responses when a cluster is starting up or experiencing temporary unavailability.
Changes
adbc.spark.temporarily_unavailable_retry
(default: 1 - enabled)adbc.spark.temporarily_unavailable_retry_timeout
(default: 900 seconds)RetryHttpHandler
class that wraps the existingHttpClientHandler
to handle 503 responsesSparkHttpConnection
to use the new retry handlerImplementation Details
When a 503 response with a Retry-After header is received:
Testing
Added unit tests to verify: