[Python] Fix flaky test test_tls_with_self_signed_certificate_succeeds #4946

[hank95179]

Issue link

This Pull Request is linked to issue (URL): #4946

Description

This PR addresses the flakiness observed in test_tls_with_self_signed_certificate_succeeds.
In high-load environments (such as CI/CD), the TLS handshake process occasionally times out due to resource contention, causing ClosingError: ... timed out failures.

Solution

Implemented a retry mechanism for client creation in tests/async_tests/test_tls_certificates.py.

• Implemented a retry mechanism for client creation in tests/async_tests/test_tls_certificates.py.
  The test now attempts to establish the connection up to 3 times with an exponential backoff (2^i seconds) before failing.
• Applied this logic to both Cluster and Standalone modes.

Verification

Verified locally by simulating connection latency. The retry logic successfully allows the test to pass even when initial connection attempts fail due to transient timeouts.

[xShinnRyuu]

In your description under Solution you say the fix has a 1-second backoff. Your implementation uses time.sleep(2**i) within your retry loop, which is a duration equal to 2 raised to the power of i seconds.

I'm unsure if that is what was intended but an exponential backoff is usually the norm for retry. Ideally, its a not that high of an exponential, because if there were more retires it would take a very long time. But three retries is fine here with base 2 exponential.

Please update your description, or adjust your implementation. Otherwise, it generally looks good.

[hank95179]

@xShinnRyuu Thank you! I've fixed the description.

[xShinnRyuu]

LGTM

[currantw]

Approved with one suggestion. Thanks for your help!

[currantw]

Nit. Similar to other pull request, I think it might make sense to extract to a helper and add a comment explaining why this is needed.

[jduo]

@hank95179 , I agree with @currantw here, especially given this is happening across multiple tests. Would you be able to add the helpers in one of your PRs and use them in the other.

Thanks.