`ExponentialRetry` timeout is off by a factor of `factor`

[shrekris-anyscale]

Background and Expected Behavior

ExponentialRetry takes in the following arguments:

class ExponentialRetry(RetryOptionsBase):
    def __init__(
        self,
        attempts: int = 3,  # How many times we should retry
        start_timeout: float = 0.1,  # Base timeout time, then it exponentially grow
        max_timeout: float = 30.0,  # Max possible timeout between tries
        factor: float = 2.0,  # How much we increase timeout each time
        statuses: Optional[Set[int]] = None,  # On which statuses we should retry
        exceptions: Optional[Set[Type[Exception]]] = None,  # On which exceptions we should retry
        retry_all_server_errors: bool = True,
        evaluate_response_callback: Optional[EvaluateResponseCallbackType] = None,
    ):

Based on the comments, I expect the first retry to happen start_timeout seconds after the original try. I expect the second retry to happen start_timeout * factor seconds after the first retry, and so on.

Observed Behavior

Instead, the first retry happens start_timeout * factor seconds after the original try. The second retry happens start_timeout * factor**2 seconds after that and so on. All the retries seem to be off by a factor of factor.

I confirmed this by logging requests I sent to a local server. I used this RetryClient:

RetryClient(retry_options=ExponentialRetry(attempts=1, start_timeout=1))

and I observed the following timings (note that the first retry happens 2 seconds after the first attempt– not 1 second):

2023-04-04 20:28:00,317 http_proxy 10.0.0.5 http_proxy.py:361 - Got request
2023-04-04 20:28:02,326 http_proxy 10.0.0.5 http_proxy.py:361 - Got request
2023-04-04 20:28:06,334 http_proxy 10.0.0.5 http_proxy.py:361 - Got request
2023-04-04 20:28:14,340 http_proxy 10.0.0.5 http_proxy.py:361 - Got request
2023-04-04 20:28:30,348 http_proxy 10.0.0.5 http_proxy.py:361 - Got request

Potential Cause

I believe the issue is that _RequestContext 1-indexes each attempt, but the math in ExponentialRetry expects 0-indexing.