Add ability to limit both total dur and num retries

[ThomWright]

Changes

The previous build_with_total_retry_duration_and_max_retries function used a calculated max. retries.

This PR adds a function to specify the max. retries, along with a max. total duration.

It also renames build_with_total_retry_duration_and_max_retries to build_with_total_retry_duration_and_limit_retries which I think fits a bit better. This is a breaking change.

Rationale

We often have latency targets we need to meet, and we shouldn't continue retrying beyond that target.

We also often want to limit the number of retries to avoid overloading the system we're calling.

In other words, we might want to:

• Retry up to X times
• Retry for no longer than a latency target (e.g. 150ms)

Consider a retry policy like the following:

• Min. interval: 25ms
• Max. interval: 100ms
• Max. retries: 3
• Base: 3
• Max total duration: 150ms

Our maximum intervals (i.e. without jitter) would be:

1. 25ms
2. 75ms
3. 100ms

For a total of 200ms. In this case, the max. total duration would kick in and prevent the third retry.

With jitter, however, our minimum intervals would be:

1. 25ms
2. 25ms
3. 25ms

For a total of 75ms. In this case, we would retry 3 times.

Other notes

• With BoundedJitter I believe the first retry will always be the same duration (no jitter). This seems like a problem.