Skip to content

Feature Request: Add built-in connection creation rate limiter with idle-retry capability for burst traffic #3890

Description

@wy92

Feature Request: Add built-in connection creation rate limiter with idle-retry capability for burst traffic

Background and Problem

Currently, when burst traffic arrives and all idle connections are exhausted, the connection pool allows all pending requests to initiate TCP dialing concurrently, as long as TotalConns < PoolSize. This causes a connection creation storm:

  • Hundreds of goroutines dial Redis at the same instant, causing TCP handshake spikes on both client and server side.
  • In scenarios where connection creation cost (e.g. ~10ms per dial) is much higher than per-request execution cost (e.g. ~1ms), creating large numbers of connections is far less efficient than waiting briefly and reusing existing connections.
  • The pool fills to PoolSize almost immediately, even for very short traffic spikes, after which most connections sit idle and eventually expire, creating periodic destroy-rebuild cycles.

Why a custom Dialer is not sufficient

The common community workaround is wrapping Dialer with a rate.Limiter, but this approach has a critical limitation:

Once a request passes the idle-check and enters the "new connection" branch, it will never go back to check for newly returned idle connections, even while waiting for a dial token. This means:

  1. Even if another request returns a connection to the idle pool a millisecond later, the waiting request will still create a brand new connection, wasting the opportunity for reuse.
  2. The rate limiter only slows down connection creation, it cannot redirect requests back to the idle queue, which is the more efficient path for short-lived requests.

This is suboptimal for workloads where reuse is cheaper than creation.

Proposed Solution

Add a built-in connection creation rate limiter to the connection pool with the following behavior:

  1. Add a new option, e.g. DialRateLimit / DialRateBurst, to control the maximum number of new connections created per second.
  2. When idle connections are exhausted and the pool would normally create a new connection, first acquire a token from the rate limiter.
  3. If a token is not immediately available, instead of blocking in the dial path, requeue the request to wait for an idle connection (the same waitForConn path used when TotalConns == PoolSize).
  4. When a connection is returned to the idle pool, wake up waiting requests as usual; if a waiting request can take the idle connection, it never needs to dial at all.
  5. Only if the request waits longer than some threshold (or if idle connections remain unavailable) does it proceed to create a new connection.

In short: when dialing would be throttled, prefer waiting for reuse over waiting to create.

Expected Benefits

  • Smooths out connection creation spikes during traffic bursts, eliminating dial storms.
  • Significantly improves overall efficiency for short-request workloads, where reuse is much cheaper than creation.
  • Reduces unnecessary connection churn caused by short traffic spikes.
  • Avoids the "can't go back to idle queue" limitation of custom Dialer-based rate limiting.
  • Remains fully backward compatible: disabled by default, only active when explicitly configured.

Additional Considerations

  • The implementation should work with both single-node and cluster clients.
  • The limiter should interact correctly with existing PoolTimeout semantics.
  • Metrics (PoolStats) should expose rate-limited dial attempts for observability.
  • This complements the existing ConnMaxLifetimeJitter feature: jitter prevents mass-expiration storms, while this feature prevents mass-creation storms.

Thank you for considering this feature. I'd be happy to discuss implementation details or contribute a draft PR if the maintainers think this is worthwhile.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions