Skip to content

fix: Set PooledConnectionLifetime for DNS re-resolution on long-lived connections - #17

Merged
phillipleblanc merged 1 commit into
trunkfrom
phillipleblanc/fix-dns-re-resolution
Mar 25, 2026
Merged

fix: Set PooledConnectionLifetime for DNS re-resolution on long-lived connections#17
phillipleblanc merged 1 commit into
trunkfrom
phillipleblanc/fix-dns-re-resolution

Conversation

@phillipleblanc

Copy link
Copy Markdown
Contributor

Problem

GrpcChannel.ForAddress creates long-lived HTTP/2 connections via SocketsHttpHandler. By default, PooledConnectionLifetime is infinite — connections are never recycled and DNS is never re-resolved.

For clients connecting to load-balanced endpoints (e.g. AWS ALBs), backend IPs can change while the HTTP/2 connection remains healthy, leaving the client stuck on a stale IP.

Fix

Set PooledConnectionLifetime = TimeSpan.FromMinutes(5) on both SocketsHttpHandler instances (unauthenticated and authenticated TLS paths). This forces periodic connection recycling, which triggers fresh DNS resolution.

This is the recommended .NET approach for handling DNS changes with long-lived HttpClient / gRPC connections.

Related

… connections

GrpcChannel.ForAddress creates long-lived HTTP/2 connections via
SocketsHttpHandler. By default, PooledConnectionLifetime is infinite,
meaning connections are never recycled and DNS is never re-resolved.

For clients connecting to load-balanced endpoints (e.g. AWS ALBs),
backend IPs can change while the HTTP/2 connection remains healthy,
leaving the client stuck on a stale IP.

Set PooledConnectionLifetime to 5 minutes so that connections are
periodically recycled, triggering fresh DNS resolution.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR addresses stale DNS resolution for long-lived gRPC HTTP/2 connections by configuring connection recycling via SocketsHttpHandler.PooledConnectionLifetime.

Changes:

  • Set PooledConnectionLifetime = TimeSpan.FromMinutes(5) on the unauthenticated TLS SocketsHttpHandler.
  • Set PooledConnectionLifetime = TimeSpan.FromMinutes(5) on the authenticated TLS SocketsHttpHandler.
Comments suppressed due to low confidence (1)

Spice/src/Flight/SpiceFlightClient.cs:94

  • PooledConnectionLifetime is only set when useTls is true (the SocketsHttpHandler path). When useTls is false the code falls back to HttpClientHandler, so the channel will still use the default (infinite) connection lifetime and won’t re-resolve DNS over time. If the DNS-staleness issue can also affect non-TLS/h2c usage, consider using a SocketsHttpHandler in the non-TLS branch as well so the lifetime can be configured consistently.
        if (useTls)
        {
            messageHandler = new SocketsHttpHandler
            {
                EnableMultipleHttp2Connections = true,
                // Force periodic connection recycling to trigger DNS re-resolution.
                // Without this, HTTP/2 connections are kept alive indefinitely and
                // the client can get stuck on stale IPs when backend targets change
                // (e.g. AWS ALB target rotation).
                PooledConnectionLifetime = TimeSpan.FromMinutes(5),
            };
        }
        else
#endif
        {
            messageHandler = new HttpClientHandler();
        }

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread Spice/src/Flight/SpiceFlightClient.cs
Comment thread Spice/src/Flight/SpiceFlightClient.cs
@phillipleblanc
phillipleblanc merged commit 0326918 into trunk Mar 25, 2026
18 checks passed
@phillipleblanc
phillipleblanc deleted the phillipleblanc/fix-dns-re-resolution branch March 25, 2026 07:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants