fix: Set PooledConnectionLifetime for DNS re-resolution on long-lived connections - #17
Merged
Merged
Conversation
… connections GrpcChannel.ForAddress creates long-lived HTTP/2 connections via SocketsHttpHandler. By default, PooledConnectionLifetime is infinite, meaning connections are never recycled and DNS is never re-resolved. For clients connecting to load-balanced endpoints (e.g. AWS ALBs), backend IPs can change while the HTTP/2 connection remains healthy, leaving the client stuck on a stale IP. Set PooledConnectionLifetime to 5 minutes so that connections are periodically recycled, triggering fresh DNS resolution.
There was a problem hiding this comment.
Pull request overview
This PR addresses stale DNS resolution for long-lived gRPC HTTP/2 connections by configuring connection recycling via SocketsHttpHandler.PooledConnectionLifetime.
Changes:
- Set
PooledConnectionLifetime = TimeSpan.FromMinutes(5)on the unauthenticated TLSSocketsHttpHandler. - Set
PooledConnectionLifetime = TimeSpan.FromMinutes(5)on the authenticated TLSSocketsHttpHandler.
Comments suppressed due to low confidence (1)
Spice/src/Flight/SpiceFlightClient.cs:94
PooledConnectionLifetimeis only set whenuseTlsis true (theSocketsHttpHandlerpath). WhenuseTlsis false the code falls back toHttpClientHandler, so the channel will still use the default (infinite) connection lifetime and won’t re-resolve DNS over time. If the DNS-staleness issue can also affect non-TLS/h2c usage, consider using aSocketsHttpHandlerin the non-TLS branch as well so the lifetime can be configured consistently.
if (useTls)
{
messageHandler = new SocketsHttpHandler
{
EnableMultipleHttp2Connections = true,
// Force periodic connection recycling to trigger DNS re-resolution.
// Without this, HTTP/2 connections are kept alive indefinitely and
// the client can get stuck on stale IPs when backend targets change
// (e.g. AWS ALB target rotation).
PooledConnectionLifetime = TimeSpan.FromMinutes(5),
};
}
else
#endif
{
messageHandler = new HttpClientHandler();
}
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
peasee
approved these changes
Mar 25, 2026
2 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
GrpcChannel.ForAddresscreates long-lived HTTP/2 connections viaSocketsHttpHandler. By default,PooledConnectionLifetimeis infinite — connections are never recycled and DNS is never re-resolved.For clients connecting to load-balanced endpoints (e.g. AWS ALBs), backend IPs can change while the HTTP/2 connection remains healthy, leaving the client stuck on a stale IP.
Fix
Set
PooledConnectionLifetime = TimeSpan.FromMinutes(5)on bothSocketsHttpHandlerinstances (unauthenticated and authenticated TLS paths). This forces periodic connection recycling, which triggers fresh DNS resolution.This is the recommended .NET approach for handling DNS changes with long-lived
HttpClient/ gRPC connections.Related