Added socket configuration hook and improved the TCP send path - #3
Open
kenedos wants to merge 1 commit into
Open
Added socket configuration hook and improved the TCP send path#3kenedos wants to merge 1 commit into
kenedos wants to merge 1 commit into
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a way for consumers to configure the underlying socket, and an optional
way to combine queued sends into a single operation. Also fixes a latent
partial-send bug in
OnSend.All defaults are unchanged. Nagle's algorithm stays enabled, and send
coalescing is opt-in and off by default.
Three changes:
ConfigureSocket(Socket)— a virtual no-op hook onTcpConnectionandTcpClientBeginSendnow issues one gathered send instead of one send per queued itemSendCoalescingTime— optional, bounded, off by default1.
ConfigureSocket(Socket)TcpConnectionholds itsSocketin a private field and never exposes it, soconsumers can't set socket options. The most commonly needed one is
NoDelay.Added a virtual method called once the socket is available and before any data
moves — from
InitonTcpConnection, and before bothConnectandConnectAsynconTcpClient:The default implementation does nothing, so socket defaults are untouched
unless a consumer overrides it.
Motivation
Measured on a game server using Yggdrasil, against a client on a fixed 150ms
simulated link:
NoDelay = trueOnly the outbound direction was affected, because the client already sets
TCP_NODELAYon its own socket.A measurement that needs no clock alignment: the server sends one packet, waits
exactly 1200ms, then sends another. The client observed that pair 890-1119ms
apart before the change, and 1214ms after — matching the server exactly. The
first packet was being held back, and the application had no supported way to
prevent it.
The current workaround is reflection over the private
_socketfield, whichdepends on an implementation detail and breaks under trimming/NativeAOT.
2. Gathered sends, and a partial-send fix
BeginSendpreviously peeked a single item and issued oneBeginSendperqueued message. It now passes the whole queue to
Socket.BeginSend(IList<ArraySegment<byte>>, ...)as one operation. No copying,no added latency — purely fewer syscalls and fewer packets when several messages
are queued at once.
While making that change I hit an existing bug:
OnSendignoredEndSend'sreturn value and unconditionally dequeued one item. If a stream socket ever
completed a send partially, the unsent tail of that item was silently dropped,
corrupting the stream. It was unlikely with one small buffer per send, but
gathering makes larger sends routine and would have made it reachable.
OnSendnow tracks a byte offset, dequeues only fully-sent items, and resumesmid-item on the next send.
3.
SendCoalescingTimeDefaults to zero, which is the existing behaviour — send as soon as possible.
When set, the first
Sendof a burst arms a timer instead of sendingimmediately, and everything queued during that window goes out together.
This gives applications that emit bursts of small messages a way to reduce
packet count without Nagle's unbounded latency: the delay never exceeds the
configured value, because it doesn't wait for the remote host to acknowledge
anything.
The timer is created lazily, so connections that don't set the property never
allocate one. It's disposed on
Close.Defaults
Nothing changes for existing consumers unless they opt in:
ConfigureSocketis an empty virtual — Nagle stays onSendCoalescingTimedefaults toTimeSpan.Zero— current send timingpending messages costs