Skip to content

Added socket configuration hook and improved the TCP send path - #3

Open
kenedos wants to merge 1 commit into
exectails:masterfrom
kenedos:tcp_send_path
Open

Added socket configuration hook and improved the TCP send path#3
kenedos wants to merge 1 commit into
exectails:masterfrom
kenedos:tcp_send_path

Conversation

@kenedos

@kenedos kenedos commented Aug 19, 2026

Copy link
Copy Markdown

Summary

Adds a way for consumers to configure the underlying socket, and an optional
way to combine queued sends into a single operation. Also fixes a latent
partial-send bug in OnSend.

All defaults are unchanged. Nagle's algorithm stays enabled, and send
coalescing is opt-in and off by default.

Three changes:

  1. ConfigureSocket(Socket) — a virtual no-op hook on TcpConnection and TcpClient
  2. BeginSend now issues one gathered send instead of one send per queued item
  3. SendCoalescingTime — optional, bounded, off by default

1. ConfigureSocket(Socket)

TcpConnection holds its Socket in a private field and never exposes it, so
consumers can't set socket options. The most commonly needed one is
NoDelay.

Added a virtual method called once the socket is available and before any data
moves — from Init on TcpConnection, and before both Connect and
ConnectAsync on TcpClient:

protected virtual void ConfigureSocket(Socket socket)
{
}

The default implementation does nothing, so socket defaults are untouched
unless a consumer overrides it.

Motivation

Measured on a game server using Yggdrasil, against a client on a fixed 150ms
simulated link:

direction before after NoDelay = true
client -> server 152ms 152ms
server -> client 454-545ms 204-226ms

Only the outbound direction was affected, because the client already sets
TCP_NODELAY on its own socket.

A measurement that needs no clock alignment: the server sends one packet, waits
exactly 1200ms, then sends another. The client observed that pair 890-1119ms
apart before the change, and 1214ms after — matching the server exactly. The
first packet was being held back, and the application had no supported way to
prevent it.

The current workaround is reflection over the private _socket field, which
depends on an implementation detail and breaks under trimming/NativeAOT.

2. Gathered sends, and a partial-send fix

BeginSend previously peeked a single item and issued one BeginSend per
queued message. It now passes the whole queue to
Socket.BeginSend(IList<ArraySegment<byte>>, ...) as one operation. No copying,
no added latency — purely fewer syscalls and fewer packets when several messages
are queued at once.

While making that change I hit an existing bug: OnSend ignored EndSend's
return value and unconditionally dequeued one item. If a stream socket ever
completed a send partially, the unsent tail of that item was silently dropped,
corrupting the stream. It was unlikely with one small buffer per send, but
gathering makes larger sends routine and would have made it reachable.

OnSend now tracks a byte offset, dequeues only fully-sent items, and resumes
mid-item on the next send.

3. SendCoalescingTime

public TimeSpan SendCoalescingTime { get; set; } = TimeSpan.Zero;

Defaults to zero, which is the existing behaviour — send as soon as possible.
When set, the first Send of a burst arms a timer instead of sending
immediately, and everything queued during that window goes out together.

This gives applications that emit bursts of small messages a way to reduce
packet count without Nagle's unbounded latency: the delay never exceeds the
configured value, because it doesn't wait for the remote host to acknowledge
anything.

The timer is created lazily, so connections that don't set the property never
allocate one. It's disposed on Close.


Defaults

Nothing changes for existing consumers unless they opt in:

  • ConfigureSocket is an empty virtual — Nagle stays on
  • SendCoalescingTime defaults to TimeSpan.Zero — current send timing
  • Gathered sends add no latency; they only affect how many syscalls a queue of
    pending messages costs

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant