Skip to content

Trim the server loop and batch sends - #141

Merged
lpgauth merged 3 commits into
masterfrom
hot-path-opts-2
Aug 23, 2026
Merged

Trim the server loop and batch sends#141
lpgauth merged 3 commits into
masterfrom
hot-path-opts-2

Conversation

@lpgauth

@lpgauth lpgauth commented Aug 22, 2026

Copy link
Copy Markdown
Owner

Summary

Follow-up to #139, driven by eprof and Linux perf traces under load (64 callers, pool of 16, HTTP test server via buoy — see lpgauth/buoy#40).

Backlog on atomics. The per-server backlog moves from an ETS table to an atomics array reachable through a persistent_term; the two ets:update_counter calls per request (6.8% of the profile) become two atomic ops, and the {Name, backlog} foil key goes away.

Timeout sweep. Per-request timeouts no longer create and cancel a BIF timer each (3.9%): the queue stores absolute deadlines and a single timer per server is armed to the earliest one. Replies never cancel it; an idle fire sweeps nothing and re-arms to the new minimum, so it fires at most once per timeout window. A request with a shorter timeout than the armed deadline re-arms the timer, so precision is unchanged.

Telemetry flag in server state. Server-side telemetry reads the enabled flag once per connection instead of hitting persistent_term four to five times per request (5.4% when disabled), and skips the iolist_size/byte_size measurement arguments when disabled.

Send batching. Casts already queued in the server mailbox are drained (up to 64) and written with one Protocol:send, cutting send NIF calls and syscalls under load. Socket messages are never reordered ahead of the casts that preceded them, and UDP keeps one datagram per request.

socket:sendv. socket:send flattens iolists with list_to_binary on every call; shackle_socket now converts iolists to iovecs and sends them with socket:sendv (OTP 27+, within shackle_socket's OTP 28 floor — #139 claimed 27.3, but {otp, select_read} lands in 28.0).

buoy benchmark, Linux container, OTP 29.0.5 (marketops branch), 8 client cores, telemetry off, medians of 5 runs:

protocol before after
shackle_tcp 237k req/s, 15.1us CPU/req 265k req/s, 12.2us (-19%)
shackle_socket 265k req/s, 10.4us CPU/req 287k req/s, 8.4us (-19%)

perf on the after-build shows the socket NIF path is now thin: the largest remaining esock cost is the oneshot select re-arm in recv_check_select (~5% of client CPU, mostly one epoll_ctl per wake), which needs a persistent-registration mode for enif_select on the OTP side to remove.

socket:send flattens iolists with list_to_binary on every call;
sendv (OTP 27+) converts the iolist to an iovec and hands it to a
single writev-style NIF call instead. Binaries keep using send.
Driven by eprof and perf under load (64 callers, pool of 16, HTTP
test server via buoy):

The backlog moves from an ETS table to an atomics array, with the
ref reachable through a persistent_term; two ets:update_counter
calls per request (6.8% of the profile) become two atomic ops.

Per-request timeouts no longer create and cancel a BIF timer each
(3.9%): the queue stores absolute deadlines and a single timer per
server is armed to the earliest one. Replies never cancel it; an
idle fire sweeps nothing and re-arms to the new minimum, so it
fires at most once per timeout window.

Server-side telemetry reads the enabled flag once per connection
into the server state instead of hitting persistent_term four to
five times per request (5.4% when disabled), and skips the
iolist_size/byte_size measurement arguments when disabled.

Casts already queued in the server mailbox are drained (up to 64)
and written with one Protocol:send, cutting send NIF calls and
syscalls under load. Socket messages are never reordered ahead of
the casts that preceded them; UDP keeps one datagram per request.

buoy benchmark (Linux, 8 client cores, telemetry off):
shackle_tcp 222k -> 273k requests/s (-21% VM CPU per request),
shackle_socket 261k -> 292k requests/s (-19%), combined with the
socket:sendv change.
{otp, select_read} lands in OTP 28.0; on 27.3 the setopt returns
{error, {invalid, {socket_option, {otp, select_read}}}}.
@lpgauth
lpgauth merged commit 28519b8 into master Aug 23, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant