Trim the server loop and batch sends - #141
Merged
Merged
Conversation
socket:send flattens iolists with list_to_binary on every call; sendv (OTP 27+) converts the iolist to an iovec and hands it to a single writev-style NIF call instead. Binaries keep using send.
Driven by eprof and perf under load (64 callers, pool of 16, HTTP test server via buoy): The backlog moves from an ETS table to an atomics array, with the ref reachable through a persistent_term; two ets:update_counter calls per request (6.8% of the profile) become two atomic ops. Per-request timeouts no longer create and cancel a BIF timer each (3.9%): the queue stores absolute deadlines and a single timer per server is armed to the earliest one. Replies never cancel it; an idle fire sweeps nothing and re-arms to the new minimum, so it fires at most once per timeout window. Server-side telemetry reads the enabled flag once per connection into the server state instead of hitting persistent_term four to five times per request (5.4% when disabled), and skips the iolist_size/byte_size measurement arguments when disabled. Casts already queued in the server mailbox are drained (up to 64) and written with one Protocol:send, cutting send NIF calls and syscalls under load. Socket messages are never reordered ahead of the casts that preceded them; UDP keeps one datagram per request. buoy benchmark (Linux, 8 client cores, telemetry off): shackle_tcp 222k -> 273k requests/s (-21% VM CPU per request), shackle_socket 261k -> 292k requests/s (-19%), combined with the socket:sendv change.
{otp, select_read} lands in OTP 28.0; on 27.3 the setopt returns
{error, {invalid, {socket_option, {otp, select_read}}}}.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Follow-up to #139, driven by eprof and Linux perf traces under load (64 callers, pool of 16, HTTP test server via buoy — see lpgauth/buoy#40).
Backlog on atomics. The per-server backlog moves from an ETS table to an atomics array reachable through a persistent_term; the two
ets:update_countercalls per request (6.8% of the profile) become two atomic ops, and the{Name, backlog}foil key goes away.Timeout sweep. Per-request timeouts no longer create and cancel a BIF timer each (3.9%): the queue stores absolute deadlines and a single timer per server is armed to the earliest one. Replies never cancel it; an idle fire sweeps nothing and re-arms to the new minimum, so it fires at most once per timeout window. A request with a shorter timeout than the armed deadline re-arms the timer, so precision is unchanged.
Telemetry flag in server state. Server-side telemetry reads the enabled flag once per connection instead of hitting persistent_term four to five times per request (5.4% when disabled), and skips the
iolist_size/byte_sizemeasurement arguments when disabled.Send batching. Casts already queued in the server mailbox are drained (up to 64) and written with one
Protocol:send, cutting send NIF calls and syscalls under load. Socket messages are never reordered ahead of the casts that preceded them, and UDP keeps one datagram per request.socket:sendv.
socket:sendflattens iolists withlist_to_binaryon every call;shackle_socketnow converts iolists to iovecs and sends them withsocket:sendv(OTP 27+, within shackle_socket's OTP 28 floor — #139 claimed 27.3, but {otp, select_read} lands in 28.0).buoy benchmark, Linux container, OTP 29.0.5 (marketops branch), 8 client cores, telemetry off, medians of 5 runs:
perf on the after-build shows the socket NIF path is now thin: the largest remaining esock cost is the oneshot select re-arm in
recv_check_select(~5% of client CPU, mostly oneepoll_ctlper wake), which needs a persistent-registration mode forenif_selecton the OTP side to remove.