erts: Skip pollset update on scheduler-polled re-arm - #11525
Open
lpgauth wants to merge 1 commit into
Open
Conversation
Contributor
CT Test Results 3 files 136 suites 54m 0s ⏱️ Results for commit 6b57651. ♻️ This comment has been updated with latest results. To speed up review, make sure that you have read Contributing to Erlang/OTP and that all checks pass. See the TESTING and DEVELOPMENT HowTo guides for details about how to run test locally. Artifacts// Erlang/OTP Github Action Bot |
An enif_select fd migrated to the scheduler pollset stays registered for IN events across triggers, so a re-arming enif_select has nothing to change in either pollset. The scheduler pollset control was already short-circuited for this case, but the call still fell through to an erts_poll_control on the normal pollset: one epoll_ctl(MOD) per wakeup that changes nothing. Skip the pollset update entirely when re-arming a read select on an fd that is enabled in the scheduler pollset. The trigger handler still clears IN from state->events, so fds that stop being re-armed migrate back to a poll thread as before. With a pipelined request/response socket NIF workload (64 callers, 16 connections), epoll_ctl calls drop from one per wakeup (~156k per 350k requests) to connection setup only (~240), worth ~2% end-to-end throughput on 8 cores.
lpgauth
force-pushed
the
sched-poll-nif-rearm
branch
from
August 23, 2026 00:53
c3ef529 to
6b57651
Compare
garazdawi
approved these changes
Aug 24, 2026
Member
|
Nice find! The state machine in check io has become really complex... maybe it is time to do something about that. I should probably also spend some time to write some whitebox tests in order to catch these types of issues. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Since the scheduler pollset migration for enif_select (OTP 28), an fd
that keeps getting re-armed for reads by the same process is moved to
the scheduler pollset, where it stays registered for IN events and
re-arming is meant to be free. That never happens in practice: the
control path is entered on every re-arm and, while the scheduler
pollset part is already short-circuited, it still falls through to an
epoll_ctl(MOD) with an empty event set on the normal pollset. One
syscall per wakeup, changing nothing.
This skips the pollset update when re-arming a read select on an fd
that is enabled in the scheduler pollset. Nothing changes in either
pollset in that case: IN stays enabled in the scheduler pollset and
the normal pollset only carries the other events, which the call does
not touch. The trigger handler still clears IN from state->events, so
an fd that stops being re-armed migrates back to a poll thread as
before, and cancel/stop still go through the control path.
With a pipelined request/response socket NIF workload (64 callers,
16 connections), epoll_ctl calls drop from one per wakeup (~156k per
350k requests) to connection setup only (~240), worth ~2% end-to-end
throughput on 8 cores.