Skip to content

Commit 3a4c991

Browse files
authored
Async and base infrastructure for cross-process shared memory. (#23688)
This is the first half of the remote HAL transport stack, split out to land independently. Everything here is in `iree/async/`, `iree/base/`, and `iree/testing/` — no net or HAL changes yet. ### Shared memory primitives Adds `iree_shm_*` for cross-platform shared memory create/open/close/map with handle passing support (memfd on Linux, shm_open on macOS, CreateFileMappingW on Windows). Includes memory sealing via `iree_shm_seal()` for making regions immutable after population — uses kernel-level F_SEAL_* on Linux, VirtualProtect defense-in-depth on Windows, unavailable on macOS. On top of that, a lock-free SPSC queue (`iree_spsc_queue_*`) designed to operate on caller-provided memory so it can live in a shared memory region. Monotonically increasing 64-bit positions with acquire-release ordering, cache-line isolation between producer and consumer, skip markers for wrap-free reads. This is the data plane for the SHM carrier. ### Cross-process notification Extends the notification system to support shared-memory epochs and cross-process wake across all three backends. The core change is an `epoch_ptr` indirection so notifications can point at either the inline epoch (local, zero behavioral change) or a caller-provided epoch in shared memory. Per-platform wake mechanisms: - io_uring: shared futex (FUTEX mode) or caller-provided eventfd (EVENT mode) - POSIX/Linux: shared futex + caller eventfd for poll loop wake - POSIX/macOS: poll() on wake fd (no futex, no cross-process condvar) - IOCP: RegisterWaitForSingleObject bridges caller Event to IOCP; WakeByAddress is per-process (virtual-address keyed), so cross-process wake uses the SetEvent path ### Shared buffer pool Extends `iree_async_buffer_pool_t` with `create_shared`/`open_shared` for cross-process zero-copy. The atomic freelist (64-bit CAS, position-independent indices) lives directly in shared memory so both processes can independently acquire and release buffers. Header magic is written last as a commit step so openers never see a valid header with uninitialized freelist state. ### Proactor performance improvements - **IOCP event waits**: Replace polling-based event wait with `NtAssociateWaitCompletionPacket` for zero-overhead kernel-level event-to-IOCP association. - **IOCP carrier freelist**: Eliminates per-I/O heap allocation in steady state by recycling carrier structs through an atomic slist freelist. Pool grows on demand, drained at destroy. - **IOCP active_carriers**: Doubly-linked list for O(1) carrier removal (was O(n) list walk on every event wait completion). - **Timer insertion fast-path**: Tail comparison before list walk turns the common case (monotonically-increasing deadlines) from O(n) to O(1). - **Inline progress callbacks**: Proactors can register progress callbacks that run inline during the event loop, enabling adaptive polling strategies without full wakeup overhead. ### Bug fixes - **Send data lifetime in io_uring**: Data referenced by send SQEs submitted from recv callbacks could be a use-after-return if the submitting function's stack frame unwound before the kernel read the data. Fixed by copying send data inline when immediate submission is possible. New `data_lifetime_test` in the socket CTS. - **Axis failure propagation**: When an axis operation failed, the failure status was silently dropped instead of propagating to semaphores waiting on that axis value. ### Testing infrastructure Adds a coordinated multi-process test harness (`iree/testing/`). A single test binary re-executes itself in different roles, with the launcher orchestrating spawn order, readiness synchronization, and result collection. Supports Linux, macOS, and Windows. First consumer is the SHM carrier cross-process tests on the remote-hal branch. Also adds `iree_async_primitive_dup()`/`close()` for cross-process handle transfer — the async primitive layer's equivalents of dup()/close(), DuplicateHandle/CloseHandle, mach_port_mod_refs.
2 parents 16155fd + 0314c27 commit 3a4c991

64 files changed

Lines changed: 8504 additions & 323 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

runtime/src/iree/async/BUILD.bazel

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -44,6 +44,7 @@ iree_runtime_cc_library(
4444
"frontier_tracker.c",
4545
"notification.c",
4646
"operation.c",
47+
"primitive.c",
4748
"proactor.c",
4849
"region.c",
4950
"semaphore.c",
@@ -134,6 +135,17 @@ iree_runtime_cc_library(
134135
# Tests
135136
#===------------------------------------------------------------------------===#
136137

138+
iree_runtime_cc_test(
139+
name = "primitive_test",
140+
srcs = ["primitive_test.cc"],
141+
deps = [
142+
":async",
143+
"//runtime/src/iree/base",
144+
"//runtime/src/iree/testing:gtest",
145+
"//runtime/src/iree/testing:gtest_main",
146+
],
147+
)
148+
137149
iree_runtime_cc_test(
138150
name = "address_test",
139151
srcs = ["address_test.cc"],

runtime/src/iree/async/CMakeLists.txt

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -61,6 +61,7 @@ iree_cc_library(
6161
"frontier_tracker.c"
6262
"notification.c"
6363
"operation.c"
64+
"primitive.c"
6465
"proactor.c"
6566
"region.c"
6667
"semaphore.c"
@@ -116,6 +117,18 @@ iree_cc_library(
116117
PUBLIC
117118
)
118119

120+
iree_cc_test(
121+
NAME
122+
primitive_test
123+
SRCS
124+
"primitive_test.cc"
125+
DEPS
126+
::async
127+
iree::base
128+
iree::testing::gtest
129+
iree::testing::gtest_main
130+
)
131+
119132
iree_cc_test(
120133
NAME
121134
address_test

0 commit comments

Comments
 (0)