Commit 3a4c991
authored
Async and base infrastructure for cross-process shared memory. (#23688)
This is the first half of the remote HAL transport stack, split out to
land independently. Everything here is in `iree/async/`, `iree/base/`,
and `iree/testing/` — no net or HAL changes yet.
### Shared memory primitives
Adds `iree_shm_*` for cross-platform shared memory create/open/close/map
with handle passing support (memfd on Linux, shm_open on macOS,
CreateFileMappingW on Windows). Includes memory sealing via
`iree_shm_seal()` for making regions immutable after population — uses
kernel-level F_SEAL_* on Linux, VirtualProtect defense-in-depth on
Windows, unavailable on macOS.
On top of that, a lock-free SPSC queue (`iree_spsc_queue_*`) designed to
operate on caller-provided memory so it can live in a shared memory
region. Monotonically increasing 64-bit positions with acquire-release
ordering, cache-line isolation between producer and consumer, skip
markers for wrap-free reads. This is the data plane for the SHM carrier.
### Cross-process notification
Extends the notification system to support shared-memory epochs and
cross-process wake across all three backends. The core change is an
`epoch_ptr` indirection so notifications can point at either the inline
epoch (local, zero behavioral change) or a caller-provided epoch in
shared memory.
Per-platform wake mechanisms:
- io_uring: shared futex (FUTEX mode) or caller-provided eventfd (EVENT
mode)
- POSIX/Linux: shared futex + caller eventfd for poll loop wake
- POSIX/macOS: poll() on wake fd (no futex, no cross-process condvar)
- IOCP: RegisterWaitForSingleObject bridges caller Event to IOCP;
WakeByAddress is per-process (virtual-address keyed), so cross-process
wake uses the SetEvent path
### Shared buffer pool
Extends `iree_async_buffer_pool_t` with `create_shared`/`open_shared`
for cross-process zero-copy. The atomic freelist (64-bit CAS,
position-independent indices) lives directly in shared memory so both
processes can independently acquire and release buffers. Header magic is
written last as a commit step so openers never see a valid header with
uninitialized freelist state.
### Proactor performance improvements
- **IOCP event waits**: Replace polling-based event wait with
`NtAssociateWaitCompletionPacket` for zero-overhead kernel-level
event-to-IOCP association.
- **IOCP carrier freelist**: Eliminates per-I/O heap allocation in
steady state by recycling carrier structs through an atomic slist
freelist. Pool grows on demand, drained at destroy.
- **IOCP active_carriers**: Doubly-linked list for O(1) carrier removal
(was O(n) list walk on every event wait completion).
- **Timer insertion fast-path**: Tail comparison before list walk turns
the common case (monotonically-increasing deadlines) from O(n) to O(1).
- **Inline progress callbacks**: Proactors can register progress
callbacks that run inline during the event loop, enabling adaptive
polling strategies without full wakeup overhead.
### Bug fixes
- **Send data lifetime in io_uring**: Data referenced by send SQEs
submitted from recv callbacks could be a use-after-return if the
submitting function's stack frame unwound before the kernel read the
data. Fixed by copying send data inline when immediate submission is
possible. New `data_lifetime_test` in the socket CTS.
- **Axis failure propagation**: When an axis operation failed, the
failure status was silently dropped instead of propagating to semaphores
waiting on that axis value.
### Testing infrastructure
Adds a coordinated multi-process test harness (`iree/testing/`). A
single test binary re-executes itself in different roles, with the
launcher orchestrating spawn order, readiness synchronization, and
result collection. Supports Linux, macOS, and Windows. First consumer is
the SHM carrier cross-process tests on the remote-hal branch.
Also adds `iree_async_primitive_dup()`/`close()` for cross-process
handle transfer — the async primitive layer's equivalents of
dup()/close(), DuplicateHandle/CloseHandle, mach_port_mod_refs.64 files changed
Lines changed: 8504 additions & 323 deletions
File tree
- runtime/src/iree
- async
- cts
- buffer
- socket
- sync
- operations
- platform
- io_uring
- cts
- iocp
- cts
- posix
- cts
- util
- base
- internal
- threading
- testing
Some content is hidden
Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
44 | 44 | | |
45 | 45 | | |
46 | 46 | | |
| 47 | + | |
47 | 48 | | |
48 | 49 | | |
49 | 50 | | |
| |||
134 | 135 | | |
135 | 136 | | |
136 | 137 | | |
| 138 | + | |
| 139 | + | |
| 140 | + | |
| 141 | + | |
| 142 | + | |
| 143 | + | |
| 144 | + | |
| 145 | + | |
| 146 | + | |
| 147 | + | |
| 148 | + | |
137 | 149 | | |
138 | 150 | | |
139 | 151 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
61 | 61 | | |
62 | 62 | | |
63 | 63 | | |
| 64 | + | |
64 | 65 | | |
65 | 66 | | |
66 | 67 | | |
| |||
116 | 117 | | |
117 | 118 | | |
118 | 119 | | |
| 120 | + | |
| 121 | + | |
| 122 | + | |
| 123 | + | |
| 124 | + | |
| 125 | + | |
| 126 | + | |
| 127 | + | |
| 128 | + | |
| 129 | + | |
| 130 | + | |
| 131 | + | |
119 | 132 | | |
120 | 133 | | |
121 | 134 | | |
| |||
0 commit comments