Skip to content

Support io_uring#2332

Open
yellowhatter wants to merge 91 commits into
eclipse-zenoh:mainfrom
ZettaScaleLabs:io_uring
Open

Support io_uring#2332
yellowhatter wants to merge 91 commits into
eclipse-zenoh:mainfrom
ZettaScaleLabs:io_uring

Conversation

@yellowhatter
Copy link
Copy Markdown
Contributor

@yellowhatter yellowhatter commented Dec 23, 2025

Summary

This PR introduces io_uring support to Zenoh behind the uring feature flag.

The implementation is intentionally scoped as an internal transport/runtime enhancement rather than a public API change. The patch adds a new internal crate, commons/zenoh-uring, and wires it into the workspace so the io_uring-specific memory, batching, reader, and writer logic can live outside the public surface area.

What this changes

Transport integration

  • Adds uring feature that is efficient only for Linux.
  • Adds io_uring support to the Zenoh transport stack.
  • Integrates the new I/O path with most links.
  • Adds automatic read-mode selection between io_uring and tokio.
  • Updates transport-side code paths to route io_uring operations through dedicated transport support code.

Link coverage

  • Adds support for streamed/socket-based links.
  • Keeps the existing transport model intact so the non-uring path still works normally.

Buffer and memory handling

  • Introduces a dedicated page-backed arena for uring buffers.
  • Allocates pinned memory with mmap/mlock and exposes it both as io_uring provided buffers and as registered iovec buffers.

Reader / writer path

  • Adds dedicated reader and writer implementations for the uring path.
  • Adds zero-copy oriented handling for split and fragmented payloads.
  • Improves RX batch reclamation so large payloads can be handled with less memory waste.
  • Tunes batch growth behavior to better support big messages and improve footprint.

Error handling and runtime integration

  • Passes Tokio executor context into the uring thread so Tokio-aware user code remains compatible.

Build / CI / test coverage

  • Adds the uring crate and dependency wiring to the workspace.
  • Ensures Zenoh still compiles when the uring feature is disabled.
  • Adds uring to CI.
  • Expands uring-related tests, including dynamic-port coverage and extended UDP test paths.

Design notes

The implementation follows a conservative enhancement approach:

  • no public API expansion,
  • no behavioral change for users who do not enable uring,
  • all uring-specific code isolated behind feature gates and internal crates,
  • transport and buffer-management changes kept close to the existing architecture instead of introducing a separate parallel stack.

Expected user impact

For users building with -F uring, this adds a new Linux I/O backend that reduces overhead in the transport path and improve handling of larger or fragmented payloads. For users who do not enable the feature, the existing build and transport behavior remain unchanged. The improvement gives 25-50% latency reduction and few % messaging rate increase. Performance tests included in comments to this PR below.

Validation

Validation for this PR is centered on:

  • successful compilation with and without uring,
  • CI coverage for the new feature,
  • transport-level tests for the new I/O path,
  • UDP and split-buffer / fragmentation scenarios,
  • correctness of buffer reclamation and RX finalization behavior.

🏷️ Label-Based Checklist

Based on the labels applied to this PR, please complete these additional requirements:

Labels: enhancement

✨ Enhancement Requirements

Since this PR enhances existing functionality:

  • Enhancement scope documented - Clear description of what is being improved
  • Minimum necessary code - Implementation is as simple as possible, doesn't overcomplicate the system
  • Backwards compatible - Existing code/APIs still work unchanged
  • No new APIs added - Only improving existing functionality
  • Tests updated - Existing tests pass, new test cases added if needed
  • Performance improvement measured - If applicable, before/after metrics provided
  • Documentation updated - Existing docs updated to reflect improvements
  • User impact documented - How users benefit from this enhancement

Remember: Enhancements should not introduce new APIs or breaking changes.

Instructions:

  1. Check off items as you complete them (change - [ ] to - [x])
  2. The PR checklist CI will verify these are completed

This checklist updates automatically when labels change, but preserves your checked boxes.

@yellowhatter yellowhatter self-assigned this Dec 23, 2025
@yellowhatter yellowhatter added the enhancement Existing things could work better label Dec 23, 2025
@codecov
Copy link
Copy Markdown

codecov Bot commented Feb 9, 2026

❌ 1 Tests Failed:

Tests completed Failed Passed Skipped
835 1 834 0
View the top 1 failed test(s) by shortest run time
zenoh-uring::reader::linux_tests::rw_parallel_and_start_stop::tcp_reader_first_exclusive_reader
Stack Traces | 2.85s run time
2 errors occurred executing test:
* process aborted with signal 6 (SIGABRT)
* thread '<unnamed>' (15276) panicked at .../core/src/panicking.rs:233:5:
  panic in a destructor during cleanup
  thread caused non-unwinding panic. aborting.

To view more test analytics, go to the Test Analytics Dashboard
📋 Got 3 mins? Take this short survey to help us improve Test Analytics.

- make growing RX batch memory (to optimize mem footprint and  support zero-copy for big payloads)
- exponential grow for uring batch arena
- tune RX arena size based on config
- add async wait for uring-global errors (as opposed to rx-task-related errors)
- supply uring thread with tokio executor context to be compatible with tokio-aware user code
Conflicts:
	io/zenoh-transport/src/unicast/universal/link.rs
@yellowhatter
Copy link
Copy Markdown
Contributor Author

100Hz:

main:
image

uring:
image

@yellowhatter
Copy link
Copy Markdown
Contributor Author

1kHz:

main:
image

uring:
image

@yellowhatter
Copy link
Copy Markdown
Contributor Author

max freq

main:
image

uring:
image

- support transport_compression for io_uring
- fix stats for io_uring
@yellowhatter yellowhatter marked this pull request as ready for review April 21, 2026 08:26
@yellowhatter yellowhatter changed the title WIP on io_uring Support io_uring Apr 21, 2026
@yellowhatter
Copy link
Copy Markdown
Contributor Author

Docs update: zenoh-rs/zenoh-web#127

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement Existing things could work better

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants