perf(datagrams): add batch send/recv APIs (N2+N4)#735
Open
ifdario wants to merge 8 commits into
Open
Conversation
Add Connection::send_datagrams / read_datagrams and the underlying
DatagramState::recv_many / Datagrams::send_many to amortize the
per-datagram connection-mutex cost, which is the dominant overhead at
high pps for a mesh VPN forwarding bursts of TUN packets.
- DatagramState::recv_many drains all buffered incoming datagrams in one
VecDeque drain instead of N pop_fronts.
- Datagrams::send_many queues a whole batch under one logical operation:
peer-support check once, drop-oldest backpressure pass once, oversized
datagrams skipped individually (not aborting the batch).
- Connection::send_datagrams takes the mutex once, wakes the driver once.
- Connection::read_datagrams is a ReadDatagrams future that parks on the
datagram_received notify then drains everything in one lock hold.
- New SendMany { queued, rejected } result struct.
- 2 new unit tests: batch send/recv roundtrip, oversized-skip behavior.
- bench: datagram benchmark harness (BenchOpt/DatagramOpt, DatagramCounters).
Baseline (loopback, release, 2-core): 440 MiB/s @ 1200B, 4.17M pps @ 64B.
Draft
4 tasks
flub
requested changes
Jul 1, 2026
flub
left a comment
flub
left a comment
Collaborator
There was a problem hiding this comment.
Probably not an unreasonable request, left some initial notes from a quick skim. I haven't looked at tests or benchmarks at all yet.
Collaborator
|
Please run |
Resolves flub's PR n0-computer#735 review: - Fix send_many over-allocation: evict oldest per-element before each push (not push-all-then-evict), so the outgoing deque never transiently holds the whole batch. A datagram larger than the budget but under max_size is still enqueued once the queue is emptied, matching send(drop=true). Locked in by datagram_batch_send_drop_oldest_per_element. - send_many now stops at the first oversized datagram and returns Ok(count) accepted from the front (mirrors write_chunks / sendmmsg "first K succeeded"). No longer returns Err(TooLarge); callers recover via &datagrams[count..]. Empty batch returns Ok(0) (datagram_batch_send_empty_is_ok). - Rename to the chunks-style API: send_many_datagrams / read_many_datagrams, returning ReadManyDatagrams. Drop the SendMany struct. - Take slices for consistency with read_many_chunks / write_chunks: send_many_datagrams(&[Bytes]) -> Result<usize, _>; read_many_datagrams(&mut [Bytes]) (fill-slots, returns count drained). recv_many return-count bug fixed (now reports slots filled, not bool). - Extract over_send_budget / would_exceed_send_budget helpers, deduped across send and send_many. - CI: fmt (project unstable config), clippy, rustdoc, all tests green.
ifdario
commented
Jul 2, 2026
flub
requested changes
Jul 8, 2026
- send_many takes a `drop` flag, mirroring send. Reject the whole batch with TooLarge up front if any datagram is oversized, so a size error is never a partial send. With drop = false, stop when the buffer is full and return the count queued. - Share the drop-oldest loop between send and send_many. - Rename the send-buffer helpers to is_send_buffer_exceeded and has_send_buffer_capacity, and drain in recv_many instead of popping. - Focus the docs and drop the lock/implementation details.
flub
requested changes
Jul 13, 2026
flub
requested changes
Jul 13, 2026
Collaborator
|
Please use the (admittedly tiny, well hidden) re-request review button whenever the PR is ready for another round of review. It is next each reviewer's name at the top right. |
flub
requested changes
Jul 16, 2026
|
|
||
| /// Direction of the datagram flood. | ||
| #[derive(Debug, Clone, Copy, PartialEq, Eq)] | ||
| pub enum Direction { |
Collaborator
There was a problem hiding this comment.
Have you found the direction to be important? In theory the direction could also be done for the stream bench, but for some reason it is not. Which is what makes me wonder if this reveals anything interesting.
Author
There was a problem hiding this comment.
Am I crazy or my comments where deleted?
The future no longer forces the caller's out slice borrow to match the connection borrow.
An unpaced flood queues datagrams far faster than the driver transmits, so drop-oldest evicts almost the whole run locally and the benchmark measures queueing speed instead of throughput. Wait for send buffer space before each send so every queued datagram reaches the wire and loss reflects the network.
…rker Derive derive_more::Display on Direction, SendMode, and Congestion so the JSON report prints them directly instead of hand-mapping strings. Replace the uni done-stream and the drain grace timeout with an in-band 1-byte DONE datagram: the sender floods, then resends the marker every 250ms until the receiver closes; the receiver counts until it sees the marker. The duplex coordination stream keeps only the S/F handshake, where F now means the peer's DONE was received.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Add
Connection::send_datagrams/read_datagramsand the underlyingDatagramState::recv_many/Datagrams::send_manyto amortize the per-datagram connection-mutex cost, which is the dominant overhead at high pps for a mesh VPN forwarding bursts of TUN packets.DatagramState::recv_manydrains all buffered incoming datagrams in oneVecDequedrain instead of Npop_fronts.Datagrams::send_manyqueues a whole batch under one logical operation: peer-support check once, drop-oldest backpressure pass once, oversized datagrams skipped individually (not aborting the batch).Connection::send_datagramstakes the mutex once, wakes the driver once.Connection::read_datagramsis aReadDatagramsfuture that parks on thedatagram_receivednotify then drains everything in one lock hold.SendMany { queued, rejected }result struct.BenchOpt/DatagramOpt,DatagramCounters).Baseline (loopback, release, 2-core): 440 MiB/s @ 1200B, 4.17M pps @ 64B.
Breaking Changes
None. All additions are new, purely additive methods and types. Existing
send_datagram/read_datagram/send_datagram_waitare unchanged.Notes & open questions
send_datagramreturnsErr(TooLarge)for an oversized datagram;send_manyskips oversized datagrams and counts them inSendMany::rejected, so a mixed-size batch doesn't lose the valid entries.Err(TooLarge)is returned only if every element is too large (or datagrams are unsupported/disabled). This is deliberate but worth a reviewer's eyes.Vecallocation per batch.Connection::send_datagramscollects the iterator into aVec<Bytes>to computerejected = total - queued. This is one small alloc per batch, negligible versus the N mutex acquisitions it replaces, but a future PR could have the protosend_manyreturn{queued, rejected}directly to avoid it.send_datagram_waithas no batch equivalent. A batched backpressuring send is a harder design (partial-batch wait semantics, per-element wakeups) and is deferred.Change checklist