Skip to content

Commit 1da66bf

Browse files
authored
feat: Enhance benchmark configuration and latency testing (#72)
* feat: Add --send-delay for latency-focused testing Introduces a new `--send-delay` command-line argument that adds a configurable pause between sending messages. This feature allows users to switch from the default high-throughput benchmark to a latency-focused test, which is useful for simulating workloads that are not CPU-bound and for measuring message latency under a controlled, predictable load. The implementation includes: - A new `--send-delay` CLI flag with microsecond-level duration parsing. - Integration into the benchmark runner to pause after each send. - Updated README.md to explain the difference between throughput and latency testing and document the new flag. AI-assisted-by: Gemini 2.5 Pro test: Add unit and integration tests for --send-delay Adds test coverage for the new send delay feature: - A unit test for the `parse_duration_micros` function to validate correct parsing of various time units. - An integration test (`test_send_delay_is_applied`) to ensure the `--send-delay` logic correctly pauses between messages during a benchmark run. AI-assisted-by: Gemini 2.5 Pro feat: Add configurable PMQ message priority Adds a `--pmq-priority` command-line flag to allow setting the message priority for the POSIX Message Queue (PMQ) mechanism. This feature enables more fine-grained control over PMQ behavior, allowing for latency-focused testing where message delivery order can be influenced by priority. The priority is passed through the configuration layers and used in the `mq_send` call. The UI has been updated to display the configured priority when a PMQ test is run. AI-assisted-by: Gemini 2.5 Pro test: Add tests and docs for PMQ priority feature Adds unit and integration tests to validate the functionality of the `--pmq-priority` flag. - A unit test in `src/cli.rs` verifies that the CLI argument is parsed correctly. - An integration test in `src/ipc/posix_message_queue.rs` confirms that the specified priority is correctly applied during the `mq_send` call. This test uses a `oneshot` channel for robust synchronization between the server and client tasks. Also updates the `README.md` to document the new flag for users. AI-assisted-by: Gemini 2.5 Pro feat: Add option to control first-message latency spike Introduces a mechanism to mitigate first-message latency spikes and provides user control over this behavior. High latency on the first message of a benchmark is a common phenomenon due to "cold start" effects like CPU cache misses, page faults, and one-time memory allocations in the measurement code path. This commit addresses this by: 1. Sending a single "canary" message before the main measurement loop begins. This message warms up the hardware and application code paths. 2. Discarding the result of this canary message by default, ensuring that the final statistics are more representative of steady-state performance. 3. Adding an `--include-first-message` command-line flag that allows users to disable this behavior and include the canary message in the results for raw performance analysis. The implementation correctly handles both message-count and duration-based tests. The UI has been updated to clearly display whether the first message is being discarded or included in every benchmark summary. AI-assisted-by: Gemini 2.5 Pro refactor(ui): Centralize benchmark configuration display The benchmark configuration display logic was duplicated, leading to inconsistent UI output and recurring bugs where new options were not displayed. This commit refactors the display logic into a single `BenchmarkConfigDisplay` struct in `benchmark.rs`, which now serves as the single source of truth. The dead `impl Display for Args` in `src/cli.rs` has been removed. This also removes an unused `use std::fmt;` import that was causing a compiler warning. AI-assisted-by: Gemini 2.5 Pro fix(benchmark): Apply send_delay during warmup phase The warmup loop was not respecting the `--send-delay` argument, causing it to send messages as fast as possible regardless of the configuration. This led to unexpected backpressure warnings, particularly for the PMQ mechanism, even when a delay was specified. This commit applies the `send_delay` within the warmup message loop, ensuring that the warmup phase accurately simulates the conditions of the actual benchmark run. AI-assisted-by: Gemini 2.5 Pro update README * address cargo fmt CI problems * feat: Add info log for ignored pmq-priority parameter When the `--pmq-priority` parameter is provided for an IPC mechanism that is not POSIX Message Queues (PMQ), the parameter is ignored. This commit adds an `info` level log message to notify the user when this occurs, so they are aware that the setting is not being applied. The message is logged for each non-PMQ mechanism specified when `--pmq-priority` is used. AI-assisted-by: Gemini 2.5 Pro * fix: Prevent server timeout with long send-delay When running a message-count-based test with a `--send-delay` that was longer than the servers internal 50ms receive timeout, the server would prematurely exit. This caused the client to fail with a "Broken pipe" error when it attempted to send its next message. This commit fixes the issue by changing the servers behavior. When a receive timeout occurs, the server now continues its loop instead of exiting. This ensures the server waits patiently for all expected messages, making the benchmark tool robust for any `--send-delay` value. AI-assisted-by: Gemini 2.5 Pro * fix: Correct Windows and macOS build failures The Windows and macOS builds were failing due to a check for the `--pmq-priority` parameter that referenced `IpcMechanism::PosixMessageQueue`, an enum variant that is only compiled on Linux. This commit resolves the build failures by wrapping the entire check in a `#[cfg(target_os = "linux")]` attribute. This ensures the platform-specific code is only compiled on Linux and is ignored on all other platforms. AI-assisted-by: Gemini 2.5 Pro
1 parent be0e3a3 commit 1da66bf

7 files changed

Lines changed: 574 additions & 190 deletions

File tree

README.md

Lines changed: 55 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -137,10 +137,62 @@ ipc-benchmark --percentiles 50 90 95 99 99.9 99.99
137137
# TCP-specific configuration
138138
ipc-benchmark -m tcp --host 127.0.0.1 --port 9090
139139

140+
# POSIX Message Queue-specific configuration
141+
ipc-benchmark -m pmq --pmq-priority 1
142+
140143
# Shared memory configuration (demonstrating a user-provided buffer size)
141144
ipc-benchmark -m shm --buffer-size 16384
142145
```
143146

147+
### First-Message Latency (Canary)
148+
149+
The first message in any benchmark often has a higher latency than subsequent messages due to "cold start" effects like CPU cache misses, memory allocation, and branch prediction misses. To provide more stable and representative results, this tool automatically sends one "canary" message before starting the measurement loop. This message and its latency are discarded by default.
150+
151+
If you need to analyze the raw performance data, including the first-message spike, you can use the `--include-first-message` flag to disable this behavior.
152+
153+
```bash
154+
# Include the first message in the final results
155+
ipc-benchmark --include-first-message
156+
### Understanding Test Types: Throughput vs. Latency
157+
158+
This benchmark suite can be used to measure two primary aspects of IPC performance: **throughput** and **latency**. The configuration you choose will determine which of these you are primarily testing.
159+
160+
#### Throughput-Focused Testing (Default Behavior)
161+
162+
By default, this tool is a **throughput benchmark**. It attempts to send messages as fast as the system will allow, with no artificial delays. This is ideal for answering questions like:
163+
164+
- "What is the maximum message rate this IPC mechanism can handle?"
165+
- "How many megabytes per second can I transfer?"
166+
167+
This mode is most useful for stress-testing the system and finding its performance limits under heavy load.
168+
169+
```bash
170+
# Throughput test: Send 100,000 messages as fast as possible
171+
./target/release/ipc-benchmark -m uds -i 100000 -s 4096
172+
```
173+
174+
#### Latency-Focused Testing (Using `--send-delay`)
175+
176+
Sometimes, you are more interested in the latency of individual messages under a controlled, predictable load, rather than the maximum possible throughput. The `--send-delay` flag allows you to introduce a fixed pause between each message sent.
177+
178+
This is a **latency-focused test**. It is ideal for answering questions like:
179+
180+
- "When my application sends a message every 10 milliseconds, what is the typical time it takes for that message to be delivered?"
181+
- "Does latency remain stable or does it spike under a consistent, non-maximal load?"
182+
183+
This mode simulates applications that are not constantly sending data at maximum speed, which is a very common real-world scenario.
184+
185+
```bash
186+
# Latency test: Send messages with a 10 millisecond delay between each one
187+
./target/release/ipc-benchmark \
188+
-m pmq \
189+
-i 10000 \
190+
-s 116 \
191+
--send-delay 10ms
192+
```
193+
194+
By using `--send-delay`, you can more accurately measure the base "travel time" of a message without the confounding factor of queue backpressure that occurs during high-throughput tests.
195+
144196
### Test Configuration Examples
145197

146198
#### High-Throughput Testing
@@ -153,14 +205,16 @@ ipc-benchmark \
153205
--buffer-size 1048576
154206
```
155207

208+
156209
#### Low-Latency Testing
157210
```bash
158211
ipc-benchmark \
159212
-m uds \
160213
--message-size 64 \
161214
--msg-count 100000 \
162215
--warmup-iterations 10000 \
163-
--percentiles 50 95 99 99.9 99.99
216+
--percentiles 50 95 99 99.9 99.99 \
217+
--send-delay 10ms
164218
```
165219

166220
#### Comparative Analysis
@@ -535,22 +589,6 @@ ipc-benchmark -m shm -i 10000 -s 1024 --buffer-size 8192
535589
# This may cause backpressure, which is a valid test scenario.
536590
```
537591

538-
### Recommended Usage
539-
540-
```bash
541-
# ✅ Optimal for latency measurement
542-
ipc-benchmark -m all -c 1 -i 10000 -s 1024
543-
544-
# ✅ Good for throughput analysis (TCP/UDS only)
545-
ipc-benchmark -m tcp,uds -c 4 -i 10000 -s 1024
546-
547-
# ✅ Shared memory test
548-
ipc-benchmark -m shm -i 10000 -s 1024
549-
550-
# ⚠️ Will automatically use c=1 for shared memory
551-
ipc-benchmark -m shm -c 4 -i 10000 -s 1024
552-
```
553-
554592
### Error Prevention
555593

556594
Common issues and solutions:
@@ -564,4 +602,3 @@ Common issues and solutions:
564602
- **Single-threaded** (`-c 1`): Most accurate latency measurements
565603
- **Simulated concurrency** (`-c 2+`): Good for throughput scaling analysis
566604
- **Shared memory**: Always single-threaded for reliability
567-

0 commit comments

Comments
 (0)