Skip to content

Commit f11d10b

Browse files
awesomebytesclaude
andcommitted
Add heavy-topic ros2 topic hz demo (rclpy vs rclcppyy) + startup story
A ros2 topic hz-style showcase on a heavy topic (1024x1024 sensor_msgs/Image, 3.0 MB) where plain rclpy visibly falls behind and the accelerated subscriber does not, plus the cold-vs-warm bringup story. Under scripts/heavy_hz_demo/: - heavy_publisher.py: C++-accelerated publisher; builds the 3 MB image once and re-publishes at a target rate, so it is never the bottleneck under test. - hz_subscriber.py: the ros2 topic hz equivalent (windowed rate, recv/expected, drops via header seq, latency). SAME script both ways -- the only difference is whether `import rclcppyy; rclcppyy.enable_cpp_acceleration()` executes (gated on HEAVY_HZ_ACCEL). It reads only header fields, never the payload. - run_heavy_hz.py: orchestrator (spawns the pair, warms up, samples per-process CPU, parses stats, prints a table) + --startup-story (cold vs warm bringup). - startup_probe.py: single-process bringup probe for the startup story. Measured (3.0 MB, RELIABLE KEEP_LAST depth 10, cyclonedds LOCALHOST): at 100 Hz both keep up but rclpy burns ~2.6x the CPU; the knee is ~200 Hz for 3 MB -- plain rclpy tops out ~210 Hz (pegging ~1.6 cores) while the accelerated subscriber climbs to 336 Hz @500 target and 417 Hz @700, at half the CPU and a third of the latency. Startup: rclcpp headers parse ~1.7s cold -> 0.0s warm (Cling PCH). RELIABLE is deliberate: net.core.rmem_max is 208 KB, so BEST_EFFORT drops whole 3 MB messages to socket-buffer overflow (noise unrelated to the subscriber). Wiring: - pixi.toml: heavydemo feature/env (solve-group=default, no extra conda deps) + tasks demo-heavy-hz / -rclpy / -rclcppyy / demo-heavy-startup; isolated XDG_CACHE_HOME=.heavy_demo_cache for the auto-PCH cache. - workspace_activation.sh: DEV/demo-only bridge, gated on PIXI_ENVIRONMENT_NAME=heavydemo, prepending a .suite_bridge symlink dir (cppyy_kit + rclcpp_kit from a sibling ../cppyy_kit checkout, override with CPPYY_KIT_SRC) to PYTHONPATH so the demo runs on the newer zero-config auto-PCH that postdates the published suite 0.1.0. No-op in every other env; the default env's published-channel dependency is untouched. - README: "Heavy-topic ros2 topic hz showcase" section with the measured tables, exact commands, the bridge note, and the honest boundaries. - test/test_heavy_hz_demo.py: headless smoke test of the machinery on plain rclpy (tiny image, short window, sanity only) so it runs in the default env / CI. pixi run test: 27 passed. pixi run lint: clean. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
1 parent b34dddf commit f11d10b

11 files changed

Lines changed: 1476 additions & 0 deletions

.gitignore

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -22,5 +22,8 @@ data/
2222
# M3 pre-release PYTHONPATH bridge (generated symlinks to the sibling cppyy_kit checkout)
2323
.suite_bridge/
2424

25+
# heavy_hz demo: isolated auto-PCH cache (XDG_CACHE_HOME for the heavydemo env)
26+
.heavy_demo_cache/
27+
2528
# Local Claude agent state / worktrees
2629
.claude/

README.md

Lines changed: 88 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -229,6 +229,94 @@ Without entering a shell, any command can be run through pixi directly, e.g.
229229
`pixi run ros2 run rclcppyy bench_sub_rclcppyy_monkeypatched.py`.
230230
</details>
231231

232+
### Heavy-topic `ros2 topic hz` showcase (dev bridge)
233+
234+
`scripts/heavy_hz_demo/` is a `ros2 topic hz`-style showcase on a **heavy** topic
235+
(a 1024×1024 `sensor_msgs/Image`, 3.0 MB per message) where a plain `rclpy`
236+
subscriber measurably falls behind and the accelerated one does not. A
237+
C++-accelerated publisher streams the image at a target rate; an hz-measuring
238+
subscriber reports achieved rate, received-vs-expected, drops, latency, and CPU.
239+
The **same** subscriber script runs both ways — the only difference is whether this
240+
one line near the top executes:
241+
242+
```python
243+
import rclcppyy; rclcppyy.enable_cpp_acceleration()
244+
```
245+
246+
> **Needs `cppyy-kit` newer than the published 0.1.0** (for the zero-config
247+
> auto-PCH the cold-vs-warm story below relies on). Until the next suite release,
248+
> the `heavydemo` pixi env bridges the newer `rclcpp_kit`/`cppyy_kit` from a sibling
249+
> `cppyy_kit` **source checkout** (default `../cppyy_kit`, override with
250+
> `CPPYY_KIT_SRC`). This is a **dev/demo-only** env: it shares the default solve
251+
> (adds no conda deps) and leaves the default env's published-channel dependency
252+
> untouched. The bridge is set up in `workspace_activation.sh`, gated on the
253+
> `heavydemo` env; it also points `XDG_CACHE_HOME` at a repo-local `.heavy_demo_cache/`
254+
> so the demo never touches your shared `~/.cache`.
255+
256+
```bash
257+
pixi install -e heavydemo # shares the default solve; no re-download
258+
pixi run -e heavydemo demo-heavy-hz # rclpy vs rclcppyy table at 500 Hz / 3 MB
259+
pixi run -e heavydemo demo-heavy-hz-rclpy # just the plain-rclpy subscriber
260+
pixi run -e heavydemo demo-heavy-hz-rclcppyy # just the accelerated subscriber
261+
pixi run -e heavydemo demo-heavy-startup # cold vs warm rclcppyy bringup
262+
263+
# Sweep it yourself (flags pass straight through the task):
264+
pixi run -e heavydemo demo-heavy-hz --rate 700 --width 1280 --height 960 --duration 15
265+
```
266+
267+
**Measured** on the reference machine (1024×1024 bgr8 = 3.0 MB, RELIABLE QoS,
268+
KEEP_LAST depth 10, cyclonedds on LOCALHOST; absolute numbers vary by machine and
269+
run — reproduce with the command above):
270+
271+
| Target Hz | Subscriber | sub CPU % | achieved Hz | dropped | avg latency (ms) |
272+
|---|---|---|---|---|---|
273+
| 100 | `rclpy` | 57.9 | 93.3 | 43 | 31.9 |
274+
| 100 | `rclcppyy` | 22.2 | 92.6 | 56 | 32.5 |
275+
| 500 | `rclpy` | 156.9 | 205.5 | 2646 | 48.2 |
276+
| 500 | `rclcppyy` | 69.9 | 335.9 | 1464 | 17.4 |
277+
| 700 | `rclpy` | 164.0 | 213.5 | 3499 | 39.6 |
278+
| 700 | `rclcppyy` | 80.9 | 417.0 | 1728 | 13.8 |
279+
280+
At 100 Hz **both keep up** (~93 Hz delivered, similar latency) — but plain `rclpy`
281+
already burns ~2.6× the CPU. Push harder and the gap opens: the **knee is around
282+
200 Hz for 3 MB on this machine**. Plain `rclpy` tops out near **~210 Hz** no
283+
matter how fast you publish (it saturates ~1.6 CPU cores), while the accelerated
284+
subscriber climbs to **336 Hz at 500 Hz target and 417 Hz at 700 Hz** — roughly
285+
**2× the throughput at half the CPU and a third of the latency**.
286+
287+
**Why:** a plain `rclpy` subscription deserialises the *entire* message — including
288+
the whole 3 MB `data` array — into a Python object **before every callback**, even
289+
though `ros2 topic hz` only needs arrival times. That per-message Python
290+
deserialisation is what saturates the core. With `enable_cpp_acceleration()` the
291+
message stays a C++ object on the `rclcpp` subscription path; the callback reads
292+
only the small header fields it touches, and the multi-megabyte payload is never
293+
copied into Python.
294+
295+
**Startup: first run warms up, the next is far faster.** The first accelerated
296+
bringup JIT-parses the `rclcpp` headers; the suite's zero-config Cling PCH bakes
297+
them so later process starts skip that parse (`demo-heavy-startup`):
298+
299+
| | cold (fresh cache) | warm (PCH cached) |
300+
|---|---|---|
301+
| `rclcpp` C++ headers parse | ~1.7 s | **0.0 s** (`Cling PCH loaded from …`) |
302+
| time to a ready subscriber | ~6.4 s | ~4.8 s |
303+
304+
The cold run prints `rclcpp C++ headers loaded (1.7s)` and schedules the PCH build
305+
at exit; the warm run prints `rclcpp C++ headers loaded (0.0s)`. (The remaining
306+
warm cost is cppyy backend init, symbol discovery, and node setup, which the header
307+
PCH does not cover — see the suite's [Freeze & Cache](https://awesomebytes.github.io/cppyy_kit/docs/FREEZE/) docs.)
308+
309+
**Honest boundaries.** RELIABLE QoS is deliberate: a 3 MB image fragments into
310+
~2000 UDP datagrams, and under BEST_EFFORT a single lost fragment drops the whole
311+
message once the socket receive buffer overflows (`net.core.rmem_max` is 208 KB
312+
here) — noise unrelated to the subscriber. RELIABLE retransmits, so delivered
313+
throughput reflects only how fast the subscriber drains its queue; the drops it
314+
still shows at high rate are the KEEP_LAST history overflowing when the subscriber
315+
cannot keep up. The publisher runs C++-accelerated in every variant, so it is never
316+
the limiter and the *subscriber* is unambiguously the component under test. The
317+
headless CI test (`test/test_heavy_hz_demo.py`) runs this machinery on plain rclpy
318+
for a couple of seconds with a tiny image — sanity only, no perf assertion.
319+
232320
### Extra demos (optional env)
233321

234322
Heavier example scripts (OpenCV, PCL, GStreamer, typer, hypothesis) live in an

0 commit comments

Comments
 (0)