|
| 1 | +# depthz_image_transport perf benchmark |
| 2 | + |
| 3 | +Local tooling for profiling `DepthzPublisher`/`DepthzSubscriber` with `perf`, |
| 4 | +using real depth data. Not part of the package's public surface or CI -- |
| 5 | +opt in with `-DDEPTHZ_BUILD_BENCHMARK=ON`. |
| 6 | + |
| 7 | +`benchmark_depthz` talks to the plugin exclusively through the public |
| 8 | +`image_transport`/`rclcpp` API (`ImageTransport::advertise`, |
| 9 | +`image_transport::create_subscription(..., "depthz", ...)`), the same way |
| 10 | +any real node would. Profiling this binary profiles exactly what a deployed |
| 11 | +publisher or subscriber node spends its time on -- pluginlib loading |
| 12 | +included -- not just the vendored codec's hot loop in isolation. |
| 13 | + |
| 14 | +## 1. Get real depth frames |
| 15 | + |
| 16 | +`extract_frames.py` pulls raw `sensor_msgs/Image` frames off an MCAP topic |
| 17 | +into a compact `.dzbm` file (see the docstring for the format): |
| 18 | + |
| 19 | +```sh |
| 20 | +python3 extract_frames.py \ |
| 21 | + --mcap ~/ws_eternal/src/Harvesting/harvest_bringup/test/test_data/rosbags/van_noord_tomato_20250919_181407/20250919_181407_0.mcap \ |
| 22 | + --topic /zed_wrist/zed_node/depth/depth_registered \ |
| 23 | + --out data/wrist_1920x1200.dzbm |
| 24 | +``` |
| 25 | + |
| 26 | +Needs `pip install mcap mcap-ros2-support`. That bag has two raw 32FC1 |
| 27 | +streams from the ZED stereo cameras worth benchmarking: |
| 28 | + |
| 29 | +| topic | resolution | frames | |
| 30 | +|---|---|---| |
| 31 | +| `/zed_base/zed_node/depth/depth_registered` | 960x600 | 10 | |
| 32 | +| `/zed_wrist/zed_node/depth/depth_registered` | 1920x1200 | 10 | |
| 33 | + |
| 34 | +Point `--mcap`/`--topic` at any other bag with a raw (uncompressed) 32FC1 or |
| 35 | +16UC1 `sensor_msgs/Image` topic -- it doesn't have to be ZED/depth-camera |
| 36 | +specific. |
| 37 | + |
| 38 | +## 2. Build |
| 39 | + |
| 40 | +```sh |
| 41 | +colcon build --packages-select depthz_image_transport \ |
| 42 | + --cmake-args -DDEPTHZ_BUILD_BENCHMARK=ON -DCMAKE_BUILD_TYPE=RelWithDebInfo |
| 43 | +source install/setup.bash |
| 44 | +``` |
| 45 | + |
| 46 | +`RelWithDebInfo` keeps optimizations on (required -- see the package's own |
| 47 | +`CMakeLists.txt` comment) while still emitting frame pointers/debug info for |
| 48 | +`perf record -g` to unwind. |
| 49 | + |
| 50 | +## 3. Run |
| 51 | + |
| 52 | +```sh |
| 53 | +benchmark_depthz --frames data/wrist_1920x1200.dzbm --mode roundtrip --iterations 300 |
| 54 | +``` |
| 55 | + |
| 56 | +`--mode`: |
| 57 | +- `roundtrip` (default): a real `DepthzPublisher` encodes each frame, a real |
| 58 | + `DepthzSubscriber` decodes it. End-to-end throughput and compression |
| 59 | + ratio. |
| 60 | +- `encode`: only the publisher plugin's encode path runs (the "subscriber" |
| 61 | + is a bare passthrough callback, kept alive only because |
| 62 | + `image_transport::Publisher` publishes on demand and needs to see a real |
| 63 | + subscriber count -- see its class doc). |
| 64 | +- `decode`: all frames are pre-encoded once (untimed) by a real |
| 65 | + `DepthzPublisher`, then the timed loop republishes those captured blobs |
| 66 | + directly onto the internal `<topic>/depthz` topic with a plain |
| 67 | + `rclcpp::Publisher`, bypassing the publisher plugin entirely, while a real |
| 68 | + `DepthzSubscriber` decodes them. This isolates decode cost from encode |
| 69 | + cost. |
| 70 | + |
| 71 | +Other flags: `--transport <name>` (benchmark any installed image_transport |
| 72 | +plugin against the same frames, e.g. `compressedDepth`), `--iterations N` |
| 73 | +(loop the dataset N times), `--warmup N` (untimed iterations first, so |
| 74 | +`thread_local` scratch buffers reach their steady-state size before the |
| 75 | +clock starts), `--zstd-level 1-3`, `--quantization MM` (depthz quantization |
| 76 | +step in millimeters; the benchmark defaults to `0` = lossless, deliberately |
| 77 | +overriding the plugin's own lossy 0.1 mm default so the bit-exact verify |
| 78 | +stays meaningful — pass `--quantization 0.1` to measure the plugin's actual |
| 79 | +default behavior), `--no-verify` (skip verification entirely; use before |
| 80 | +profiling so the comparison doesn't show up as noise in the flamegraph), |
| 81 | +`--qos-depth N`. |
| 82 | + |
| 83 | +Output reports wall time, fps, MB/s (raw), compression ratio, and (unless |
| 84 | +`--no-verify`) a verification against the source frames: bit-exact for |
| 85 | +lossless depthz, the documented ± step/2 error bound plus NaN preservation |
| 86 | +for quantized depthz, and informational-only for other transports. |
| 87 | + |
| 88 | +## 4. Profile with perf |
| 89 | + |
| 90 | +```sh |
| 91 | +perf stat -d -- benchmark_depthz --frames data/wrist_1920x1200.dzbm --mode encode --no-verify |
| 92 | +perf record -g --call-graph dwarf -o encode.perf.data -- \ |
| 93 | + benchmark_depthz --frames data/wrist_1920x1200.dzbm --mode encode --no-verify |
| 94 | +perf report -i encode.perf.data |
| 95 | +``` |
| 96 | + |
| 97 | +`run_perf.sh` automates all of the above (build, extract both ZED streams, |
| 98 | +run `perf stat` + `perf record` for all 3 modes x 2 resolutions) and drops |
| 99 | +results under `results/`: |
| 100 | + |
| 101 | +```sh |
| 102 | +./run_perf.sh |
| 103 | +``` |
| 104 | + |
| 105 | +Because the benchmark drives a `SingleThreadedExecutor` by hand (publish, |
| 106 | +then `spin_some` until the callback fires, repeat) rather than a background |
| 107 | +spin thread, encode and decode samples land on the same call stack you'd |
| 108 | +expect from the source -- `dpred_encode`/`build_value_dict`/`zstd_append` |
| 109 | +for encode, `dpred_decode`/`predict_unpack`/`ZSTD_decompressDCtx` for |
| 110 | +decode -- with no cross-thread noise to untangle. |
0 commit comments