Skip to content

feat(dc_mcap_writer,docs): add MCAP as a passthrough Destination - #319

Merged
Minipada merged 3 commits into
jazzyfrom
feature/210-add-mcap-as-a-destination-ros-2-bag-stor
Aug 12, 2026
Merged

feat(dc_mcap_writer,docs): add MCAP as a passthrough Destination#319
Minipada merged 3 commits into
jazzyfrom
feature/210-add-mcap-as-a-destination-ros-2-bag-stor

Conversation

@Minipada

Copy link
Copy Markdown
Owner

Summary

  • Records can now be recorded as .mcap files for ros2 bag info/Foxglove replay. Vector has no MCAP sink, so per ADR-0003 this needed a blessed-vs-passthrough decision recorded before implementing — docs/adr/0009-mcap-as-a-passthrough-destination.md records the choice: passthrough, not a dc_bridge-side blessed Destination. The issue's stated justification for the Bridge-side option (local-first durability surviving a Shipper outage) isn't a real gap: the Shipper is already at-least-once and disk-buffered per blessed sink (ADR-0002).
  • New package dc_mcap_writer (ament_python, no ROS deps of its own): a standalone process that consumes the public dc.<tag> routes over a Vector socket sink (NDJSON) and writes rotated .mcap files, one JSON-schema Channel per Tag (the foxglove/mcap jsonschema/writer.cpp pattern the issue links). Rotates by size or duration, whichever comes first.
  • Demo (dc_demos/config/mcap_sink.toml, params/mcap_recording.yaml, launch/mcap_recording.launch.py) and a new tutorial (doc/src/dc/demos/mcap_recording.md) following the existing Elasticsearch/InfluxDB passthrough demo shape.
  • mcap added as a Python dependency (pyproject.toml/requirements.txt), matching how numpy/opencv-python/pydantic/Pillow are already managed outside rosdep.

Test plan

  • 12-case pytest suite for dc_mcap_writer (rotation by size/duration, per-Tag channel routing, all three time_format conversions, malformed-NDJSON handling, mcap round-trip via mcap.reader) — all pass in a throwaway venv (mcap/pytest installed via uv, no ROS 2 available in this environment)
  • CLI --stdin smoke test end-to-end: piped NDJSON in, read the resulting .mcap back independently, confirmed Channels/schema/timestamps round-trip correctly
  • pre-commit run against every changed/new file (black/isort/pyupgrade/flake8/xmllint/yaml/toml clean; pycln/poetry-requirements fail in this environment the same pre-existing way earlier work already logged in progress.txt)
  • colcon build --packages-select dc_mcap_writer and ros2 bag info/Foxglove against a real recorded file — needs a ROS 2 Jazzy environment (the podman-based dc-workspace image used for prior C++ verification), not available in this session

Closes #210

Records can now be recorded as .mcap files for ros2 bag/Foxglove replay.
Since Vector has no MCAP sink, this is the ADR-0009 passthrough shape:
dc_mcap_writer, a standalone process, consumes the public dc.<tag>
routes over a Vector socket sink and writes rotated .mcap files with
one JSON-schema Channel per Tag, leaving dc_bridge itself unchanged.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LsKvbNGBjYBiTkQzqbacp5
Signed-off-by: David Bensoussan <d.bensoussan@proton.me>
@codecov

codecov Bot commented Aug 11, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 40.81%. Comparing base (588c495) to head (e7c394f).

Additional details and impacted files
@@           Coverage Diff           @@
##            jazzy     #319   +/-   ##
=======================================
  Coverage   40.81%   40.81%           
=======================================
  Files          82       82           
  Lines        4919     4919           
=======================================
  Hits         2007     2007           
  Misses       2912     2912           
Flag Coverage Δ
cpp-jazzy 40.81% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Minipada and others added 2 commits August 11, 2026 15:48
…MCAP passthrough

Adds real subprocess/signal smoke tests and wires dc_mcap_writer into the
zero-loss e2e harness as a second, independently-verified passthrough
alongside the existing NDJSON one. Getting it to actually pass surfaced
and fixed three real bugs: mcap was never installed in the CI workspace
image (colcon build doesn't fetch install_requires), a filename collision
across process restarts, mcap's default chunking silently losing every
buffered message on an abrupt kill, and ros2 run not forwarding signals
to the process it starts (the last one was the actual root cause keeping
the file corrupted even after the first two fixes).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LsKvbNGBjYBiTkQzqbacp5
Signed-off-by: David Bensoussan <d.bensoussan@proton.me>
…Destination

Manually copying a Vector TOML into ~/.dc/ and starting dc_mcap_writer in
a second terminal broke the YAML-configured-Destination flow every other
sink follows. dc_bringup.launch.py now reads a dc_mcap_writer: block from
the same params file (sibling to dc_bridge:, not nested in its
destinations list, since that's parsed/validated in dc_bridge's own C++
and ADR-0009 explicitly avoids touching it), generates the matching
Vector passthrough sink, and starts dc_mcap_writer automatically via
ExecuteProcess (python3 -m dc_mcap_writer.cli directly, not ros2 run,
which doesn't forward signals to its child). Verified end-to-end against
the rebuilt workspace image, including a regression check that existing
demos without the block behave byte-for-byte as before.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LsKvbNGBjYBiTkQzqbacp5
Signed-off-by: David Bensoussan <d.bensoussan@proton.me>
@Minipada
Minipada merged commit 12ac04f into jazzy Aug 12, 2026
8 checks passed
@Minipada
Minipada deleted the feature/210-add-mcap-as-a-destination-ros-2-bag-stor branch August 16, 2026 14:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant