Skip to content

Conversation

@luke-alloy
Copy link

@luke-alloy luke-alloy commented Oct 30, 2025

Description

Adds circular logging by split count, record size and duration to rosbag2 recording.

  • CLI: adds --max-splits, --max-record-size, --max-record-duration.
  • Writer: deletes oldest files when exceeding size, duration, or split count. Limits are enforced together; the oldest files are pruned when any of the three conditions above are satisfied.

Fixes #2217

Is this user-facing behavior change?

Yes.

  • New and updated recording limits are available and enforced together after each split:
    • --max-splits N: limit number of retained bagfile splits.
    • --max-record-size BYTES: limit total recorded size across all splits.
    • --max-record-duration SECONDS: limit total recorded duration across all splits.
  • Examples:
    • ros2 bag record -a --max-bag-size 100MB --max-splits 5
    • ros2 bag record -a --max-bag-duration 60 --max-record-size 5_000_000_000
    • ros2 bag record -a --max-bag-size 200MB --max-record-duration 300

Did you use Generative AI?

Yes.

  • Tool/model: GPT-5
  • Used for: naming suggestions, CLI/help text wording, and drafting PR description.
  • Code impact: assisted in scaffolding changes (option wiring, param names); final code implementation was completed, built, run, and manually reviewed and tested locally.

Additional Information

  • All limits (size, duration, count) are checked together after each bag split instead of on each msg write
  • The writer logs a warning when deleting files due to limits. Adjust --log-level if needed.
  • metadata.yaml currently retains aggregate message_count and topics_with_message_count across the entire capture, even when older bagfiles are pruned by circular logging. File lists and durations reflect pruning, but those two fields still represent the full session totals.
  • Circular logging changes are only in sequential_writer.cpp (deleting the oldest file if needed); the rest involve adding StorageOptions parameters, updating the shell UI, and adjusting related unit tests.

- Add StorageOptions.max_splits (+ YAML/params, Python bindings)
- CLI: --max-splits (before size/duration)
- Writer: delete oldest files when exceeding size, duration, or split count

Signed-off-by: Luke Sy <[email protected]>
Signed-off-by: Luke Sy <[email protected]>
@luke-alloy luke-alloy force-pushed the feature/ros2bag-circular-logging-size-duration-2217 branch from 309537d to 7b4c6bb Compare October 30, 2025 13:20
@luke-alloy luke-alloy marked this pull request as ready for review October 30, 2025 21:29
@luke-alloy luke-alloy changed the title Feature/ros2bag circular logging size duration 2217 Feature/rosbag2 circular logging size duration 2217 Oct 30, 2025
Comment on lines +167 to +178
parser.add_argument(
'--max-record-size', type=int, default=0,
help='Maximum total size in bytes for the entire recording across all splits. '
'Oldest bagfiles will be deleted when this limit is exceeded (circular logging). '
'Default: %(default)d, no limit on total recording size. '
'Requires --max-bag-size or --max-bag-duration to be set.')
parser.add_argument(
'--max-record-duration', type=int, default=0,
help='Maximum total duration in seconds for the entire recording across all splits. '
'Oldest bagfiles will be deleted when this limit is exceeded (circular logging). '
'Default: %(default)d, no limit on total recording duration. '
'Requires --max-bag-size or --max-bag-duration to be set.')
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i am not against this PR. but i am not sure if those 2 optional arguments need to be supported by rosbag2... because i think those can be easily calculated by --max-splits option above? i think that using --max-splits with --max-bag-size or --max-bag-duration would be more straight-forward for user experience.

besides that,

  • (already) too many optional arguments for ros2 bag record... this generates more and more complication.
  • additional error handling needs to be implemented in the rosbag2 system. e.g) what if user accidentally set --max-record-size with smaller value than --max-bag-size.

@luke-alloy luke-alloy force-pushed the feature/ros2bag-circular-logging-size-duration-2217 branch from 30f9f5c to cf16699 Compare November 10, 2025 16:06
@luke-alloy
Copy link
Author

I've committed changes that should address the unit test error. I'm also happy to include @fujitatomoya ’s suggestion to remove --max-record-size and --max-record-duration, pending confirmation to proceed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Feature: Add circular logging (by size/duration) to ros2 bag record

2 participants