Skip to content

Add file-backed ring buffer with per-topic configurability#8

Merged
marc-hanheide merged 2 commits into
mainfrom
copilot/implement-file-based-ring-buffer
Mar 13, 2026
Merged

Add file-backed ring buffer with per-topic configurability#8
marc-hanheide merged 2 commits into
mainfrom
copilot/implement-file-based-ring-buffer

Conversation

Copilot AI commented Mar 13, 2026

Copy link
Copy Markdown

The in-memory ring buffer can consume excessive RAM for high-bandwidth topics like camera images. This adds a file-backed alternative that keeps only a small index in RAM while writing payloads to a temporary file, selectable per topic.

New: FileRingBuffer

  • Drop-in replacement for RingBuffer (identical API: add, snapshot, size, memory_bytes)
  • Payloads stored in tempfile.TemporaryFile (auto-deleted on GC/close); in-memory index holds only (timestamp_ns, file_offset, record_size) per message
  • Automatic file compaction when dead bytes (purged records) exceed live bytes

Config: per-topic buffer key

topics:
  - type: "sensor_msgs/msg/Image"
    buffer: file        # new — file-backed; default is "memory"
    max_frequency: 1.0
    qos: { ... }

  - pattern: "/**"
    buffer: memory      # explicit default; in-memory RingBuffer (unchanged behaviour)

example_config.yaml updated to use buffer: file for sensor_msgs/msg/Image.

Node changes (ringbuffer_node.py)

  • Topics with buffer: file get a dedicated FileRingBuffer(duration) stored in self._file_bufs
  • Snapshot merges messages from the shared memory buffer and all per-topic file buffers, sorted by timestamp before writing the rosbag

💬 Send tasks to Copilot coding agent from Slack and Teams to turn conversations into code. Copilot posts an update in your thread when it's finished.

Co-authored-by: marc-hanheide <1153084+marc-hanheide@users.noreply.github.com>
Copilot AI changed the title [WIP] Add file based solution for ring buffer Add file-backed ring buffer with per-topic configurability Mar 13, 2026
Copilot AI requested a review from marc-hanheide March 13, 2026 08:24
@marc-hanheide
marc-hanheide marked this pull request as ready for review March 13, 2026 09:32
@marc-hanheide
marc-hanheide merged commit a447546 into main Mar 13, 2026
3 checks passed
@marc-hanheide
marc-hanheide deleted the copilot/implement-file-based-ring-buffer branch March 13, 2026 09:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants