Skip to content

Make it easy to extract typed rerun components for .rrd recordings #12140

@polynuma

Description

@polynuma

Is your feature request related to a problem? Please describe.
reconstructing a typed rr.Image / rr.Pinhole / rr.Point3d from a .rrd file requires too many lines of the code. That is, after querying an .rrd file (load_recording -> view -> select -> read_all), there's not a simple way to deserialize the columnar format into an iterable set of rows of the original type. Here's an example of recreating an rr.Image - the least complicated of all

    rrd = rr.dataframe.load_recording(rrd_path)
    timeline="frame"
    path="/camera/0/color/image"
    view = rrd.view(index=timeline, contents=path)
    table = view.select().read_all()
    frame_col = table[timeline]
    buf_col_name = f"{path}:Image:buffer"
    buf_col   = table[buf_col_name]
    fmt_col_name = f"{path}:Image:format"
    fmt_col   = table[fmt_col_name]
    for row_idx in range(table.num_rows):
        # ---- timestamp or sequence number ----
        ts_scalar = frame_col[row_idx]
        frame_ts = ts_scalar.value
        # ---- zero-copy bytes ----
        inner = buf_col[row_idx][0]
        pixels = inner.values.to_numpy(zero_copy_only=False)
        # ---- width/height ----
        fmt = fmt_col[row_idx][0]
        w = int(fmt["width"])
        h = int(fmt["height"])
        # ---- reshape -> BGRA8 ----
        img_np = pixels.reshape(h, w, 4)
        # yield
        yield frame_ts, rr.Image(img_np)

As you can see, we must query each pyarrow column with hand-crafted deserializer instead of relying on the type automatically having decorators/macros under the hood to handle. Although this is a consequence of an OLAP / columnar storage format, that does not mean users must suffer !

Describe the solution you'd like
Akin to a typical ECS, we should be able to iter the table given a type

def iter_color_frames(rrd_path: str):
  rrd = rr.dataframe.load_recording(rrd_path)
  view = rrd.view(index=timeline, contents="...path")
  for row in view.select(rr.Image):
     ....

Describe alternatives you've considered
there's lots of ways to slice this problem ... but it can be measured by # lines of code for devs

Additional context
Mirror game engines with ECS semantics - like Bevy

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request👀 needs triageThis issue needs to be triaged by the Rerun team

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions