Skip to content

dc_bridge: Measurement custom keys are dropped on File metadata Records #419

Description

@Minipada

Problem

A Measurement's custom_key_str_list keys are appended to the JSON of the Records it emits, so
they reach a PostgreSQL Destination as columns like any other field. File metadata Records do not
get the same treatment: the Bridge's Uploader builds file_status and group_complete rows from a
fixed field set, so any custom key a Measurement declares is silently dropped on the File side.

The result is an asymmetry that is invisible until you go looking for it. Configure
custom_key_str_list: ["site"] on a camera Measurement and site appears in dc_records and never
in dc_files, with nothing logged and no error — the Records and the Files of the same Measurement
end up labelled differently.

What to build

Custom keys declared by a Measurement reach its File metadata Records the same way they reach its
Records, so a Measurement's labelling is consistent across both Destinations.

Acceptance criteria

  • file_status Records carry the custom keys declared in the emitting Measurement's custom_key_str_list
  • group_complete Records carry them too, so a File group and its members are labelled consistently
  • A key that collides with a field the Uploader already emits does not silently overwrite it; the behaviour is defined and tested
  • A Measurement declaring no custom keys produces byte-identical rows to today
  • Covered by an Uploader test that asserts on the emitted row, not on internal structure
  • doc/src/dc/measurements.md's description of custom_key_str_list reflects that the keys reach Files as well as Records

Notes

The columns still have to exist in the Destination — the PostgreSQL sink maps JSON keys onto
existing columns 1:1 and does not add them — so this change makes the keys available on the File
path, it does not create schema.

Blocked by

None — can start immediately.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    ready-for-agentPRD ready for agent implementation

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions