Skip to content

fix(dc_bridge): carry Measurement custom keys onto File metadata Records - #420

Merged
Minipada merged 1 commit into
jazzyfrom
feature/419-dc-bridge-measurement-custom-keys-are-dr
Aug 21, 2026
Merged

fix(dc_bridge): carry Measurement custom keys onto File metadata Records#420
Minipada merged 1 commit into
jazzyfrom
feature/419-dc-bridge-measurement-custom-keys-are-dr

Conversation

@Minipada

Copy link
Copy Markdown
Owner

Closes #419

Problem

A Measurement's custom_key_str_list keys are appended to the JSON of its Records, so they
reach a PostgreSQL Destination as columns. The Uploader built its file_status and
group_complete rows from a fixed field set, so the same keys were silently dropped on
the File side: configure custom_key_str_list: ["site"] on a camera Measurement and site
appeared in dc_records and never in dc_files, with nothing logged.

What changed

The Record names its custom keys. Measurement::addCustomKeys() now writes a
custom_keys array alongside the values it already inlines. Without it the Uploader has no
way to tell site from a measured field — every top-level key of a Record looks the same
from the Bridge. A Measurement with no custom keys writes nothing, exactly as before.

The Uploader carries them onto every File metadata row. parse_file_group() reads the
declaration and its values into FileGroup::custom_keys; status.cpp appends them last,
to uploaded/missing/deleted rows, retention's shed rows, and the group_complete
marker, so a File group and its members are labelled the same way.

Collisions

Two kinds of key are dropped instead of written — in both cases from every row kind, not
just the ones that happen to carry the field, so a File and its group marker can never
disagree about what a name means:

Key Behaviour
Names a field the Uploader computes (storage_type, size, remote_path, …) The Uploader's value is kept; the key is reported in ProcessSummary::dropped_custom_keys and the Bridge logs a warning
name, id, robot_name Not repeated and not reported — the rows already carry those values as group_name, robot_id and robot_name

The second row matters for the demos, which all configure
custom_key_str_list: ["robot_name", "id"]. Emitting a bare id would write a robot
identifier into whatever an id column happens to be — a serial primary key, for instance.

Scope

The columns still have to exist in the Destination. The PostgreSQL sink maps JSON keys onto
existing columns 1:1, so this makes the keys available on the File path; it does not
create schema.

Group-merged Records are unaffected: the Group node namespaces its members' fields under
their group_key, so those keys are no longer the Measurement's own labelling and the
parse is deliberately top-level only.

Tests

dc_bridge/test/uploader_test.cpp (4 new, asserting on the emitted rows):

  • custom keys reach every file_status row (uploaded + deleted) and the group_complete marker
  • a key naming a computed field never overwrites it and is reported in the summary
  • robot_name/id are not repeated and not reported
  • with no custom keys, both row kinds have exactly the key set they had before

dc_measurements/test/test_measurement_dummy.cpp (2 new): a Record with custom keys carries
the value and the declaration; without them the Record is untouched.

colcon test: dc_bridge 169/169, dc_measurements 295/295.

Docs

doc/src/dc/measurements.mdcustom_key_str_list's row, plus a "Custom keys on Files"
section covering the collision rule and the ALTER TABLE caveat.

🤖 Generated with Claude Code

https://claude.ai/code/session_015oGKFjBD4WKxMHYv3iKXBo

A Measurement's `custom_key_str_list` keys reached its Records but not the
Uploader's `file_status`/`group_complete` rows, which were built from a fixed
field set — so the Records and the Files of one Measurement ended up labelled
differently, with nothing logged.

A Record now names its custom keys in a `custom_keys` field, which is the only
way the Uploader can tell `site` from a measured field. Those keys are appended
to every File metadata row, including retention's shed rows.

Dropped rather than written, in both cases from every row kind so a File and its
group marker can't disagree:

- a key naming a field the Uploader computes itself (`storage_type`, `size`, …)
  — the Uploader's value is kept and the Bridge warns;
- `name`, `id` and `robot_name`, which the rows already carry as `group_name`,
  `robot_id` and `robot_name` — re-emitting `id` would put a robot identifier in
  whatever an `id` column happens to be.

A Measurement with no custom keys produces the rows it did before.

Closes #419

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015oGKFjBD4WKxMHYv3iKXBo
Signed-off-by: David Bensoussan <d.bensoussan@proton.me>
@codecov

codecov Bot commented Aug 21, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 81.57895% with 7 lines in your changes missing coverage. Please review.
✅ Project coverage is 71.12%. Comparing base (0ed5a64) to head (68f2ce0).

Files with missing lines Patch % Lines
dc_bridge/src/bridge_node.cpp 0.00% 5 Missing ⚠️
dc_bridge/src/uploader/group.cpp 80.00% 2 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##            jazzy     #420      +/-   ##
==========================================
+ Coverage   70.80%   71.12%   +0.33%     
==========================================
  Files         114      114              
  Lines        7440     7478      +38     
==========================================
+ Hits         5267     5318      +51     
+ Misses       2173     2160      -13     
Flag Coverage Δ
cpp-jazzy 71.12% <81.58%> (+0.33%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@Minipada
Minipada merged commit 2eab34c into jazzy Aug 21, 2026
15 of 16 checks passed
@github-actions
github-actions Bot deleted the feature/419-dc-bridge-measurement-custom-keys-are-dr branch August 21, 2026 15:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant