Skip to content

Commit 5c453b9

Browse files
kongchen1992meta-codesync[bot]
authored andcommitted
{Feature} Core - Sync EyeGazeConfigurationLayout to v2
Summary: Explanation: Extend `EyeGazeConfigurationLayout` to v2 so `EyeGazePlayer` surfaces three additional per-stream configuration fields: - `user_calibration_params_json`: verbatim per-user calibration parameters JSON for the ML eye-tracking source; empty for the geometric source or when the user has not been calibrated. - `field_provenance_single`: packed uint32 (2 bits per slot, indexed by `SingleFieldId`) describing where each per-eye field comes from — direct sensor output, hardcoded constant, calculated from other fields, or not produced at all. - `field_provenance_combined`: same idea for combined / whole-frame fields, indexed by `CombinedFieldId`. Also closes a pre-existing gap where `algorithm_name` / `algorithm_version` were declared in the DataLayout but never copied into `EyeGazeConfiguration`, so downstream consumers could not tell which eye-tracking implementation produced the data. New types under `projectaria::tools::datalayout`: - `enum class FieldProvenance : uint8_t` with values `SUPPORTED (0)`, `CALCULATED (1)`, `HARDCODED (2)`, `NOT_PRODUCED (3)` — the numeric values ARE the on-disk wire format (2 bits per slot). - `enum class SingleFieldId : uint32_t` (5 slots: `GazeOrigin`, `GazeDirection`, `EntrancePupilPosition`, `PupilDiameter`, `Blink`). - `enum class CombinedFieldId : uint32_t` (6 slots: `GazeOriginCombined`, `GazeDirectionCombined`, `ConvergenceDistance`, `InterocularDistance`, `FoveatedGaze`, `SpatialGazePoint`). - `constexpr` template helpers `getFieldProvenance(packed, id)` / `setFieldProvenance(packed, id, provenance)` to unpack / pack a single slot. The `EyeGazeConfiguration` struct exposes the raw packed uint32s as `field_provenance_single` / `field_provenance_combined` and typed accessor methods `getSingleFieldProvenance(SingleFieldId)` / `getCombinedFieldProvenance(CombinedFieldId)` that return `FieldProvenance` values directly. Backwards-compatible with recordings that pre-date these fields: VRS `DataPiece::get()` returns each field's default (empty string / 0) when the field is absent on disk. `0` decodes as `SUPPORTED` for every slot, which is the correct semantics for a source that produces every signal directly. Python bindings under `projectaria_tools.core.sensor_data` expose the same enums (as `enum.IntEnum` in the type stubs) and the same wrapper methods, so Python callers can either read `config.field_provenance_single` as an `int` or call `config.get_single_field_provenance(SingleFieldId.GAZE_ORIGIN)` for a typed result. Reproducibility: Four new C++ tests in `core/data_provider/test/EyeGazePlayerTest.cpp` cover the packed-uint32 helpers (round-trip + cross-slot non-interference), the numeric wire-format contract, an end-to-end v2 config read via `EyeGazePlayer`, and a v1 back-compat read using a locally-defined stripped config layout. One new Python test in `core/python/test/corePyBindTest.py` reads the checked-in Gen2 fixture, asserts the v2 fields fall back to defaults on that (v1-era) fixture, and pins the Python-visible enum values against the wire format. Reviewed By: PiotrBrzyski Differential Revision: D113303274 fbshipit-source-id: 5f1233fbd651785ab8e34ddbce4c9c2b979e7ec8
1 parent fa5b58d commit 5c453b9

7 files changed

Lines changed: 457 additions & 2 deletions

File tree

core/data_provider/data_layout/EyeGazeMetadata.h

Lines changed: 53 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -21,9 +21,51 @@
2121

2222
namespace projectaria::tools::datalayout {
2323

24+
// 2 bits per slot on the packed uint32. Numeric values are the on-disk
25+
// wire format and must not be renumbered.
26+
enum class FieldProvenance : uint8_t {
27+
SUPPORTED = 0, // direct sensor/model output
28+
CALCULATED = 1, // derived on-device from other valid fields
29+
HARDCODED = 2, // constant from device calibration; valid==true but does not track state
30+
NOT_PRODUCED = 3, // source does not populate the field; valid==false
31+
};
32+
33+
// Slot indices (2 bits each) inside EyeGazeConfigurationLayout::fieldProvenanceSingle.
34+
enum class SingleFieldId : uint32_t {
35+
GazeOrigin = 0,
36+
GazeDirection = 1,
37+
EntrancePupilPosition = 2,
38+
PupilDiameter = 3,
39+
Blink = 4,
40+
};
41+
42+
// Slot indices (2 bits each) inside EyeGazeConfigurationLayout::fieldProvenanceCombined.
43+
enum class CombinedFieldId : uint32_t {
44+
GazeOriginCombined = 0,
45+
GazeDirectionCombined = 1,
46+
ConvergenceDistance = 2,
47+
InterocularDistance = 3,
48+
FoveatedGaze = 4,
49+
SpatialGazePoint = 5,
50+
};
51+
52+
template <typename FieldId>
53+
constexpr FieldProvenance getFieldProvenance(uint32_t packed, FieldId id) {
54+
const auto shift = static_cast<uint32_t>(id) * 2u;
55+
return static_cast<FieldProvenance>((packed >> shift) & 0b11u);
56+
}
57+
58+
template <typename FieldId>
59+
constexpr uint32_t setFieldProvenance(uint32_t packed, FieldId id, FieldProvenance p) {
60+
const auto shift = static_cast<uint32_t>(id) * 2u;
61+
return (packed & ~(0b11u << shift)) | ((static_cast<uint32_t>(p) & 0b11u) << shift);
62+
}
63+
2464
class EyeGazeConfigurationLayout : public vrs::AutoDataLayout {
2565
public:
26-
static constexpr uint32_t kVersion = 1;
66+
// v2 adds user_calibration_params_json + field_provenance_{single,combined};
67+
// VRS field mapping is name-based so older readers skip transparently.
68+
static constexpr uint32_t kVersion = 2;
2769
vrs::DataPieceValue<uint32_t> streamId{"stream_id"};
2870
// Algorithm version to distinguish different eye tracking implementations. Formatted in string
2971
// as "Major.minor",
@@ -37,6 +79,16 @@ class EyeGazeConfigurationLayout : public vrs::AutoDataLayout {
3779
// Indicates the accuracy of the user ET calibration. Lower is better
3880
vrs::DataPieceValue<float> userCalibrationError{"user_calibration_error"};
3981

82+
// v2: verbatim calibration_params.json for the ML eye-tracking source.
83+
// Empty for the geometric source or when userCalibrated == false.
84+
vrs::DataPieceString userCalibrationParamsJson{"user_calibration_params_json"};
85+
// v2: per-field provenance for per-eye fields on SingleEyeGazeLayoutStruct.
86+
// Packed 2 bits/slot, indexed by SingleFieldId. Applies to both eyes.
87+
vrs::DataPieceValue<uint32_t> fieldProvenanceSingle{"field_provenance_single"};
88+
// v2: per-field provenance for combined/whole-frame fields on EyeGazeLayout.
89+
// Packed 2 bits/slot, indexed by CombinedFieldId.
90+
vrs::DataPieceValue<uint32_t> fieldProvenanceCombined{"field_provenance_combined"};
91+
4092
vrs::AutoDataLayoutEnd endLayout;
4193
};
4294

core/data_provider/players/EyeGazePlayer.cpp

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -211,6 +211,14 @@ bool EyeGazePlayer::onDataLayoutRead(
211211
configRecord_.nominalRateHz = configLayout.nominalRateHz.get();
212212
configRecord_.userCalibrated = configLayout.userCalibrated.get();
213213
configRecord_.userCalibrationError = configLayout.userCalibrationError.get();
214+
// v2 fields; VRS's no-arg .get() returns the DataPiece's default when
215+
// the field is not present on disk (v1 recording), which matches our
216+
// struct-member defaults (empty string / 0 == all-SUPPORTED).
217+
configRecord_.algorithmName = configLayout.algorithmName.get();
218+
configRecord_.algorithmVersion = configLayout.algorithmVersion.get();
219+
configRecord_.userCalibrationParamsJson = configLayout.userCalibrationParamsJson.get();
220+
configRecord_.fieldProvenanceSingle = configLayout.fieldProvenanceSingle.get();
221+
configRecord_.fieldProvenanceCombined = configLayout.fieldProvenanceCombined.get();
214222
return true;
215223
} else if (header.recordType == vrs::Record::Type::DATA) {
216224
// Read data as datalayout data type

core/data_provider/players/EyeGazePlayer.h

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -22,6 +22,7 @@
2222

2323
#include <sophus/se3.hpp>
2424
#include <optional>
25+
#include <string>
2526
#include <utility>
2627

2728
namespace projectaria::tools::data_provider {
@@ -31,6 +32,23 @@ struct EyeGazeConfiguration {
3132
double nominalRateHz;
3233
bool userCalibrated;
3334
float userCalibrationError;
35+
36+
// Populated from EyeGazeConfigurationLayout v2. For v1 recordings, VRS
37+
// returns each DataPiece's default (empty string / 0 == all-SUPPORTED).
38+
std::string algorithmName;
39+
std::string algorithmVersion;
40+
std::string userCalibrationParamsJson;
41+
uint32_t fieldProvenanceSingle{0};
42+
uint32_t fieldProvenanceCombined{0};
43+
44+
[[nodiscard]] datalayout::FieldProvenance getSingleFieldProvenance(
45+
datalayout::SingleFieldId id) const {
46+
return datalayout::getFieldProvenance(fieldProvenanceSingle, id);
47+
}
48+
[[nodiscard]] datalayout::FieldProvenance getCombinedFieldProvenance(
49+
datalayout::CombinedFieldId id) const {
50+
return datalayout::getFieldProvenance(fieldProvenanceCombined, id);
51+
}
3452
};
3553

3654
using EyeGazeCallback =

core/data_provider/test/CompareDataHelper.h

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -401,6 +401,11 @@ inline void compare(
401401
EXPECT_DOUBLE_EQ(eyeGazeConfig1.nominalRateHz, eyeGazeConfig2.nominalRateHz);
402402
EXPECT_EQ(eyeGazeConfig1.userCalibrated, eyeGazeConfig2.userCalibrated);
403403
EXPECT_FLOAT_EQ(eyeGazeConfig1.userCalibrationError, eyeGazeConfig2.userCalibrationError);
404+
EXPECT_EQ(eyeGazeConfig1.algorithmName, eyeGazeConfig2.algorithmName);
405+
EXPECT_EQ(eyeGazeConfig1.algorithmVersion, eyeGazeConfig2.algorithmVersion);
406+
EXPECT_EQ(eyeGazeConfig1.userCalibrationParamsJson, eyeGazeConfig2.userCalibrationParamsJson);
407+
EXPECT_EQ(eyeGazeConfig1.fieldProvenanceSingle, eyeGazeConfig2.fieldProvenanceSingle);
408+
EXPECT_EQ(eyeGazeConfig1.fieldProvenanceCombined, eyeGazeConfig2.fieldProvenanceCombined);
404409
}
405410

406411
inline void compare(

0 commit comments

Comments
 (0)