v0.5.18: fix: Treat rhs operand of composite ReportingSet as optional (#2239)

[SanjayVas]

Highlights

• Support for sample interval wrapping. This will improve privacy budget utilization and can be enabled after HMSS is enabled (feat: Allow sampling interval wrapping around 1 #2207)
• Measurement Methodology is now required and Metrics are reported as failed if their variance cannot be computed (refactor: Metric State is set to FAILED when Measurement Result issues are present #2238 )
• The Noise Corrector now generates diagnostics and provides a mechanism for logging them to a storage bucket. (feat: Report post processor returns data quality proto. #2205 feat: Write noise correction log to storage. #2242)
• The Reporting API has new GetBasicReport and ListBasicReports methods (Implement GetBasicReport and ListBasicReports methods #2196). This completes MC API Phase I.
  • These are supported by the new GetImpressionQualificationFilter and ListImpressionQualificationFilters methods (Implement ImpressionQualificationFilters service #2197)
  • This requires a Cloud Spanner database. If you are using the reporting Terraform module, it will create one for you.
  • To enable, set the --basic-reports-enabled option of the Reporting internal API server to true. When this option is true, the --impression-qualification-filter-config-file, --spanner-project, --spanner-instance, and --spanner-database options become required.
• The Reporting API has a new InvidateMetric method (Allow invalidation of Metrics #2195)
  • This will set the Metric state to INVALID and prevent it from being reused by future Report resources.
  • The state of Report resources that reference that Metric will be set to FAILED.
• The DataProvider resource in the CMMS API has a new data_availability_intervals field to specify the availability interval for each ModelLine (feat: Implement DataProviders/ReplaceDataAvailabilityIntervals #2188)
  • This is intended to replace the singular data_availability_interval field.
  • There is a corresponding ReplaceDataAvailabilityIntervals method.
• The EventGroup resource in the CMMS API has new event_group_metadata and media_types fields (Allow DataProviders to specify build number and warnings when fulfilling Requisitions #2190)
  • While the event_group_metadata field is intended to eventually replace the metadata field, but both fields should continue to be populated until the field is deprecated.
• The Requisition resource in the CMMS API has a new fulfillment_context field (Allow DataProviders to specify build number and warnings when fulfilling Requisitions #2190)
  • This allows DataProvider callers to specify a build label and warning information when fulfilling a Requistion.

Bugs Addressed

• "Missing value for metric" error on ListReports
• HMSS Variance is Reported as Zero
• CreateReportingSet throws INVALID_ARGUMENT when rhs isn't set

What's Changed

• feat: implement public IQF API by @roaminggypsy in feat: implement public IQF API #2122
• feat: Add Metrics Service Method for Invalidating a Metric by @tristanvuong2021 in feat: Add Metrics Service Method for Invalidating a Metric #2119
• feat!: Add the Public and Internal Basic Reports Services to the Reporting Servers by @tristanvuong2021 in feat!: Add the Public and Internal Basic Reports Services to the Reporting Servers #2097
• test: Update In-Process Integration Test to include Basic Reports by @tristanvuong2021 in test: Update In-Process Integration Test to include Basic Reports #2118
• feat: add control plane public api by @Marco-Premier in feat: add control plane public api #2126
• feat: update tee sdk to consume new ControlPlane API by @Marco-Premier in feat: update tee sdk to consume new ControlPlane API #2129
• feat: add servers for contorlplane api by @Marco-Premier in feat: add servers for contorlplane api #2141
• ci: Add test coverage report for cc tests by @laureanobrs in ci: Add test coverage report for cc tests #2145
• fix: Increase memory limit of Reporting public API server in cloud config by @SanjayVas in fix: Increase memory limit of Reporting public API server in cloud config #2146
• feat: Labeled impression proto by @jojijac0b in feat: Labeled impression proto #2157
• feat: Support media_types and event_group_metadata in EventGroup by @SanjayVas in feat: Support media_types and event_group_metadata in EventGroup #2151
• add InProcessControlPlane by @stevenwarejones in add InProcessControlPlane #2152
• feat: Add K8s configs for TEE ControlPlane API server by @Marco-Premier in feat: Add K8s configs for TEE ControlPlane API server #2144
• feat: Report post processor returns the correction result with status. by @ple13 in feat: Report post processor returns the correction result with status. #2166
• feat: Create encrypted dek proto by @jojijac0b in feat: Create encrypted dek proto #2159
• feat!: add internal iqf service to internal reporting server by @roaminggypsy in feat!: add internal iqf service to internal reporting server #2125
• feat: check impression qualification filter existence in insert basic report by @roaminggypsy in feat: check impression qualification filter existence in insert basic report #2139
• build: attach CLI uber jars to new releases by @laureanobrs in build: attach CLI uber jars to new releases #2174
• test: implement IQF integration test by @roaminggypsy in test: implement IQF integration test #2149
• feat: Support DataProvider.data_availability_intervals by @SanjayVas in feat: Support DataProvider.data_availability_intervals #2173
• ci: Add CodeCov integration by @laureanobrs in ci: Add CodeCov integration #2169
• feat: implement reporting cli command to invalidate metric, and tests by @roaminggypsy in feat: implement reporting cli command to invalidate metric, and tests #2168
• fix: Missing Metrics no longer cause a problem when retrieving Reports by @tristanvuong2021 in fix: Missing Metrics no longer cause a problem when retrieving Reports #2177
• feat: Implement DataProviders/ReplaceDataAvailabilityIntervals by @SanjayVas in feat: Implement DataProviders/ReplaceDataAvailabilityIntervals #2188
• feat: Support Requisition.fulfillment_context by @SanjayVas in feat: Support Requisition.fulfillment_context #2191
• feat: Add internal_api_server_image to Secure Computation by @renjiezh in feat: Add internal_api_server_image to Secure Computation #2204
• docs: Fix Reporting deployment guide w.r.t. BasicReports by @SanjayVas in docs: Fix Reporting deployment guide w.r.t. BasicReports #2199
• build: Fix attach-clis-to-release workflow not finding artifacts by @laureanobrs in build: Fix attach-clis-to-release workflow not finding artifacts #2202
• refactor: Remove unused imports from impression_qualification_filters_service.proto by @SanjayVas in refactor: Remove unused imports from impression_qualification_filters_service.proto #2206
• feat: Add media_types and event_group_metadata fields to Reporting EventGroup by @SanjayVas in feat: Add media_types and event_group_metadata fields to Reporting EventGroup #2153
• feat: Add data_availability_interval to Reporting EventGroup by @SanjayVas in feat: Add data_availability_interval to Reporting EventGroup #2176
• feat: Report post processor returns data quality proto. by @ple13 in feat: Report post processor returns data quality proto. #2205
• fix: Fix gcloud-postgres-update-schema in scan-image workflow by @renjiezh in fix: Fix gcloud-postgres-update-schema in scan-image workflow #2208
• Add DataWatcher by @stevenwarejones in Add DataWatcher #1990
• feat: add git workflow to deploy data watcher by @Marco-Premier in feat: add git workflow to deploy data watcher #2172
• feat: Add terraform script to run secure computation cloud test by @Marco-Premier in feat: Add terraform script to run secure computation cloud test #2175
• feat: add git workflow to push tee app images to artifact registry by @Marco-Premier in feat: add git workflow to push tee app images to artifact registry #2180
• feat: add secure computation deployment integration by @Marco-Premier in feat: add secure computation deployment integration #2212
• fix: filter out no-protocol duchy in requisition conversion from internal to public by @roaminggypsy in fix: filter out no-protocol duchy in requisition conversion from internal to public #2217
• feat!: Persist EventGroup data availability interval in DB columns by @SanjayVas in feat!: Persist EventGroup data availability interval in DB columns #2216
• fix: terraform file for edp aggregator by @Marco-Premier in fix: terraform file for edp aggregator #2235
• fix: Reduce batch size of BigQuery Metrics inserts by @tristanvuong2021 in fix: Reduce batch size of BigQuery Metrics inserts #2160
• feat: Allow sampling interval wrapping around 1 by @renjiezh in feat: Allow sampling interval wrapping around 1 #2207
• fix: public control plane api create proper internal api request by @stevenwarejones in fix: public control plane api create proper internal api request #2240
• chore: add Secure Computation Services Provider Rule by @stevenwarejones in chore: add Secure Computation Services Provider Rule #2228
• fix: fix prober by @roaminggypsy in fix: fix prober #2100
• fix: Remove artifact registry from mig terraform file by @tristanvuong2021 in fix: Remove artifact registry from mig terraform file #2243
• fix: Remove deploy cloud function from update cmms workflow by @tristanvuong2021 in fix: Remove deploy cloud function from update cmms workflow #2244
• fix: Variance calculation for hmss. by @ple13 in fix: Variance calculation for hmss. #2225
• feat: Write noise correction log to storage. by @ple13 in feat: Write noise correction log to storage. #2242
• refactor: Metric State is set to FAILED when Measurement Result issues are present by @tristanvuong2021 in refactor: Metric State is set to FAILED when Measurement Result issues are present #2238
• fix: Treat rhs operand of composite ReportingSet as optional by @SanjayVas in fix: Treat rhs operand of composite ReportingSet as optional #2239

Full Changelog: v0.5.17...v0.5.18-rc2

---

This discussion was created from the release v0.5.18: fix: Treat rhs operand of composite ReportingSet as optional (#2239).