[WIP] Live check metrics

[jerbly]

[jsuereth]

Interesting - Is the convention going to be check at the highest level, but higher-level things don't include lower level details?

E.g. NumberDataPoint is part of a Metric and has metric identifying things, but SampleMetric wouldn't?

Also, can you rename to SampledNumberDataPoint?

[jerbly]

This area is not quite right yet. A NumberDataPoint without the information from the Metric is not so useful, but I need a way to pass it in around in a consistent way.

Yes, good call on the renaming. Will do.

[jerbly]

I'm looking into the consistency of the framework. For each data_point, I want to be able to provide advice at that level (checking attributes required for example), but I need context from the parent group. I'm hoping to get this change done today. This will change rego so the input is a tuple of (sample,semconv).

[jerbly]

Just pushed this with docs to explain and a rego example.

# This example shows how to use the registry_group provided in the input.
# If the metric's unit is "By" the value in this data-point must be an integer.
deny contains make_advice(advice_type, advice_level, value, message) if {
	input.sample.number_data_point
	value := input.sample.number_data_point.value
	input.registry_group.unit == "By"
	value != floor(value) # not a good type check, but serves as an example
	advice_type := "invalid_data_point_value"
	advice_level := "violation"
	message := "Value must be an integer when unit is 'By'"
}

[jsuereth]

Can you add a check to make sure attributes on the metric match attributes defined on the group?

[jerbly]

This is already done internally in rust rather than in rego since I assumed this was a fundamental thing. See the attribute_required advice under each data_point in the screenshot above. This is at the data_point level since the attributes are provided with each point.

[codecov]

Codecov Report

Attention: Patch coverage is 87.33333% with 19 lines in your changes missing coverage. Please review.

Project coverage is 76.9%. Comparing base (6301f33) to head (5028a2b).

Files with missing lines	Patch %	Lines
crates/weaver_live_check/src/advice.rs	73.9%	18 Missing ⚠️
crates/weaver_live_check/src/sample_metric.rs	97.8%	1 Missing ⚠️

Additional details and impacted files

@@           Coverage Diff           @@
##            main    #728     +/-   ##
=======================================
+ Coverage   76.7%   76.9%   +0.2%     
=======================================
  Files         65      66      +1     
  Lines       5012    5146    +134     
=======================================
+ Hits        3846    3960    +114     
- Misses      1166    1186     +20     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.