Component(s)
processor/cumulativetodelta
What happened?
Description
The cumulativetodeltaprocessor correctly drops sum and exponential histogram datapoints when a counter reset is detected (the new value is lower than the previous one (restart)). However, when a regular histogram is reset, the processor does not drop the point. Instead, it forwards the new datapoint with the full cumulative bucket counts and sum but tagged as AggregationTemporalityDelta.
This produces a corrupted delta: a one-time spike that includes both pre-reset and post-reset events. Downstream consumers will over count.
Where
processor/cumulativetodeltaprocessor/internal/tracking/tracker.go, in Convert for pmetric.MetricTypeHistogram:
delta := value.Clone()
// Calculate deltas unless histogram count was reset
if valid && delta.Count >= prevValue.Count {
delta.Count -= prevValue.Count
delta.Sum -= prevValue.Sum
for index, prevBucket := range prevValue.BucketCounts {
delta.BucketCounts[index] -= prevBucket
}
}
out.HistogramValue = &delta // when reset, this is the full cumulative value, not a delta
When delta.Count < prevValue.Count the subtraction is skipped but the point is still returned with valid=true, so the processor emits it as a delta.
Steps to Reproduce
Send the following sequence to the processor (one cumulative histogram metric, three points):
| t |
count |
sum |
bucket_counts |
| 1 |
10 |
100 |
[10] |
| 2 |
20 |
200 |
[20] |
| 3 |
5 |
50 |
[5] |
Expected Result
Match the sum / exponential histogram paths: on reset, mark the point invalid so the processor drops it (keep in memory waiting next datapoint to compute the delta, if >=).
| t |
count |
sum |
bucket_counts |
| 2 |
10 |
100 |
[10] |
| (t=3 dropped) |
|
|
|
Actual Result
| t |
count |
sum |
bucket_counts |
| 2 |
10 |
100 |
[10] |
| 3 |
5 |
50 |
[5] |
I attached a diff file with a unit test to reproduce that bug
cumulativetodelta-histo-reset-bug.patch
Collector version
main
Environment information
Environment
N/A — bug is in the processor logic itself.
OpenTelemetry Collector configuration
processors:
cumulativetodelta:
Log output
Additional context
No response
Tip
React with 👍 to help prioritize this issue. Please use comments to provide useful context, avoiding +1 or me too, to help us triage it. Learn more here.
Component(s)
processor/cumulativetodelta
What happened?
Description
The
cumulativetodeltaprocessorcorrectly drops sum and exponential histogram datapoints when a counter reset is detected (the new value is lower than the previous one (restart)). However, when a regular histogram is reset, the processor does not drop the point. Instead, it forwards the new datapoint with the full cumulative bucket counts and sum but tagged asAggregationTemporalityDelta.This produces a corrupted delta: a one-time spike that includes both pre-reset and post-reset events. Downstream consumers will over count.
Where
processor/cumulativetodeltaprocessor/internal/tracking/tracker.go, inConvertforpmetric.MetricTypeHistogram:When
delta.Count < prevValue.Countthe subtraction is skipped but the point is still returned withvalid=true, so the processor emits it as a delta.Steps to Reproduce
Send the following sequence to the processor (one cumulative histogram metric, three points):
Expected Result
Match the sum / exponential histogram paths: on reset, mark the point invalid so the processor drops it (keep in memory waiting next datapoint to compute the delta, if >=).
Actual Result
I attached a diff file with a unit test to reproduce that bug
cumulativetodelta-histo-reset-bug.patch
Collector version
main
Environment information
Environment
N/A — bug is in the processor logic itself.
OpenTelemetry Collector configuration
Log output
Additional context
No response
Tip
React with 👍 to help prioritize this issue. Please use comments to provide useful context, avoiding
+1orme too, to help us triage it. Learn more here.