Self-observability: Allow receivers to report partial success

[dashpole]

Component(s)

receiver/receiverhelper

Is your feature request related to a problem? Please describe.

The receiverhelper self-observability for receivers currently assumes that if an error occurred, then the entire batch fails:

opentelemetry-collector/receiver/receiverhelper/obsreport.go

Lines 184 to 196 in 8ba3ae3

	if err != nil {
	numAccepted = 0
	// If gate is enabled, we distinguish between refused and failed.
	if NewReceiverMetricsGate.IsEnabled() {
	if consumererror.IsDownstream(err) {
	numRefused = numReceivedItems
	} else {
	numFailedErrors = numReceivedItems
	}
	} else {
	// When the gate is disabled, all errors are considered "refused".
	numRefused = numReceivedItems
	}

. In the Prometheus receiver, we would like to be able to record "translation failures" using standard collector metrics, instead of inventing our own: open-telemetry/opentelemetry-collector-contrib#45233 (review)

Describe the solution you'd like

I would like to add a new error type, similar to NewMetrics (for exporters), which allows receivers to pass the elements that the receiver rejected.

I'm not sure if it these should be considered "dropped" or "rejected", but "rejected" seems correct to me.

Describe alternatives you've considered

Add our own metric and track it separately

Additional context

cc @open-telemetry/prometheus-interoperability

Tip

_{React with 👍 to help prioritize this issue. Please use comments to provide useful context, avoiding +1 or me too, to help us triage it. Learn more here.}

[dashpole]

#13423 is related

[braydonk]

I had a PR for this where we had some discussion: #14152

I was trying to target this same shortcoming the receiverhelper has, but in exporterhelper. The solution ends up being largely the same though.

I have a branch on my fork that is based on #14152 which shows how exporterhelper will consume the error. #14152

However in this case I did go with the "dropped" semantics. I wasn't particularly aware that we wanted to make a distinction between "rejected" and "dropped", but I was going with "dropped" because I was targeting scenarios where we know the number that failed but not exactly which ones, making them unrecoverable. The case where errors are recoverable (i.e. the aforementioned Metrics errors) makes more sense for "rejected" because we can say which particular metrics were rejected by the consumer and recover/retry them. But I'm not sure I have the particular meaning between those two clear in my head.

[braydonk]

I am going to start on an RFC around this on Wednesday to hopefully cover both the receiverhelper and exporterhelper use cases.

[Sanchit2662]

Overview

This proposal introduces a new error type, consumererror.PartialError, to allow receivers to report partial success when processing batches.

Today, if a receiver returns any error, receiverhelper assumes the entire batch failed (numAccepted = 0), even when some items were processed successfully. This makes receiver metrics inaccurate.
The goal of this API is to let receivers explicitly report how many items failed, enabling correct observability without changing existing behavior.

---

API design

package consumererror

type PartialError struct {
    err         error
    failedCount int
}

func NewPartialError(err error, failedCount int) error
func GetPartialError(err error) *PartialError
func (p *PartialError) FailedCount() int

• failedCount represents the number of items that failed
• The wrapped error describes why those items failed
• Receivers that don’t use this API remain unaffected

---

Intended usage

Receivers may return PartialError when:

• some items are successfully processed
• some items fail due to parsing/translation issues or downstream rejection

If the error is wrapped with consumererror.NewDownstream(...), the failed items are treated as refused; otherwise they are treated as receiver-local failures.

---

receiverhelper behavior

When an error is returned:

• If a PartialError is present:

  • failed = FailedCount()
  • accepted = totalItems - failed

• Otherwise:

  • existing behavior is preserved (entire batch failed)

This enables accurate accepted, failed, and refused receiver metrics while remaining backward compatible.

---

Error composition

PartialError composes cleanly with existing consumer error wrappers:

• PartialError → receiver-local partial failure
• Downstream(PartialError) → downstream rejected some items
• Permanent(PartialError) → non-retryable partial failure

---

Alternatives considered

• Signal-specific partial error APIs → larger API surface and duplication
• Returning accepted count instead of failed count → less intuitive than error-based reporting