Metric description conflict: duplicate pending_sends_buffered metric across runtime-control and completion scopes

[andborja]

Pre-filing checklist

• I searched existing issues and didn't find a duplicate

Component(s)

Rust OTAP dataflow (rust/otap-dataflow/)

Bug Description

Description:

When the Prometheus pull exporter is configured, every scrape logs:

"Instrument description conflict, using existing"

Two metric sets in control_plane_metrics.rs declare a metric with the same name pending_sends.buffered but different descriptions:

• pipeline.runtime_control (RuntimeControlMetrics):

  otel-arrow/rust/otap-dataflow/crates/engine/src/control_plane_metrics.rs

  Line 76 in ebf2403

  #[metric(name = "pending_sends.buffered", unit = "{message}")]

• pipeline.completion (PipelineCompletionMetrics):

  otel-arrow/rust/otap-dataflow/crates/engine/src/control_plane_metrics.rs

  Line 253 in ebf2403

  #[metric(name = "pending_sends.buffered", unit = "{message}")]

The opentelemetry-prometheus exporter flattens all OTel scopes into a single Prometheus namespace keyed by metric name. When both scopes produce a metric named pending_sends_buffered, the first description seen wins and the second is silently dropped (see validate_metrics in the exporter):

https://github.com/open-telemetry/opentelemetry-rust/blob/main/opentelemetry-prometheus/src/lib.rs#L452

Although this is only a description conflict (both are Gauge, so no data is lost), it produces a noisy warning on every Prometheus scrape and means one of the two metrics loses its description.

Suggested fix:

Rename the metrics to disambiguate them.

Both introduced in commit 3dca283.

Steps to Reproduce

Configure a prometheus exporter for internal telemetry and run an instance of dataflow engine. Check the logs.

Expected Behavior

The conflict to be resolved, so the log message is no longer present.

Actual Behavior

The log message is present every time a scrapper is called over the prometheus endpoint.

OTel-Arrow Version

latest

Environment

No response

Configuration

Log Output

Additional Context

No response

[andborja]

Fixed at:
#2735

[cijothomas]

can these metrics be differentiated via scope name?

[andborja]

can these metrics be differentiated via scope name?

I'm not aware of the internal sdk supporting scope.

[cijothomas]

can these metrics be differentiated via scope name?

I'm not aware of the internal sdk supporting scope.

Not sure I follow. The metric-set name is used to create meter name:
https://github.com/open-telemetry/otel-arrow/blob/main/rust/otap-dataflow/crates/telemetry/src/metrics/dispatcher.rs#L62
It is totally valid to have same metric names, as long as they are in different scope (i.e Meters). And we cannot enforce metric names would be globally unique ever - it should be just unique within a scope only.

In other words, I am not following how renaming the metric in df_engine is a fix?