Component(s)

exporter/prometheus

What happened?

Description

After upgrading otel collector contrib distro from v0.117.0 to v0.118.0 we noticed suddenly we were missing otel collector metrics, i.e. otelcol_exporter_queue_capacity for the collector. After some investigation we found that that this was caused by the sevice.name label on the metrics no longer being translated to service_name and since our backend doesn't support full utf-8 yet it was refusing these metrics now.

Originally opened as bug on Prometheus prometheus/prometheus#16027.
This is caused by this PR #37375 upgrading the prometheus common dependency.

The changelog warns:
⚠️ This releases switches internal global to UTF8Validation from LegacyValidation. This is a breaking change, relaxing the validation. We don't intend to add more schemas and we have to have a global for unmarshalling interfaces, thus the change was made ⚠️

As it warns this is a breaking change, especially for people exporting these metrics in prometheus format into backend which do not yet support utf-8.

Proposed fix PR #37938, tested locally it reverts to the original behavior escaping the dots with underscores, if this solution is acceptable I'll make it a full PR with changelog test cases etc.

Steps to Reproduce

1. Run otel collector build v0.118.0 or newer
2. Scrape the metrics from the otel collector with Prometheus 3.X

Expected Result

For the metric labels to be still translated to underscores

Actual Result

Dots in labels no longer being translated to underscores

Collector version

v0.118.0

Environment information

Environment

OS: (e.g., "Ubuntu 20.04")
Compiler(if manually compiled): (e.g., "go 14.2")

OpenTelemetry Collector configuration

receivers:
  otlp:
    protocols:
      grpc:
        endpoint: 0.0.0.0:4317
      http:
        endpoint: 0.0.0.0:4318
  prometheus:
    config:
      scrape_configs:
        - job_name: otel-collector
          scrape_interval: 5s
          static_configs:
            - targets: [localhost:8888]
exporters:
  otlphttp/thanos:
    endpoint: "http://localhost:10908"
    tls:
      insecure: true
  debug:
    verbosity: detailed
  prometheusremotewrite/thanos:
    endpoint: "http://localhost:10908/api/v1/write"
    resource_to_telemetry_conversion:
      enabled: false
    target_info:
      enabled: true
processors:
extensions:
  health_check:
  pprof:
service:
  telemetry:
    logs:
      level: "debug"
  extensions: [pprof, health_check]
  pipelines:
    metrics:
      receivers:
        - prometheus
        - otlp
      processors:
      exporters:
        - prometheusremotewrite/thanos

Log output

Additional context

No response