[beats receivers] OTel based pipeline must continue to report output metrics for the elastic_agent integration

[cmacknz]

The Beat pipeline and output produce detailed metrics, when using a collector based pipeline we must produce equivalent or similar metrics from the exporter. The output metrics related to errors, retries, etc are heavily featured in the elastic_agent integration.

Ideally the metrics from the collector are backwards compatible with the existing metrics. If they cannot have the same names then the same semantic meaning is the next best thing. We are generally not concerned with breaking changes here.

https://github.com/elastic/integrations/blob/037cda7c46f84ca7c1d8055105f5d7fab59fda50/packages/elastic_agent/data_stream/filebeat_metrics/fields/beat-stats-fields.yml#L229-L279

        - name: output
          type: group
          fields:
            - name: type
              type: keyword
              description: >
                Type of output
            - name: events
              type: group
              fields:
                - name: acked
                  type: long
                  metric_type: counter
                  description: >
                    Number of events acknowledged

                - name: active
                  type: long
                  metric_type: counter
                  description: >
                    Number of active events
                - name: batches
                  type: long
                  metric_type: counter
                  description: >
                    Number of event batches
                - name: dropped
                  type: long
                  metric_type: counter
                  description: >
                    Number of events dropped
                - name: duplicates
                  type: long
                  metric_type: counter
                  description: >
                    Number of events duplicated
                - name: failed
                  type: long
                  metric_type: counter
                  description: >
                    Number of events failed
                - name: toomany
                  type: long
                  metric_type: counter
                  description: >
                    Number of too many events
                - name: total
                  type: long
                  metric_type: counter
                  description: >
                    Total number of events processed by the output

[elasticmachine]

Pinging @elastic/elastic-agent-data-plane (Team:Elastic-Agent-Data-Plane)

[leehinman]

https://opentelemetry.io/docs/collector/internal-telemetry/