Update span enrichment to clear the span ID and span Name

[isaacaflores2]

Overview

This PR updates the span enrichment to clear the span id and span name. This allows the span id and name to be only represented by the elastic attributes (span.id, span.name or transaction.id , transaction.name) only. The ES Exporter can then use both the elastic attributes and the SpanID and SpanName in the ECS encoder and the resulting document will contain the correct attributes based on the enrichment applied in the elasticapmprocessor/opentelemetry-lib.

The clear_span_id and clear_span_name configs are disabled by default for backwards compatibility. Collectors that do not require strict parity with the apm-data flow can keep these configs disabled (example should be the EDOT collectors). Collectors that would like to closely match the apm-data behavior can enable these configs as needed.

Related to: open-telemetry/opentelemetry-collector-contrib#45014 and elastic/opentelemetry-collector-components#943

Testing

• Updated unit
• Spin up local collector with es exporter to validate indexed documents. Example processor/elasticapm config:

processor:
 elasticapm:
      elastic_transaction:
        clear_span_id:
          enabled: true
        clear_span_name:
          enabled: true

[axw]

Is this relevant to ElasticSpanConfig? The struct's doc comment says:

// ElasticSpanConfig configures the enrichment attributes for the spans
// which are NOT identified as elastic transaction.

... so it should always be span.*, and never transaction.*?

[isaacaflores2]

Oh I messed up the config comments. The comments should mention span.id and span.name for the ElasticSpanConfig. And the ElasticTransactionConfig should mention transaction.id and transaction.name

[axw]

I see, makes sense now - thanks :)

[axw]

Actually... no, I'm still confused.

If we're enriching a span, and it should have a span.id, why would we need to clear span ID?

[axw]

Oh, I think I see now: we're adding an "id" attribute to the span attributes, and then clearing the top-level ID field. This seems a bit unnecessary. Why not just leave the top-level ID field set, and not add another attribute?

Could we remove "ID", "Name", "ClearSpanID", and "ClearSpanName" from ElasticSpanConfig?

[isaacaflores2]

@axw @gregkalapos Yes I think that makes sense since the es exporter will continue to use the top-level ID and Name fields. And my original intent was to avoid incorrectly adding span.* fields for Elastic Transactions. I will test this e2e locally to make sure I am not missing anything and push a change to remove "ID", "Name", "ClearSpanID", and "ClearSpanName" from ElasticSpanConfig.

[isaacaflores2]

I removed the above mention enrichments/configs from the ElasticSpanConfig. span.id and span.name continue to be added to the final document by the es exporter

[gregkalapos]

Looks ok to me.

I think @axw raises a good point that for spans this may not be need, only for transactions.

I don't really have a strong opinion on that - this way it's more consistent (spans and transactions are treated the same), if we drop the config for spans, then we don't do this "unnecessary" step for spans, but spans and transactions are treated differently.

To me this seems ok as is now in the PR - but would be ok with it, if we'd drop span configs as well.

[axw]

Thanks @isaacaflores2!