Skip to content

[exporter/elasticsearch] Multiple identical spans can result in duplicate documents in elasticsearch #43649

@bxYu

Description

@bxYu

Component(s)

exporter/elasticsearch

Is your feature request related to a problem? Please describe.

Phenomenon

Due to some inevitable special circumstances, multiple identical spans may enter es exporter, and eventually multiple redundant documents can be seen in ES

The reasons for this phenomenon

In exporter.go, when the pushTraceRecord function returns
bulkIndexerSession.Add(ctx, index.Index, "", "", buf, nil, docappender.ActionCreate),
the docID field is an empty string, which causes the downstream to automatically generate a non-repetitive docID to write to ES. Therefore, multiple identical spans will correspond to multiple docIDs, thus being written to ES multiple times

Describe the solution you'd like

Feature Request

Could a configuration item be added to support "docID generated based on SpanID", so that multiple identical spans can only correspond to one document in ES? This is very important to us. Appreciate!

Describe alternatives you've considered

No response

Additional context

No response

Tip

React with 👍 to help prioritize this issue. Please use comments to provide useful context, avoiding +1 or me too, to help us triage it. Learn more here.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions