-
Notifications
You must be signed in to change notification settings - Fork 3.5k
Allow to specify an ingest pipeline to be used before writing documents #47227
Copy link
Copy link
Open
Labels
Description
Component(s)
exporter/opensearch
Is your feature request related to a problem? Please describe.
An optional ingest pipeline in opensearch would allow post-processing of the received documents (by the collector) and before writing the documents into opensearch. Options could be:
- adjust
mappingof incoming documents - enrich documents
- trigger other pipelines / notifications
- ...
Describe the solution you'd like
The opensearchclient bulk indexer (https://github.com/opensearch-project/opensearch-go/blob/main/opensearchutil/bulk_indexer.go#L86) already supports a pipeline option to be passed. The opensearchexporter? just needs to allow such an pipeline option and pass it to the client.
Describe alternatives you've considered
No response
Additional context
https://docs.opensearch.org/latest/ingest-pipelines/
Tip
React with 👍 to help prioritize this issue. Please use comments to provide useful context, avoiding +1 or me too, to help us triage it. Learn more here.
Reactions are currently unavailable