Description
Hello team,
I would like to reach out with a small enhancement request, please.
Currently, this logstash logback encoder is doing a great job forwarding application logs to Logstash (which will then send them to ElasticSearch).
However, in ElasticSearch, they live inside a regular index.
Would it be possible from this project to send the logs to ElasticSearch, but to a DataStream, not just a regular index:
https://www.elastic.co/guide/en/elasticsearch/reference/current/data-streams.html
(Copy paste from ElasticSearch official doc:)
A data stream lets you store append-only time series data across multiple indices while giving you a single named resource for requests. Data streams are well-suited for logs, events, metrics, and other continuously generated data.
(see screenshot for datastream)
Currently, this is only create a plain index (please see screenshot)
Rationale
The reason behind this request is that by definition, logs are very well suited for ElasticSearch DataStream, because they are data stream in the first place.
However, as of now, one needs to manually convert the log index into DataStream from the regular Index on the ElasticSearch backend in order to benefit from the synergy between this encoder and elasticsearch data stream.
Would be a great feature if this encoder can directly push the logs as it is doing today, directly to a data stream instead.
Thank you