Versions:
- Elastic Agent: 9.4.2
- Logstash: 9.4.2
- Elasticsearch: 9.4.2
Operating System(s):
(All agents are Fleet-managed.)
Summary
When Elastic Agent's metrics collection ("Collect agent metrics") is enabled (in Agent Policies->Settings->Agent monitoring), and data is routed through Logstash, documents in .metrics-* data streams are duplicated, resulting in 409 version_conflict_engine_exception errors during indexing (Logstash errors).
The issue appears to be specific to Elastic Agent metrics (not logs). Disabling agent metrics collection immediately stops both the metrics traffic and the version conflict errors.
Environment
- Fleet-managed Elastic Agents (Deployed on several Windows and Linux Hosts)
- Logstash used as output between Elastic Agent and Elasticsearch
- Metrics and logs use the same Logstash output
Steps to Reproduce
Deploy a Fleet-managed Elastic Agent (tested on both Linux and Windows).
Configure the agent to send data through Logstash (Logstash Output).
Enable Collect agent metrics in the Fleet policy.
Expected Behavior
Agent metrics documents should be indexed once without conflicts and errors.
Actual Behavior
Documents in .metrics-* data streams seem to get indexed multiple times, resulting in 409 version_conflict_engine_exception errors.
The conflicts occur consistently while agent metrics collection is enabled.
Additional Findings
Disabling Collect agent metrics immediately stops:
.metrics-* traffic
409 version_conflict_engine_exception errors
The issue is reproducible with minimal Fleet policies and no additional integrations enabled.
Agent diagnostics confirm that logs and metrics share the same Logstash output configuration.
The behavior occurs with both:
- a single Logstash host
- multiple Logstash hosts with loadbalance: true in Elastic Agent configs
Other logs and integration data do not appear to exhibit the same duplication behavior.
The issue appears isolated to Elastic Agent metrics data streams.
Questions
Is this a known issue affecting Elastic Agent metrics when routed through Logstash?
Is this expected behaviour for metrics?
Versions:
Operating System(s):
(All agents are Fleet-managed.)
Summary
When Elastic Agent's metrics collection ("Collect agent metrics") is enabled (in Agent Policies->Settings->Agent monitoring), and data is routed through Logstash, documents in .metrics-* data streams are duplicated, resulting in 409
version_conflict_engine_exceptionerrors during indexing (Logstash errors).The issue appears to be specific to Elastic Agent metrics (not logs). Disabling agent metrics collection immediately stops both the metrics traffic and the version conflict errors.
Environment
Steps to Reproduce
Deploy a Fleet-managed Elastic Agent (tested on both Linux and Windows).
Configure the agent to send data through Logstash (Logstash Output).
Enable Collect agent metrics in the Fleet policy.
Expected Behavior
Agent metrics documents should be indexed once without conflicts and errors.
Actual Behavior
Documents in
.metrics-*data streams seem to get indexed multiple times, resulting in 409version_conflict_engine_exceptionerrors.The conflicts occur consistently while agent metrics collection is enabled.
Additional Findings
Disabling Collect agent metrics immediately stops:
.metrics-*traffic409
version_conflict_engine_exceptionerrorsThe issue is reproducible with minimal Fleet policies and no additional integrations enabled.
Agent diagnostics confirm that logs and metrics share the same Logstash output configuration.
The behavior occurs with both:
Other logs and integration data do not appear to exhibit the same duplication behavior.
The issue appears isolated to Elastic Agent metrics data streams.
Questions
Is this a known issue affecting Elastic Agent metrics when routed through Logstash?
Is this expected behaviour for metrics?