[beats receivers] Logs contain security exceptions for `metrics-elastic_agent.collector-default`

[cmacknz]

Using a Fleet managed 9.2.0-SNAPSHOT agent the logs contain security exceptions attempting to write to the metrics-elastic_agent.collector-default index.

{"log.level":"error","@timestamp":"2025-10-02T21:11:27.345Z","message":"failed to index document","log.origin.stack_trace":"github.com/open-telemetry/opentelemetry-collector-contrib/exporter/elasticsearchexporter.flushBulkIndexer\n\tgithub.com/open-telemetry/opentelemetry-collector-contrib/exporter/elasticsearchexporter@v0.135.0/bulkindexer.go:540\ngithub.com/open-telemetry/opentelemetry-collector-contrib/exporter/elasticsearchexporter.(*syncBulkIndexerSession).Flush\n\tgithub.com/open-telemetry/opentelemetry-collector-contrib/exporter/elasticsearchexporter@v0.135.0/bulkindexer.go:205\ngithub.com/open-telemetry/opentelemetry-collector-contrib/exporter/elasticsearchexporter.(*sessionList).Flush.func1\n\tgithub.com/open-telemetry/opentelemetry-collector-contrib/exporter/elasticsearchexporter@v0.135.0/exporter.go:613\ngolang.org/x/sync/errgroup.(*Group).Go.func1\n\tgolang.org/x/sync@v0.17.0/errgroup/errgroup.go:93","log.origin":{"file.line":540,"file.name":"elasticsearchexporter@v0.135.0/bulkindexer.go","function":"github.com/open-telemetry/opentelemetry-collector-contrib/exporter/elasticsearchexporter.flushBulkIndexer"},"resource":{"service.instance.id":"afd06acf-37d5-44cd-934d-e452c3c1736a","service.name":"/opt/Elastic/Agent/data/elastic-agent-9.2.0-SNAPSHOT-e917e0/elastic-agent","service.version":"9.2.0"},"otelcol.component.kind":"exporter","otelcol.signal":"logs","error.type":"security_exception","error.reason":"","ecs.version":"1.6.0","otelcol.component.id":"elasticsearch/_agent-component/monitoring","index":"metrics-elastic_agent.collector-default","ecs.version":"1.6.0"}
{"log.level":"error","@timestamp":"2025-10-02T21:12:27.424Z","message":"failed to index document","error.type":"security_exception","ecs.version":"1.6.0","log.origin.stack_trace":"github.com/open-telemetry/opentelemetry-collector-contrib/exporter/elasticsearchexporter.flushBulkIndexer\n\tgithub.com/open-telemetry/opentelemetry-collector-contrib/exporter/elasticsearchexporter@v0.135.0/bulkindexer.go:540\ngithub.com/open-telemetry/opentelemetry-collector-contrib/exporter/elasticsearchexporter.(*syncBulkIndexerSession).Flush\n\tgithub.com/open-telemetry/opentelemetry-collector-contrib/exporter/elasticsearchexporter@v0.135.0/bulkindexer.go:205\ngithub.com/open-telemetry/opentelemetry-collector-contrib/exporter/elasticsearchexporter.(*sessionList).Flush.func1\n\tgithub.com/open-telemetry/opentelemetry-collector-contrib/exporter/elasticsearchexporter@v0.135.0/exporter.go:613\ngolang.org/x/sync/errgroup.(*Group).Go.func1\n\tgolang.org/x/sync@v0.17.0/errgroup/errgroup.go:93","log.origin":{"file.line":540,"file.name":"elasticsearchexporter@v0.135.0/bulkindexer.go","function":"github.com/open-telemetry/opentelemetry-collector-contrib/exporter/elasticsearchexporter.flushBulkIndexer"},"otelcol.component.id":"elasticsearch/_agent-component/monitoring","otelcol.component.kind":"exporter","index":"metrics-elastic_agent.collector-default","error.reason":"","resource":{"service.instance.id":"afd06acf-37d5-44cd-934d-e452c3c1736a","service.name":"/opt/Elastic/Agent/data/elastic-agent-9.2.0-SNAPSHOT-e917e0/elastic-agent","service.version":"9.2.0"},"otelcol.signal":"logs","ecs.version":"1.6.0"}

The metrics-elastic_agent.collector-default data stream there is not writable without adding it to the list of known monitoring data streams in Fleet in https://github.com/elastic/kibana/blob/eed21dbca87029b245c44f5d8d4d64c9f491c877/x-pack/platform/plugins/shared/fleet/common/constants/agent_policy.ts#L17-L35

It may also need he data stream to be defined in
https://github.com/elastic/integrations/tree/main/packages/elastic_agent

Otherwise we'll see index security exceptions in the logs for every collection period for every Fleet managed agent.

[elasticmachine]

Pinging @elastic/elastic-agent-control-plane (Team:Elastic-Agent-Control-Plane)

[cmacknz]

It's coming from:

    metricbeatreceiver/_agent-component/prometheus/metrics-monitoring:
        http:
            enabled: true
            host: unix:///opt/Elastic/Agent/data/tmp/6-Q5121mA789biwQZlm1piZtOuNIgYI0.sock
        logging:
            with_fields:
                component:
                    binary: metricbeat
                    dataset: elastic_agent.metricbeat
                    id: prometheus/metrics-monitoring
                    type: prometheus/metrics
                log:
                    source: prometheus/metrics-monitoring
        management.otel.enabled: true
        metricbeat:
            modules:
                - failure_threshold: 5
                  hosts:
                    - localhost:8888
                  id: metrics-monitoring-collector
                  index: metrics-elastic_agent.collector-default
                  metrics_path: /metrics
                  metricsets:
                    - collector
                  module: prometheus
                  namespace: default
                  period: 1m0s

[cmacknz]

If we can't get this fixed before 9.2 we'll have to disable this input so we aren't writing error messages into the logs every minute that users can't turn off or fix.

[faec]

(Picking this up because it's caused by my change.) It wasn't supposed to be ingesting into that index in the first place, I think this should be a relatively straightforward fix.