otel: support system/metrics input with experimental otel runtime

[mauri870]

What does this PR do?

This PR enables the system/metrics input to work with the otel runtime. It needs to be toggled explicitly by setting _runtime_experimental: otel in the input configuration.

Checklist

• I have read and understood the pull request guidelines of this project.
• My code follows the style guidelines of this project
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• I have made corresponding change to the default configuration files
• I have added tests that prove my fix is effective or that my feature works
• I have added an entry in ./changelog/fragments using the changelog tool
• I have added an integration test or an E2E test

Related issues

• Closes https://github.com/elastic/ingest-dev/issues/4886
• Closes https://github.com/elastic/ingest-dev/issues/4882
• Closes https://github.com/elastic/ingest-dev/issues/4884
• Closes https://github.com/elastic/ingest-dev/issues/4885

[mergify]

This pull request does not have a backport label. Could you fix it @mauri870? 🙏
To fixup this pull request, you need to add the backport labels for the needed
branches, such as:

• backport-./d./d is the label that automatically backports to the 8./d branch. /d is the digit
• backport-active-all is the label that automatically backports to all active branches.
• backport-active-8 is the label that automatically backports to all active minor branches for the 8 major.
• backport-active-9 is the label that automatically backports to all active minor branches for the 9 major.

[elasticmachine]

Pinging @elastic/elastic-agent-control-plane (Team:Elastic-Agent-Control-Plane)

[elasticmachine]

Pinging @elastic/elastic-agent-data-plane (Team:Elastic-Agent-Data-Plane)

[pkoutsovasilis]

TestAgentMetricsInput had a flaky run here @mauri870 , so please have a look. The only this that comes to mind is that when this was retried it provisioned an isolated ESS stack so please do make sure that there is no interference when TestAgentMetricsInput is running in parallel from multiple runners

[swiatekm]

TestAgentMetricsInput had a flaky run here @mauri870 , so please have a look. The only this that comes to mind is that when this was retried it provisioned an isolated ESS stack so please do make sure that there is no interference when TestAgentMetricsInput is running in parallel from multiple runners

That looks like the system.network.in.errors metric doesn't always appear in the output.

@mauri870 I think what we should actually check is that the documents have the same fields, but not that the values are the same. Just ignoring all the fields containing metrics makes this test not very useful.

[elastic-sonarqube]

Quality Gate passed

Issues
0 New issues
0 Fixed issues
0 Accepted issues

Measures
0 Security Hotspots
No data about Coverage
0.0% Duplication on New Code

See analysis details on SonarQube

[elasticmachine]

⏳ Build in-progress, with failures

• Buildkite Build
• Commit: 2bea714

Failed CI Steps

• Win2022:non-sudo:default
• Win2022:non-sudo:default
• Win2025:non-sudo:default
• Win2025:non-sudo:default

History

• 💔 Build #21408 failed ea8a07c
• 💔 Build #21405 failed 1ee22f1
• 💛 Build #21337 was flaky 49c5f9d
• 💔 Build #21329 failed b7612b0
• 💛 Build #21325 was flaky 98b18a7

cc @mauri870

[mauri870]

Found during testing that event.duration is sometimes missing from the ingested document with beats receivers on Windows. Pushed a fix in elastic/beats#44442.