[otel]: Add e2e test for monitoring metrics in otel mode

[khushijain21]

What does this PR do?

This PR adds e2e tests for self-monitoring metrics exposed using beatreceivers. It also asserts document equivalency for metrics exposed by normal mode vs otel mode.

Why is it important?

Required to safely transition running elastic-agent in otel mode.

Checklist

• I have read and understood the pull request guidelines of this project.
• My code follows the style guidelines of this project
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• I have made corresponding change to the default configuration files
• I have added tests that prove my fix is effective or that my feature works
• I have added an entry in ./changelog/fragments using the changelog tool
• I have added an integration test or an E2E test

Related issues

• Closes https://github.com/elastic/ingest-dev/issues/4877
• Closes https://github.com/elastic/ingest-dev/issues/4879

[mergify]

This pull request does not have a backport label. Could you fix it @khushijain21? 🙏
To fixup this pull request, you need to add the backport labels for the needed
branches, such as:

• backport-./d./d is the label that automatically backports to the 8./d branch. /d is the digit
• backport-active-all is the label that automatically backports to all active branches.
• backport-active-8 is the label that automatically backports to all active minor branches for the 8 major.
• backport-active-9 is the label that automatically backports to all active minor branches for the 9 major.

[cmacknz]

Does this overlap with what #7622 intends to do?

[khushijain21]

Does this overlap with what #7622 intends to do?

#7622 adds test for metric inputs whereas this is for monitoring metrics collected by elastic-agent.

[mergify]

This pull request is now in conflicts. Could you fix it? 🙏
To fixup this pull request, you can check out it locally. See documentation: https://help.github.com/articles/checking-out-pull-requests-locally/

git fetch upstream
git checkout -b e2emetrics upstream/e2emetrics
git merge upstream/main
git push upstream e2emetrics

[cmacknz]

Thanks for clarifying!

[leehinman]

This looks like it is for agent monitoring. We really want this test to be for something that isn't agent monitoring. So a different Metricbeat module that would be a different config.

[khushijain21]

This looks like it is for agent monitoring. We really want this test to be for something that isn't agent monitoring. So a different Metricbeat module that would be a different config.

I am not sure I understand. This test is intended for agent monitoring so

[khushijain21]

This changeset seems to drop libbeat.output/* and libbeat.pipeline.* Currently in investigation before this PR can be merged

[leehinman]

one small change, otherwise lets merge and I'll have one more round of adding to this test for https://github.com/elastic/ingest-dev/issues/4876

[khushijain21]

/test

[elastic-sonarqube]

Quality Gate passed

Issues
0 New issues
0 Fixed issues
0 Accepted issues

Measures
0 Security Hotspots
No data about Coverage
0.0% Duplication on New Code

See analysis details on SonarQube

[elasticmachine]

💚 Build Succeeded

• Buildkite Build
• Commit: 7eb543c

History

• 💔 Build #21607 failed 2cec25c
• 💔 Build #21605 failed 1fbed5e
• 💔 Build #21548 failed ac0b0e8
• 💔 Build #21539 failed 6edea59
• 💔 Build #21524 failed a3eada8

cc @khushijain21