[Flaky Test]: `TestBeatsServerless/TestRunAndCheckData ` – elastic-agent logged an unexpected error: Error fetching data for metricset system.process

[ycombinator]

Failing test case

TestBeatsServerless/TestRunAndCheckData

Error message

elastic-agent logged an unexpected error: Error fetching data for metricset system.process: non fatal error; reporting partial metrics: error fetching PID metrics for 1 processes, most likely a "permission denied" error. Enable debug logging to determine the exact cause.

Build

https://buildkite.com/elastic/elastic-agent-extended-testing/builds/5776#0193f478-fb7b-4f04-b543-140fe8b4a56b

OS

Linux

Stacktrace and notes

>>> (linux-amd64-ubuntu-2404-default) Test output (sudo) (stdout): === FAIL: testing/integration TestBeatsServerless/TestRunAndCheckData (80.50s)
>>> (linux-amd64-ubuntu-2404-default) Test output (sudo) (stdout): beats_serverless_test.go:197:
>>> (linux-amd64-ubuntu-2404-default) Test output (sudo) (stdout): Error Trace:	/home/ubuntu/agent/testing/integration/beats_serverless_test.go:197
>>> (linux-amd64-ubuntu-2404-default) Test output (sudo) (stdout): Error:      	Received unexpected error:
>>> (linux-amd64-ubuntu-2404-default) Test output (sudo) (stdout): elastic-agent logged an unexpected error: Error fetching data for metricset system.process: non fatal error; reporting partial metrics: error fetching PID metrics for 1 processes, most likely a "permission denied" error. Enable debug logging to determine the exact cause.
>>> (linux-amd64-ubuntu-2404-default) Test output (sudo) (stdout): Test:       	TestBeatsServerless/TestRunAndCheckData

[elasticmachine]

Pinging @elastic/elastic-agent (Team:Elastic-Agent)

[ycombinator]

Another failure: https://buildkite.com/elastic/elastic-agent-extended-testing/builds/5759#0193f0d3-125d-4eec-b350-eb7236489ad2

[ycombinator]

Another failure: https://buildkite.com/elastic/elastic-agent-extended-testing/builds/5729#0193e160-c93c-412b-b4c5-5e3fd50db6fc

[ycombinator]

This is the error message that's seen in the failing test:

Error fetching data for metricset system.process: non fatal error; reporting partial metrics: error fetching PID metrics for 1 processes, most likely a "permission denied" error. Enable debug logging to determine the exact cause.

It looks like this message was introduced on December 12, 2024 ago via elastic/elastic-agent-system-metrics#195. A new release of elastic-agent-system-metrics, 0.11.5, containing the above change (and other changes as well), was made on the same day. This new release was pulled into Beats on the same day as well.

@VihasMakwana you are probably most familiar with this area of code so I'm assigning this issue to you for investigation. Thanks.

[ycombinator]

Another failure: https://buildkite.com/elastic/elastic-agent-extended-testing/builds/5643#0193dcda-eafb-402e-9506-a5c073ea60b7

[ycombinator]

@VihasMakwana I think we can close this issue now that you have resolved the underlying cause of the test flakiness?

[VihasMakwana]

Closed by #6438

[archon810]

What version is the fix available in? We're currently seeing this error in our production logs on version 8.17.1.