[ipc/listener][windows] Implement retry mechanism for listener creation

[VihasMakwana]

What does this PR do?

Adds a retry mechanism when creating a named pipe listener on Windows.
Instead of failing immediately if listener creation fails, it now retries for up to 5 seconds with a short delay between attempts.

Why is it important?

Sometimes the named pipe might not be immediately available. This was observed when using monitoring with beat receivers.

Checklist

• I have read and understood the pull request guidelines of this project.
• My code follows the style guidelines of this project
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• I have made corresponding change to the default configuration files
• I have added tests that prove my fix is effective or that my feature works
• I have added an entry in ./changelog/fragments using the changelog tool
• I have added an integration test or an E2E test

[elasticmachine]

Pinging @elastic/elastic-agent-control-plane (Team:Elastic-Agent-Control-Plane)

[mergify]

This pull request does not have a backport label. Could you fix it @VihasMakwana? 🙏
To fixup this pull request, you need to add the backport labels for the needed
branches, such as:

• backport-./d./d is the label that automatically backports to the 8./d branch. /d is the digit
• backport-active-all is the label that automatically backports to all active branches.
• backport-active-8 is the label that automatically backports to all active minor branches for the 8 major.
• backport-active-9 is the label that automatically backports to all active minor branches for the 9 major.

[leehinman]

I don't think this will fix https://github.com/elastic/ingest-dev/issues/6133

That error is:

filebeatreceiver:could not start the HTTP server for the API: failed to listen on the named pipe \\.\pipe\xTEtpJ7117ppc6OYvJCaYHbDW8mLjXGe.sock: open \\.\pipe\xTEtpJ7117ppc6OYvJCaYHbDW8mLjXGe.sock: Access is denied.

That error is coming from filebeatreceiver, not elastic-agent. I think we need to do the retry in the libbeat api server:

https://github.com/elastic/beats/blob/e55e0bedd6176c7c6c88dfdf012c14620cb85a29/libbeat/api/server.go#L73-L79

[VihasMakwana]

I don't think this will fix elastic/ingest-dev#6133

That error is:

filebeatreceiver:could not start the HTTP server for the API: failed to listen on the named pipe \\.\pipe\xTEtpJ7117ppc6OYvJCaYHbDW8mLjXGe.sock: open \\.\pipe\xTEtpJ7117ppc6OYvJCaYHbDW8mLjXGe.sock: Access is denied.

That error is coming from filebeatreceiver, not elastic-agent. I think we need to do the retry in the libbeat api server:

https://github.com/elastic/beats/blob/e55e0bedd6176c7c6c88dfdf012c14620cb85a29/libbeat/api/server.go#L73-L79

Oh!
I faced the same problem with elastic-agent and beatreceivers and this is what fixed for me.
Anyways, I'll apply the same fix in libbeat.

[VihasMakwana]

I don't think this will fix elastic/ingest-dev#6133

That error is:

filebeatreceiver:could not start the HTTP server for the API: failed to listen on the named pipe \\.\pipe\xTEtpJ7117ppc6OYvJCaYHbDW8mLjXGe.sock: open \\.\pipe\xTEtpJ7117ppc6OYvJCaYHbDW8mLjXGe.sock: Access is denied.

That error is coming from filebeatreceiver, not elastic-agent. I think we need to do the retry in the libbeat api server:

https://github.com/elastic/beats/blob/e55e0bedd6176c7c6c88dfdf012c14620cb85a29/libbeat/api/server.go#L73-L79

Thank you for letting me know, Lee!

[elastic-sonarqube]

Quality Gate passed

Issues
0 New issues
0 Fixed issues
0 Accepted issues

Measures
0 Security Hotspots
90.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube

[elasticmachine]

💚 Build Succeeded

• Buildkite Build
• Commit: 9642553

History

• 💔 Build #27731 failed 0ed1c06

cc @VihasMakwana

[blakerouse]

To make this change this is really going to have to take a context. Which I know is not great because changing the function signature means it needs to be updated everywhere. But if this is going to take up to 5 seconds, context cancelling will need to be handled. The user might Ctrl-C the elastic-agent it should not be stuck in this section and wait up to 5 seconds before it returns.

[VihasMakwana]

@blakerouse what if we create a signals.Notify(..) here and exit on sigint or sigterm, instead of passing context all the way from command?

[cmacknz]

https://github.com/search?q=repo%3Aelastic%2Felastic-agent%20CreateListener&type=code

It looks to me like there are only 3 calls to this, two of which have a context already. The one that doesn't could just be given context.TODO() without making anything worse. Listening to signals in more than one place would be way messier IMO and possibly complicate the problems we are already having with the Windows service lifetime being managed properly.

elastic-agent/pkg/component/runtime/conn_info_server.go

Lines 31 to 38 in 278440a

	func newConnInfoServer(log *logger.Logger, comm Communicator, address string) (*connInfoServer, error) {
	var (
	listener net.Listener
	err error
	)
	if ipc.IsLocal(address) {
	listener, err = ipc.CreateListener(log, address)