[plugins] Remove support for go plugins

[leehinman]

Proposed commit message

Remove support for go plugins. No plugins were ever shipped and this removes the plugin registry which was a package level global variable. The package level global variable needs to be removed for beatreceivers where multiple instance of beat receivers will exist in one process.

Checklist

• My code follows the style guidelines of this project
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• I have made corresponding change to the default configuration files
• I have added tests that prove my fix is effective or that my feature works
• I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

Disruptive User Impact

Should be none since we didn't ship any plugins. However, if someone compiled their own plugins this would be a breaking change.

Author's Checklist

• [ ]

How to test this PR locally

Run integration tests

Related issues

• Releates [beatreceiver] global paths definition #44903

Use cases

Screenshots

Logs

[github-actions]

🤖 GitHub comments

Expand to view the GitHub comments

Just comment with:

• run docs-build : Re-trigger the docs validation. (use unformatted text in the comment!)

[mergify]

This pull request does not have a backport label.
If this is a bug or security fix, could you label this PR @leehinman? 🙏.
For such, you'll need to label your PR with:

• The upcoming major version of the Elastic Stack
• The upcoming minor version of the Elastic Stack (if you're not pushing a breaking change)

To fixup this pull request, you need to add the backport labels for the needed
branches, such as:

• backport-8./d is the label to automatically backport to the 8./d branch. /d is the digit
• backport-active-all is the label that automatically backports to all active branches.
• backport-active-8 is the label that automatically backports to all active minor branches for the 8 major.
• backport-active-9 is the label that automatically backports to all active minor branches for the 9 major.

[elasticmachine]

Pinging @elastic/elastic-agent-data-plane (Team:Elastic-Agent-Data-Plane)

[leehinman]

This will have to be switched back to official elastic go-quark repo. The only change between this and official v0.3.0 is a backport of elastic/go-quark@12841a4 which allows linking to the correct GLIBC version depending on OS arch.

[mauri870]

I wonder if we can just require the particular commit instead of a fork here, since we plan to go back to the elastic package anyway.

[cmacknz]

My approval is pending the removal of this replace statement :)

[mauri870]

LGTM, I did a quick search and could not find any more references to plugins in the codebase.

[rdner]

I thought the whole idea behind plugins was to let users to extend Beats functionality.

How can we be so sure that by deleting the plugin support we would not disrupt our users?

Some of the plugin support was added by external contributors (autodiscover). I personally don't have enough confidence to completely delete the plugin support.

[rdner]

The autodiscover plugin support was an external contribution #6457

This makes me believe someone somewhere might be using it. How can we be sure?

[mauri870]

Perhaps we can send an email to the contributors mailing list?

[rdner]

If so, I think it should not be limited to just autodiscover plugins.

[cmacknz]

Plugins originally came from #3217 and depend on https://pkg.go.dev/plugin#hdr-Warnings.

There are no tests or documentation, the chances of anyone successfully using this is very low but I can find references to people trying to use them in discuss:

https://discuss.elastic.co/search?q=plugin%20%23elastic-stack%3Abeats

• https://discuss.elastic.co/t/problems-with-custom-beat-plugin/231389
• https://discuss.elastic.co/t/is-it-possible-to-develop-a-custom-plugin-for-filebeat/339952

That's someone trying and failing to write a plugin and another one that claims plugins aren't supported by a non-elastic user. A lot of the time "plugin" is used for someone else like "module" or writing a custom processor or output in a forked Beats repository.

I think I'm pro removal overall, if we were getting a lot of questions about this we probably would have deprecated it already.

[rdner]

As discussed with @mauri870 and @cmacknz it's good to go.

[pierrehilbert]

@nfritts / @bturquet could we have someone to review here please?

[elasticmachine]

Pinging @elastic/obs-ds-hosted-services (Team:obs-ds-hosted-services)

[elasticmachine]

Pinging @elastic/sec-linux-platform (Team:Security-Linux Platform)

[cmacknz]

@leehinman what's the status of this one? Does it need to get tracked in the global paths follow up work issue?

[leehinman]

@leehinman what's the status of this one? Does it need to get tracked in the global paths follow up work issue?

Not strickly required, but it does reduce the amount of work in global paths follow up issue. I added a link there. I should be able to get back this next week.

[leehinman]

go-quark v0.4.0 fixes the building on arm linux, but changes signature of quark.OpenQueue, which auditbeat uses. I'm closing this for now since it isn't absolutely required for global paths issue. We can revisit if it comes up again.

[mergify]

This pull request is now in conflicts. Could you fix it? 🙏
To fixup this pull request, you can check out it locally. See documentation: https://help.github.com/articles/checking-out-pull-requests-locally/

git fetch upstream
git checkout -b 44903c_remove_libbeat_plugin upstream/44903c_remove_libbeat_plugin
git merge upstream/main
git push upstream 44903c_remove_libbeat_plugin

[orestisfl]

Given this PR is old, in conflict and uses the old release-note format, I've opened a new one here: #51320