Remove experimental loadable Go plugin support by orestisfl · Pull Request #51320 · elastic/beats

orestisfl · 2026-06-17T11:31:54Z

Proposed commit message

Remove experimental loadable Go plugin support

The loadable Go plugin support was experimental: it was opt-in via the
`--plugin` flag, which loaded Go plugin bundles at startup and logged an
"experimental" warning when used. It was never promoted to a stable feature and
no plugins were ever shipped.

This removes the libbeat/plugin package, the `--plugin` flag, and the related
plugin registration helpers (Plugin, RegisterLoader, MustRegisterLoader) across
libbeat, filebeat, heartbeat, metricbeat and packetbeat, along with the global
plugin registry, which was a package-level global variable.

Co-authored-by: Lee E. Hinman <lee.e.hinman@elastic.co>

Checklist

My code follows the style guidelines of this project
I have commented my code, particularly in hard-to-understand areas
I have made corresponding changes to the documentation (couldn't find anything)
~~I have made corresponding change to the default configuration files~~
I have added tests that prove my fix is effective or that my feature works. Where relevant, I have used the stresstest.sh script to run them under stress conditions and race detector to verify their stability.
I have added an entry in ./changelog/fragments using the changelog tool.

Disruptive User Impact

The plugin support was experimental and no plugins were ever shipped, so no stable functionality is affected. The --plugin flag is no longer recognized and anyone who compiled their own Go plugins against these experimental APIs will no longer be able to load them.

How to test this PR locally

Confirm the --plugin flag is gone, e.g. filebeat --help no longer lists it.

Related issues

Closes [plugins] Remove support for go plugins #45840 (old PR)

The loadable Go plugin support was experimental: it was opt-in via the `--plugin` flag, which loaded Go plugin bundles at startup and logged an "experimental" warning when used. It was never promoted to a stable feature and no plugins were ever shipped. This removes the libbeat/plugin package, the `--plugin` flag, and the related plugin registration helpers (Plugin, RegisterLoader, MustRegisterLoader) across libbeat, filebeat, heartbeat, metricbeat and packetbeat, along with the global plugin registry, which was a package-level global variable. Relates: elastic#44903 Co-authored-by: Lee E. Hinman <lee.e.hinman@elastic.co>

github-actions · 2026-06-17T11:32:04Z

🤖 GitHub comments

Just comment with:

run docs-build : Re-trigger the docs validation. (use unformatted text in the comment!)
/test : Run the Buildkite pipeline.

coderabbitai · 2026-06-17T11:33:02Z

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Enterprise

Run ID: 392dd155-6098-4eb4-9dd1-5f27154652af

📥 Commits

Reviewing files that changed from the base of the PR and between 4b15a31 and 6d4b80c.

📒 Files selected for processing (23)

changelog/fragments/1781686561-remove-libbeat-plugin.yaml
filebeat/include/list.go
filebeat/input/plugin.go
heartbeat/include/list.go
heartbeat/monitors/plugin/plugin.go
libbeat/autodiscover/appenders/registry.go
libbeat/autodiscover/builder/plugin.go
libbeat/autodiscover/providers/plugin.go
libbeat/cmd/instance/beat.go
libbeat/cmd/root.go
libbeat/outputs/codec/plugin.go
libbeat/outputs/plugin.go
libbeat/plugin/cli.go
libbeat/plugin/cli_stub.go
libbeat/plugin/load.go
libbeat/plugin/load_stub.go
libbeat/plugin/plugin.go
libbeat/processors/add_kubernetes_metadata/registry.go
libbeat/processors/registry.go
metricbeat/module/plugin.go
packetbeat/protocols/plugin.go
x-pack/libbeat/cmd/instance/beat.go
x-pack/libbeat/processors/add_nomad_metadata/registry.go

💤 Files with no reviewable changes (21)

libbeat/plugin/cli.go
libbeat/outputs/plugin.go
libbeat/autodiscover/providers/plugin.go
libbeat/outputs/codec/plugin.go
libbeat/autodiscover/builder/plugin.go
libbeat/plugin/load.go
x-pack/libbeat/processors/add_nomad_metadata/registry.go
packetbeat/protocols/plugin.go
filebeat/input/plugin.go
libbeat/autodiscover/appenders/registry.go
libbeat/plugin/load_stub.go
libbeat/processors/add_kubernetes_metadata/registry.go
libbeat/plugin/plugin.go
heartbeat/include/list.go
filebeat/include/list.go
metricbeat/module/plugin.go
x-pack/libbeat/cmd/instance/beat.go
libbeat/plugin/cli_stub.go
libbeat/cmd/instance/beat.go
libbeat/processors/registry.go
libbeat/cmd/root.go

📝 Walkthrough

Walkthrough

The PR removes the experimental Go plugin system from libbeat and all dependent beats. The entire libbeat/plugin package is deleted, along with every per-subsystem Plugin() helper function and init() loader registration in filebeat, heartbeat, metricbeat, packetbeat, outputs, autodiscover, and processor packages. The --plugin CLI flag and the plugin.Initialize() call in (*Beat).InitWithSettings (both OSS and x-pack variants) are removed. Generated include/list.go files drop the now-unnecessary blank imports, and a breaking-change changelog fragment is added.

🚥 Pre-merge checks | ✅ 2

✅ Passed checks (2 passed)

Check name	Status	Explanation
Linked Issues check	✅ Passed	PR comprehensively removes Go plugin support across all beat components, eliminating the global plugin registry and --plugin flag as required by `#45840`.
Out of Scope Changes check	✅ Passed	All changes are directly aligned with plugin removal objectives; no out-of-scope modifications detected.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

Create PR with unit tests

🛠️ Update Documentation

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

infra-vault-gh-plugin-prod · 2026-06-17T11:37:57Z

Pinging @elastic/obs-ds-hosted-services (Team:obs-ds-hosted-services)

infra-vault-gh-plugin-prod · 2026-06-17T11:37:59Z

Pinging @elastic/elastic-agent-data-plane (Team:Elastic-Agent-Data-Plane)

infra-vault-gh-plugin-prod · 2026-06-17T11:38:01Z

Pinging @elastic/sec-linux-platform (Team:Security-Linux Platform)

blakerouse

Nice!

cmacknz · 2026-06-17T14:58:13Z

Building x-pack/filebeat on MacOS, it looks like just this PR shrinks it from 178M to 158M when compared to main.

mauri870 · 2026-06-17T15:43:41Z

Building x-pack/filebeat on MacOS, it looks like just this PR shrinks it from 178M to 158M when compared to main.

On linux it goes from 246M to 195M for me.

rdner

The change looks good to me.

The packaging step on ARM is failing, I tried to restart but looks like the failure is legit.

Also the linter issues need to be resolved.

orestisfl · 2026-06-18T15:41:42Z

The change looks good to me.

The packaging step on ARM is failing, I tried to restart but looks like the failure is legit.

Yes this is blocked until quark backports elastic/go-quark@12841a4 in 0.3.0 that beats is using or we upgrade to

CC @haesbaert

haesbaert · 2026-06-18T16:12:25Z

The change looks good to me.
The packaging step on ARM is failing, I tried to restart but looks like the failure is legit.

Yes this is blocked until quark backports elastic/go-quark@12841a4 in 0.3.0 that beats is using or we upgrade to

CC @haesbaert

Heya, I think the linker likely changed, since that symbol never really existed in arm64, that's pretty much the issue the wolfi people stumbled and hence we did the fix.

I'll update quark, the API changed a bit so more things are likely needed, but it's time to do it anyway.

leehinman

Thanks for picking this up. LGTM.

orestisfl · 2026-06-25T06:54:30Z

Putting in draft mode as it's blocked on #51320 (comment)

github-actions · 2026-07-10T10:12:39Z

TL;DR

The Filebeat: Log input as Filestream Python Integration Tests job did not reach the tests. Docker image construction failed twice while downloading Kafka 3.6.0 from Apache with curl exit code 28 (timeout), so this is an infrastructure/dependency availability failure rather than a regression from the plugin-removal changes.

Remediation

Re-run the failed Buildkite job; the existing Kafka image step already retries the download (retry --min 1 --max 180).
If the timeout persists, investigate access/availability of the Apache archive endpoint or use a healthy mirror/cache for the Kafka archive, then rerun mage PythonIntegTestLogAsFilestream.

Investigation details

Root Cause

The failure occurs during docker compose ... build, before any Python integration test starts. The Kafka image build downloads kafka_2.13-3.6.0.tgz from archive.apache.org; the command exits with status 28 on both build attempts, and Mage then reports exit status 17. The PR changes remove Go plugin support and do not modify testing/environments/docker/kafka/Dockerfile or the Kafka test setup.

Evidence

Build: https://buildkite.com/elastic/beats/builds/49180
Job/step: :ubuntu: Filebeat: Log input as Filestream Python Integration Tests (https://buildkite.com/elastic/filebeat/builds/34333)
Key log excerpt: failed to solve: process ... curl ... https://archive.apache.org/dist/kafka/${KAFKA_VERSION}/kafka_2.13-${KAFKA_VERSION}.tgz ... did not complete successfully: exit code: 28 (lines 127 and 131); docker compose ... build failed with exit code 17 (line 133).
Source: testing/environments/docker/kafka/Dockerfile:15-19 downloads Kafka 3.6.0 and wraps the download with the retry helper.

Verification

Not run locally: the failure is in the Buildkite Docker image build, and reproducing the full integration environment is unavailable here.

What is this? | From workflow: PR Buildkite Detective

Give us feedback! React with 🚀 if perfect, 👍 if helpful, 👎 if not.

mergify · 2026-07-15T07:50:23Z

This pull request is now in conflicts. Could you fix it? 🙏
To fixup this pull request, you can check out it locally. See documentation: https://help.github.com/articles/checking-out-pull-requests-locally/

git fetch upstream
git checkout -b remove-go-plugins upstream/remove-go-plugins
git merge upstream/main
git push upstream remove-go-plugins

# Conflicts: # filebeat/input/plugin.go # libbeat/autodiscover/appenders/registry.go # libbeat/autodiscover/builder/plugin.go # libbeat/autodiscover/providers/plugin.go # libbeat/outputs/codec/plugin.go # libbeat/outputs/plugin.go # libbeat/plugin/load.go # libbeat/plugin/plugin.go # libbeat/processors/add_kubernetes_metadata/registry.go # libbeat/processors/registry.go # metricbeat/module/plugin.go # x-pack/libbeat/processors/add_nomad_metadata/registry.go

orestisfl added Team:obs-ds-hosted-services Label for the Observability Hosted Services team breaking change backport-skip Skip notification from the automated backport with mergify Team:Elastic-Agent-Data-Plane Label for the Agent Data Plane team labels Jun 17, 2026

orestisfl self-assigned this Jun 17, 2026

botelastic Bot added needs_team Indicates that the issue/PR needs a Team:* label and removed needs_team Indicates that the issue/PR needs a Team:* label labels Jun 17, 2026

orestisfl mentioned this pull request Jun 17, 2026

[plugins] Remove support for go plugins #45840

Closed

6 tasks

orestisfl requested review from blakerouse, cmacknz, leehinman and rdner June 17, 2026 11:33

orestisfl added the Team:Security-Linux Platform Linux Platform Team in Security Solution label Jun 17, 2026

orestisfl marked this pull request as ready for review June 17, 2026 11:37

orestisfl requested review from a team as code owners June 17, 2026 11:37

This comment has been minimized.

Sign in to view

blakerouse approved these changes Jun 17, 2026

View reviewed changes

rdner reviewed Jun 18, 2026

View reviewed changes

leehinman approved these changes Jun 18, 2026

View reviewed changes

orestisfl marked this pull request as draft June 25, 2026 06:54

haesbaert mentioned this pull request Jul 7, 2026

auditbeat: Update to quark 0.6.0 #51767

Merged

6 tasks

orestisfl added 2 commits July 10, 2026 09:54

Merge remote-tracking branch 'upstream/main' into remove-go-plugins

446e320

Merge branch 'main' into remove-go-plugins

e537632

orestisfl marked this pull request as ready for review July 10, 2026 11:38

Merge branch 'main' into remove-go-plugins

a6069f1

orestisfl requested a review from haesbaert July 10, 2026 11:38

Uh oh!

Conversation

orestisfl commented Jun 17, 2026

Proposed commit message

Checklist

Disruptive User Impact

How to test this PR locally

Related issues

Uh oh!

github-actions Bot commented Jun 17, 2026

🤖 GitHub comments

Uh oh!

coderabbitai Bot commented Jun 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Walkthrough

Uh oh!

infra-vault-gh-plugin-prod Bot commented Jun 17, 2026

Uh oh!

infra-vault-gh-plugin-prod Bot commented Jun 17, 2026

Uh oh!

infra-vault-gh-plugin-prod Bot commented Jun 17, 2026

Uh oh!

This comment has been minimized.

blakerouse left a comment

Choose a reason for hiding this comment

Uh oh!

cmacknz commented Jun 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mauri870 commented Jun 17, 2026

Uh oh!

rdner left a comment

Choose a reason for hiding this comment

Uh oh!

orestisfl commented Jun 18, 2026

Uh oh!

haesbaert commented Jun 18, 2026

Uh oh!

leehinman left a comment

Choose a reason for hiding this comment

Uh oh!

orestisfl commented Jun 25, 2026

Uh oh!

github-actions Bot commented Jul 10, 2026

TL;DR

Remediation

Root Cause

Evidence

Verification

Uh oh!

mergify Bot commented Jul 15, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

coderabbitai Bot commented Jun 17, 2026 •

edited

Loading

cmacknz commented Jun 17, 2026 •

edited

Loading