Load Npcap wpcap.dll lazily to avoid blocking upgrades on Windows

[samuelvl]

On Windows importing gopacket/pcap used to load wpcap.dll in the package init() (see elastic/gopacket#7).

When elastic-agent runs the collector in process mode, every collector process (not only packetbeat) loads wpcap.dll at startup:

PS C:\Windows\system32> tasklist /m wpcap.dll

Image Name                     PID Modules
========================= ======== ============================================
elastic-otel-collector.exe    1234 wpcap.dll
elastic-otel-collector.exe    5678 wpcap.dll
elastic-otel-collector.exe    9012 wpcap.dll

While any of these processes holds the DLL open, the packetbeat installer cannot replace wpcap.dll to upgrade Npcap.

Load wpcap.dll lazily instead of at package init so only packetbeat loads it and only when needed.

Checklist

• My code follows the style guidelines of this project
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• I have made corresponding change to the default configuration files
• I have added tests that prove my fix is effective or that my feature works.
• I have added an entry in ./changelog/fragments using the changelog tool.

Disruptive User Impact

None. wpcap.dll is now loaded on demand instead of at startup. Packetbeat behaves the same but other beats no longer hold the DLL open.

How to test this PR locally

On Windows with an older Npcap already installed, run Packetbeat with the Network Traffic input plus other Beats. Packetbeat should upgrade Npcap and start capturing instead of failing with exit status 2.

Automated: mage systemTest on Windows runs TestWindowsNpcapInstaller, which now also verifies that a process only importing the capture code does not load wpcap.dll.

Related issues

• Closes Packetbeat fails to upgrade on Windows when a newer Npcap version is available elastic-agent#14517
• Requires Make npcap DLL loading lazy instead of loading at init gopacket#7

[github-actions]

🤖 GitHub comments

Just comment with:

• run docs-build : Re-trigger the docs validation. (use unformatted text in the comment!)
• /test : Run the Buildkite pipeline.

[mergify]

This pull request is now in conflicts. Could you fix it? 🙏
To fixup this pull request, you can check out it locally. See documentation: https://help.github.com/articles/checking-out-pull-requests-locally/

git fetch upstream
git checkout -b fix-packetbeat-pcap-upgrade upstream/fix-packetbeat-pcap-upgrade
git merge upstream/main
git push upstream fix-packetbeat-pcap-upgrade

[github-actions]

TL;DR

The current Buildkite failure is a Metricbeat go vet compile error, not a Packetbeat/Npcap test failure. metricbeat/mb/module/runner_test.go:173 still passes a *logp.Logger to module.NewWrapper, whose current signature requires *beat.Info.

Remediation

• Update metricbeat/mb/module/runner_test.go:173 to pass &beat.Info{Logger: logptest.NewTestingLogger(t, "")} as the third module.NewWrapper argument.
• Validate with cd metricbeat && go test ./mb/module -run TestRunnerStop_ClientClosedAfterPublishGoroutine -count=1, then rerun make -C metricbeat check update/Buildkite.

Investigation details

Root Cause

go vet ./... fails while compiling the metricbeat/mb/module_test package. The failing call is:

metricbeat/mb/module/runner_test.go:173
m, err := module.NewWrapper(config, mb.Registry, logptest.NewTestingLogger(t, ""), beatmonitoring.NewMonitoring(), paths.New(), module.WithMetricSetInfo())

But metricbeat/mb/module/wrapper.go:93 defines NewWrapper as:

func NewWrapper(config *conf.C, r *mb.Register, info *beat.Info, monitoring beatmonitoring.Monitoring, p *paths.Path, options ...Option) (*Wrapper, error)

So the third argument must be a *beat.Info, matching nearby tests such as metricbeat/mb/module/wrapper_test.go:116.

Evidence

• Build: https://buildkite.com/elastic/beats/builds/48832
• Job/step: Metricbeat: Run check/update
• Key log excerpt:

# github.com/elastic/beats/v7/metricbeat/mb/module_test
# [github.com/elastic/beats/v7/metricbeat/mb/module_test]
vet: mb/module/runner_test.go:173:51: cannot use logptest.NewTestingLogger(t, "") (value of type *logp.Logger) as *beat.Info value in argument to module.NewWrapper
Error: failed running go vet, please fix the issues reported: running "go vet ./..." failed with exit code 1

The PR diff shown for this run does not modify metricbeat/mb/module/runner_test.go, so this looks like a base-branch Metricbeat test compile break that this PR’s Buildkite run is surfacing, rather than a failure in the Packetbeat/Npcap changes.

Verification

I reproduced the same compile failure locally with:

cd metricbeat && go test ./mb/module -run TestRunner -count=1

The command failed with the same error:

mb/module/runner_test.go:173:51: cannot use logptest.NewTestingLogger(t, "") (value of type *logp.Logger) as *beat.Info value in argument to module.NewWrapper

Follow-up

The previous detective comment on this PR was for an earlier merge-conflict failure. This run has a different root cause and remediation.

---

What is this? | From workflow: PR Buildkite Detective

Give us feedback! React with 🚀 if perfect, 👍 if helpful, 👎 if not.

[infra-vault-gh-plugin-prod]

Pinging @elastic/sec-linux-platform (Team:Security-Linux Platform)

[infra-vault-gh-plugin-prod]

Pinging @elastic/sec-windows-platform (Team:Security-Windows Platform)

[mergify]

Tick the box to add this pull request to the merge queue (same as @mergifyio queue).

• Queue this pull request

[github-actions]

@Mergifyio backport 9.5 9.4 9.3

[mergify]

backport 9.5 9.4 9.3

✅ Backports have been created

Details

• #51828 [9.5](backport #51716) Load Npcap wpcap.dll lazily to avoid blocking upgrades on Windows has been created for branch 9.5
• #51829 [9.4](backport #51716) Load Npcap wpcap.dll lazily to avoid blocking upgrades on Windows has been created for branch 9.4
• #51830 [9.3](backport #51716) Load Npcap wpcap.dll lazily to avoid blocking upgrades on Windows has been created for branch 9.3