[9.3](backport #51345) Use native Go for Linux FIPS builds by mergify[bot] · Pull Request #51953 · elastic/beats

mergify · 2026-07-15T06:43:34Z

Proposed commit message

Go 1.24+ ships its own FIPS 140-3 certified crypto module (GOFIPS140), so we no longer need the Microsoft Go fork (GOEXPERIMENT=systemcrypto) which required CGO and a linked OpenSSL. This switches auditbeat, filebeat, and metricbeat Linux FIPS builds to use the native module instead.

Changes:

fips-settings.yaml: replace GOEXPERIMENT=systemcrypto + cgo: true with GOFIPS140: v1.0.0
crossbuild.go: stop selecting the MS Go -fips crossbuild Docker image
build.go / settings.go: remove dead CGO field and OR
checkFIPS() / VerifyFIPSBinary(): check GOFIPS140 + DefaultGODEBUG=fips140=on instead of GOEXPERIMENT + OpenSSL ELF symbols
CI: switch FIPS test steps from AWS FIPS image to standard GCP image; unskip (SSH key issue was caused by OS-level FIPS enforcement, which is gone)

Mirrors elastic/elastic-agent@07f98a8.

Checklist

My code follows the style guidelines of this project
~~I have commented my code, particularly in hard-to-understand areas~~
~~I have made corresponding changes to the documentation~~
~~I have made corresponding change to the default configuration files~~
I have added tests that prove my fix is effective or that my feature works.
I have added an entry in ./changelog/fragments using the changelog tool.

Disruptive User Impact

FIPS binaries no longer link against system OpenSSL — no runtime OpenSSL dependency.

How to test this PR locally

cd x-pack/filebeat
FIPS=true mage Build

# check buildinfo
go version -m filebeat | grep -E 'GOFIPS140|DefaultGODEBUG'
# expect: GOFIPS140=v1.0.0-... and DefaultGODEBUG=...fips140=on...

Related issues

Closes FIPS Integration Tests not compatible with current Git SSH keys #50269

This is an automatic backport of pull request #51345 done by [Mergify](https://mergify.com).

* feat: use native Go for Linux FIPS builds Switch Linux FIPS builds from Microsoft Go (GOEXPERIMENT=systemcrypto) to the upstream Go native FIPS module (GOFIPS140=v1.0.0, CMVP cert #5247). Drops CGO requirement and the Microsoft Go cross-build Docker image. Updates package tests to verify GOFIPS140 and DefaultGODEBUG buildinfo instead of GOEXPERIMENT and OpenSSL ELF symbols. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * chore: clean up MS Go FIPS artifacts - Remove GOEXPERIMENT=systemcrypto from CI pipelines and ECH test script - Switch FIPS unit/ECH test steps from AWS FIPS image to standard GCP image; unskip - Update VerifyFIPSBinary to check GOFIPS140+DefaultGODEBUG instead of GOEXPERIMENT+OpenSSL symbols - Strengthen GOFIPS140 assertion to require v1.0.0 version; improve error messages - Remove dead CGO field and OR from FIPSConfig/DefaultBuildArgs Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * chore: golangci-lint fixes Signed-off-by: Dominik Rosiek <dominik.rosiek@elastic.co> * fix: propagate FIPS env to unit tests and pre-populate module cache for crossbuild - Inject FIPS compile env vars (GOFIPS140=v1.0.0) into go test args so that test binaries actually enforce FIPS crypto at runtime, not just compile with the requirefips tag. - Pre-populate golang.org/fips140 in the host module cache before FIPS crossbuild containers mount it read-only, avoiding a write failure when the toolchain tries to unpack the module inside the container. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: update TestConnectionTLS for native Go FIPS Add verification_mode:strict so Go's native TLS stack performs chain verification (InsecureSkipVerify=false) and the FIPS key-size check triggers. Update the expected error string to match native Go's message. Simplify the test comment to remove implementation details. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * mod: pin elastic-agent-libs to main for native Go FIPS validation Picks up the unreleased FIPS 140-3 peer-cert key-type enforcement (elastic/elastic-agent-libs#426), which TestConnectionTLS depends on. Update the test's expected error string to match the actual message that check produces, rather than Go's own (unreachable, since a custom TLS callback is always installed) internal FIPS error. This is a temporary pin for CI validation ahead of a tagged elastic-agent-libs release containing the fix. * notice: regenerate for elastic-agent-libs pin bump * mod: bump elastic-agent-libs to released v0.45.0 Swaps the temporary pseudo-version pin (unreleased main commit) for the now-tagged release containing the FIPS 140-3 peer-cert key-type enforcement fix (elastic/elastic-agent-libs#426). Same commit content (d72ff2012311), now backed by a real release. * notice: regenerate for elastic-agent-libs v0.45.0 * fix: address reviewer findings on FIPS migration - gotest.go: wire fipsTestEnv() into makeGoTestArgsForPackage too, so per-module integration test builds (FIPS=true mage goIntegTest on a single module) get GOFIPS140 set, not just the requirefips tag. - crossbuild.go: drop the bogus "G122" gosec rule code from the nolint comment on os.Chown (gosec has no G122; nolint:gosec suppresses by linter name, so the code was misleading, not functional). - connection_test.go: cover both elastic-agent-libs FIPS callback paths (verification_mode strict and none use different callbacks) and assert the RSA-1024-specific error message instead of the more generic FIPS-rejection substring. * test: require GOFIPS140 value to start with v1.0.0, not just contain it Contains would tolerate v1.0.0 appearing anywhere in the buildinfo value; HasPrefix asserts it's actually the version identifier itself. * test: describe TestConnectionTLS modes by behavior, not libs internals The comment named elastic-agent-libs' unexported checkAllChainsFIPS and fipsVerifyNoneCallback, leaking implementation details of a dependency with no compatibility guarantee. Describe the observable strict-vs-none distinction instead. --------- Signed-off-by: Dominik Rosiek <dominik.rosiek@elastic.co> Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> (cherry picked from commit a829d56) # Conflicts: # .buildkite/libbeat/pipeline.libbeat.yml # .buildkite/x-pack/pipeline.xpack.filebeat.yml # .buildkite/x-pack/pipeline.xpack.metricbeat.yml

mergify · 2026-07-15T06:43:37Z

Cherry-pick of a829d56 has failed:

On branch mergify/bp/9.3/pr-51345
Your branch is up to date with 'origin/9.3'.

You are currently cherry-picking commit a829d56a0.
  (fix conflicts and run "git cherry-pick --continue")
  (use "git cherry-pick --skip" to skip this patch)
  (use "git cherry-pick --abort" to cancel the cherry-pick operation)

Changes to be committed:
	modified:   .buildkite/scripts/custom_fips_ech_test.sh
	modified:   NOTICE.txt
	new file:   changelog/fragments/1781782084-use-native-go-for-linux-fips-builds.yaml
	modified:   dev-tools/mage/build.go
	modified:   dev-tools/mage/crossbuild.go
	modified:   dev-tools/mage/fips-settings.yaml
	modified:   dev-tools/mage/gotest.go
	modified:   dev-tools/mage/settings.go
	modified:   dev-tools/packaging/package_test.go
	modified:   go.mod
	modified:   go.sum
	modified:   libbeat/esleg/eslegclient/connection_test.go
	modified:   testing/go-ech/ech.go

Unmerged paths:
  (use "git add <file>..." to mark resolution)
	both modified:   .buildkite/libbeat/pipeline.libbeat.yml
	both modified:   .buildkite/x-pack/pipeline.xpack.filebeat.yml
	both modified:   .buildkite/x-pack/pipeline.xpack.metricbeat.yml

To fix up this pull request, you can check it out locally. See documentation: https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/reviewing-changes-in-pull-requests/checking-out-pull-requests-locally

github-actions · 2026-07-15T06:43:43Z

🤖 GitHub comments

Just comment with:

run docs-build : Re-trigger the docs validation. (use unformatted text in the comment!)
/test : Run the Buildkite pipeline.

infra-vault-gh-plugin-prod · 2026-07-15T06:43:50Z

Pinging @elastic/elastic-agent-control-plane (Team:Elastic-Agent-Control-Plane)

Mergify left unresolved conflict markers in three Buildkite pipeline files. Resolve by keeping 9.3's image versions (1782878510) and dropping IMAGE_UBUNTU_X86_64_FIPS, which no step references after the switch to native Go FIPS (GOFIPS140=v1.0.0). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

github-actions · 2026-07-15T06:52:14Z

TL;DR

The Buildkite failures are caused by unresolved cherry-pick conflict markers from the Mergify backport, not by the native Go FIPS code. The markers make all three affected pipeline YAML files invalid, so both pipeline upload steps fail and pre-commit reports check-merge-conflict failures.

Remediation

Resolve the conflicts in .buildkite/libbeat/pipeline.libbeat.yml, .buildkite/x-pack/pipeline.xpack.filebeat.yml, and .buildkite/x-pack/pipeline.xpack.metricbeat.yml; remove every <<<<<<<, =======, and >>>>>>> marker while retaining the intended 9.3 image settings and the incoming FIPS pipeline changes.
Run pre-commit run --all-files and validate each affected pipeline with buildkite-agent pipeline upload before rerunning the Buildkite build.

Investigation details

Root Cause

The PR head contains unresolved cherry-pick conflict blocks in the pipeline environment sections. The conflict markers are at .buildkite/libbeat/pipeline.libbeat.yml:14, :22, and :28; .buildkite/x-pack/pipeline.xpack.filebeat.yml:12, :15, and :17; and .buildkite/x-pack/pipeline.xpack.metricbeat.yml:12, :15, and :17.

Evidence

Build: https://buildkite.com/elastic/beats/builds/49460
Failed jobs: beats-libbeat pipeline upload, beats-xpack-filebeat pipeline upload, beats-xpack-metricbeat pipeline upload, and beats-xpack-dockerlogbeat pre-commit.
Pipeline upload error: pipeline parsing of "pipeline.libbeat.yml" failed: line 14: could not find expected : (the file-specific failures report the equivalent error at line 12).
Pre-commit error: check-merge-conflict found <<<<<<<, =======, and >>>>>>> in all three files.
The PR already records the failed cherry-pick and identifies these same three files as unmerged paths.

Verification

No source changes were made by this investigation; the failing Buildkite logs and PR head were inspected. The required verification is to rerun the pre-commit and pipeline-upload commands after conflict resolution.

What is this? | From workflow: PR Buildkite Detective

Give us feedback! React with 🚀 if perfect, 👍 if helpful, 👎 if not.

github-actions · 2026-07-15T07:06:53Z

TL;DR

The golangci-lint Ubuntu job failed on a new gosec G706 log-injection finding at dev-tools/mage/gotest.go:411, where log.Printf includes DEV_OS and DEV_ARCH. Remove the environment-derived values from this warning (or validate them against a fixed allowlist before logging), then rerun the lint check.

Remediation

Change the warning to log only a constant message, or log only validated platform values from a fixed set; do not pass raw os.Getenv results to log.Printf.
Rerun the golangci-lint workflow and confirm the Ubuntu job completes successfully.

Investigation details

Root Cause

The golangci-lint action ran golangci-lint v2.12.2 --timeout=90m --whole-files and reported G706: Log injection via taint analysis (gosec) for the warning that formats devOS and devArch, which are read from DEV_OS and DEV_ARCH. The job then exited with issues found.

Evidence

Workflow: https://github.com/elastic/beats/actions/runs/29395530096
Job/step: lint (ubuntu-latest) / golangci-lint
Key log excerpt: dev-tools/mage/gotest.go:411:14: G706: Log injection via taint analysis (gosec) followed by log.Printf("Warning: skipping -race flag for unsupported platform %s/%s\n", devOS, devArch)
The Windows and macOS jobs were cancelled after the Ubuntu failure; they did not provide independent failures.

Validation

No source changes were made by this investigation; the workflow log was inspected.

Follow-up

The earlier detective report on this PR concerns unresolved Buildkite YAML conflict markers and is a separate failure from this golangci-lint finding.

What is this? | From workflow: PR Actions Detective

Give us feedback! React with 🚀 if perfect, 👍 if helpful, 👎 if not.

DEV_OS/DEV_ARCH are trusted build-time env vars set by CI, not user input. Add nolint:gosec to match the fix already present on main. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

mergify · 2026-07-15T10:01:38Z

Tick the box to add this pull request to the merge queue (same as @mergifyio queue).

Queue this pull request

mergify Bot added the backport label Jul 15, 2026

mergify Bot requested a review from a team as a code owner July 15, 2026 06:43

mergify Bot added the conflicts There is a conflict in the backported pull request label Jul 15, 2026

mergify Bot requested review from a team as code owners July 15, 2026 06:43

mergify Bot requested review from khushijain21 and orestisfl and removed request for a team July 15, 2026 06:43

mergify Bot assigned macdewee Jul 15, 2026

botelastic Bot added the needs_team Indicates that the issue/PR needs a Team:* label label Jul 15, 2026

github-actions Bot added Team:Elastic-Agent-Control-Plane Label for the Agent Control Plane team dequeued labels Jul 15, 2026

botelastic Bot removed the needs_team Indicates that the issue/PR needs a Team:* label label Jul 15, 2026

mergify Bot mentioned this pull request Jul 15, 2026

Use native Go for Linux FIPS builds #51345

Merged

6 tasks

mergify Bot removed the dequeued label Jul 15, 2026

fix: suppress gosec G706 on dev-tool log line

d77e9b3

DEV_OS/DEV_ARCH are trusted build-time env vars set by CI, not user input. Add nolint:gosec to match the fix already present on main. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

macdewee approved these changes Jul 15, 2026

View reviewed changes

macdewee merged commit 3e81f1d into 9.3 Jul 15, 2026
206 checks passed

macdewee deleted the mergify/bp/9.3/pr-51345 branch July 15, 2026 09:22

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[9.3](backport #51345) Use native Go for Linux FIPS builds#51953

[9.3](backport #51345) Use native Go for Linux FIPS builds#51953
macdewee merged 3 commits into
9.3from
mergify/bp/9.3/pr-51345

mergify Bot commented Jul 15, 2026

Uh oh!

mergify Bot commented Jul 15, 2026

Uh oh!

github-actions Bot commented Jul 15, 2026

Uh oh!

infra-vault-gh-plugin-prod Bot commented Jul 15, 2026

Uh oh!

github-actions Bot commented Jul 15, 2026

Root Cause

Evidence

Verification

Uh oh!

github-actions Bot commented Jul 15, 2026

Root Cause

Evidence

Validation

Follow-up

Uh oh!

Uh oh!

mergify Bot commented Jul 15, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Uh oh!

Conversation

mergify Bot commented Jul 15, 2026

Proposed commit message

Checklist

Disruptive User Impact

How to test this PR locally

Related issues

Uh oh!

mergify Bot commented Jul 15, 2026

Uh oh!

github-actions Bot commented Jul 15, 2026

🤖 GitHub comments

Uh oh!

infra-vault-gh-plugin-prod Bot commented Jul 15, 2026

Uh oh!

github-actions Bot commented Jul 15, 2026

TL;DR

Remediation

Root Cause

Evidence

Verification

Uh oh!

github-actions Bot commented Jul 15, 2026

TL;DR

Remediation

Root Cause

Evidence

Validation

Follow-up

Uh oh!

Uh oh!

mergify Bot commented Jul 15, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant