Skip to content

Commit d2d429d

Browse files
macdeweemergify[bot]
macdewee
authored and
mergify[bot]
committed
Use native Go for Linux FIPS builds (#51345)
* feat: use native Go for Linux FIPS builds Switch Linux FIPS builds from Microsoft Go (GOEXPERIMENT=systemcrypto) to the upstream Go native FIPS module (GOFIPS140=v1.0.0, CMVP cert #5247). Drops CGO requirement and the Microsoft Go cross-build Docker image. Updates package tests to verify GOFIPS140 and DefaultGODEBUG buildinfo instead of GOEXPERIMENT and OpenSSL ELF symbols. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * chore: clean up MS Go FIPS artifacts - Remove GOEXPERIMENT=systemcrypto from CI pipelines and ECH test script - Switch FIPS unit/ECH test steps from AWS FIPS image to standard GCP image; unskip - Update VerifyFIPSBinary to check GOFIPS140+DefaultGODEBUG instead of GOEXPERIMENT+OpenSSL symbols - Strengthen GOFIPS140 assertion to require v1.0.0 version; improve error messages - Remove dead CGO field and OR from FIPSConfig/DefaultBuildArgs Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * chore: golangci-lint fixes Signed-off-by: Dominik Rosiek <dominik.rosiek@elastic.co> * fix: propagate FIPS env to unit tests and pre-populate module cache for crossbuild - Inject FIPS compile env vars (GOFIPS140=v1.0.0) into go test args so that test binaries actually enforce FIPS crypto at runtime, not just compile with the requirefips tag. - Pre-populate golang.org/fips140 in the host module cache before FIPS crossbuild containers mount it read-only, avoiding a write failure when the toolchain tries to unpack the module inside the container. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: update TestConnectionTLS for native Go FIPS Add verification_mode:strict so Go's native TLS stack performs chain verification (InsecureSkipVerify=false) and the FIPS key-size check triggers. Update the expected error string to match native Go's message. Simplify the test comment to remove implementation details. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * mod: pin elastic-agent-libs to main for native Go FIPS validation Picks up the unreleased FIPS 140-3 peer-cert key-type enforcement (elastic/elastic-agent-libs#426), which TestConnectionTLS depends on. Update the test's expected error string to match the actual message that check produces, rather than Go's own (unreachable, since a custom TLS callback is always installed) internal FIPS error. This is a temporary pin for CI validation ahead of a tagged elastic-agent-libs release containing the fix. * notice: regenerate for elastic-agent-libs pin bump * mod: bump elastic-agent-libs to released v0.45.0 Swaps the temporary pseudo-version pin (unreleased main commit) for the now-tagged release containing the FIPS 140-3 peer-cert key-type enforcement fix (elastic/elastic-agent-libs#426). Same commit content (d72ff2012311), now backed by a real release. * notice: regenerate for elastic-agent-libs v0.45.0 * fix: address reviewer findings on FIPS migration - gotest.go: wire fipsTestEnv() into makeGoTestArgsForPackage too, so per-module integration test builds (FIPS=true mage goIntegTest on a single module) get GOFIPS140 set, not just the requirefips tag. - crossbuild.go: drop the bogus "G122" gosec rule code from the nolint comment on os.Chown (gosec has no G122; nolint:gosec suppresses by linter name, so the code was misleading, not functional). - connection_test.go: cover both elastic-agent-libs FIPS callback paths (verification_mode strict and none use different callbacks) and assert the RSA-1024-specific error message instead of the more generic FIPS-rejection substring. * test: require GOFIPS140 value to start with v1.0.0, not just contain it Contains would tolerate v1.0.0 appearing anywhere in the buildinfo value; HasPrefix asserts it's actually the version identifier itself. * test: describe TestConnectionTLS modes by behavior, not libs internals The comment named elastic-agent-libs' unexported checkAllChainsFIPS and fipsVerifyNoneCallback, leaking implementation details of a dependency with no compatibility guarantee. Describe the observable strict-vs-none distinction instead. --------- Signed-off-by: Dominik Rosiek <dominik.rosiek@elastic.co> Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> (cherry picked from commit a829d56) # Conflicts: # .buildkite/libbeat/pipeline.libbeat.yml # .buildkite/x-pack/pipeline.xpack.filebeat.yml # .buildkite/x-pack/pipeline.xpack.metricbeat.yml
1 parent c01c713 commit d2d429d

16 files changed

Lines changed: 163 additions & 108 deletions

File tree

‎.buildkite/libbeat/pipeline.libbeat.yml‎

Lines changed: 11 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -11,13 +11,21 @@ env:
1111
IMAGE_UBUNTU_X86_64: "platform-ingest-beats-ubuntu-2204-1782878510"
1212
IMAGE_WIN_10: "platform-ingest-beats-windows-10-pro"
1313
IMAGE_WIN_11: "platform-ingest-beats-windows-11-pro"
14+
<<<<<<< HEAD
1415
IMAGE_WIN_10_VERSION: "1.0.1782878510"
1516
IMAGE_WIN_11_VERSION: "1.0.1782878510"
1617
IMAGE_WIN_2016: "platform-ingest-beats-windows-2016-1782878510"
1718
IMAGE_WIN_2019: "platform-ingest-beats-windows-2019-1782878510"
1819
IMAGE_WIN_2022: "platform-ingest-beats-windows-2022-1782878510"
1920
IMAGE_UBUNTU_X86_64_FIPS: "platform-ingest-beats-ubuntu-2204-fips-1782878510"
2021

22+
=======
23+
IMAGE_WIN_10_VERSION: "1.0.1781496166"
24+
IMAGE_WIN_11_VERSION: "1.0.1781496166"
25+
IMAGE_WIN_2016: "platform-ingest-beats-windows-2016-1781496166"
26+
IMAGE_WIN_2019: "platform-ingest-beats-windows-2019-1781496166"
27+
IMAGE_WIN_2022: "platform-ingest-beats-windows-2022-1781496166"
28+
>>>>>>> a829d56a0 (Use native Go for Linux FIPS builds (#51345))
2129
IMAGE_BEATS_WITH_HOOKS_LATEST: "docker.elastic.co/ci-agent-images/platform-ingest/buildkite-agent-beats-ci-with-hooks:latest"
2230

2331
#Deps
@@ -97,7 +105,6 @@ steps:
97105
context: "libbeat: Ubuntu x86_64 Unit Tests"
98106

99107
- label: ":ubuntu: Libbeat: Ubuntu x86_64 Go Unit Tests with fips provider and requirefips build tag"
100-
skip: "https://github.com/elastic/beats/issues/50269: ssh keys not FIPS"
101108
key: "mandatory-linux-unit-test-fips-tag"
102109
command: |
103110
cd libbeat
@@ -106,13 +113,11 @@ steps:
106113
automatic:
107114
- limit: 1
108115
agents:
109-
provider: "aws"
110-
image: "${IMAGE_UBUNTU_X86_64_FIPS}"
111-
instanceType: "m5.xlarge"
116+
provider: "gcp"
117+
image: "${IMAGE_UBUNTU_X86_64}"
118+
machineType: "${GCP_DEFAULT_MACHINE_TYPE}"
112119
env:
113120
FIPS: "true"
114-
GOEXPERIMENT: "systemcrypto"
115-
ASDF_PYTHON_VERSION: "3.9.13"
116121
artifact_paths:
117122
- "libbeat/build/*.xml"
118123
- "libbeat/build/*.json"

‎.buildkite/scripts/custom_fips_ech_test.sh‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -16,5 +16,5 @@ ech_up $STACK_VERSION
1616
echo "~~~ Running custom FIPS ECH tests"
1717

1818
pushd $BEAT_PATH
19-
GOEXPERIMENT=systemcrypto SNAPSHOT=true FIPS=true mage build fipsECHTest
19+
SNAPSHOT=true FIPS=true mage build fipsECHTest
2020
popd

‎.buildkite/x-pack/pipeline.xpack.filebeat.yml‎

Lines changed: 7 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -9,8 +9,12 @@ env:
99
GCP_HI_PERF_MACHINE_TYPE: "c2d-highcpu-16"
1010
GCP_WIN_MACHINE_TYPE: "n2-standard-8"
1111

12+
<<<<<<< HEAD
1213
IMAGE_UBUNTU_X86_64: "platform-ingest-beats-ubuntu-2204-1782878510"
1314
IMAGE_UBUNTU_X86_64_FIPS: "platform-ingest-beats-ubuntu-2204-fips-1782878510"
15+
=======
16+
IMAGE_UBUNTU_X86_64: "platform-ingest-beats-ubuntu-2204-1781496166"
17+
>>>>>>> a829d56a0 (Use native Go for Linux FIPS builds (#51345))
1418
IMAGE_WIN_10: "platform-ingest-beats-windows-10-pro"
1519
IMAGE_WIN_11: "platform-ingest-beats-windows-11-pro"
1620
IMAGE_WIN_10_VERSION: "1.0.1782878510"
@@ -221,10 +225,8 @@ steps:
221225
context: "x-pack/filebeat: Go fips140=only Integration Tests"
222226

223227
- label: ":ubuntu: x-pack/filebeat: FIPS ECH Integration Tests"
224-
skip: "https://github.com/elastic/beats/issues/50269: ssh keys not FIPS"
225228
env:
226229
ASDF_TERRAFORM_VERSION: "1.9.3"
227-
ASDF_PYTHON_VERSION: "3.9.13" # Not needed by ECH tests, but needed by VM
228230
# We are temporarily using the Production CFT environment instead of the Staging GovCloud
229231
# one. This is being done until issues with creating deployments in Staging GovCloud are
230232
# fixed. Once this happens, uncomment the lines below and delete the gcp-us-west2 line.
@@ -238,9 +240,9 @@ steps:
238240
automatic:
239241
- limit: 1
240242
agents:
241-
provider: "aws"
242-
image: "${IMAGE_UBUNTU_X86_64_FIPS}"
243-
instanceType: "m5.xlarge"
243+
provider: "gcp"
244+
image: "${IMAGE_UBUNTU_X86_64}"
245+
machineType: "${GCP_DEFAULT_MACHINE_TYPE}"
244246
artifact_paths:
245247
- "x-pack/filebeat/build/*.xml"
246248
- "x-pack/filebeat/build/*.json"

‎.buildkite/x-pack/pipeline.xpack.metricbeat.yml‎

Lines changed: 7 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -9,8 +9,12 @@ env:
99
GCP_HI_PERF_MACHINE_TYPE: "c2d-highcpu-16"
1010
GCP_WIN_MACHINE_TYPE: "n2-standard-8"
1111

12+
<<<<<<< HEAD
1213
IMAGE_UBUNTU_X86_64: "platform-ingest-beats-ubuntu-2204-1782878510"
1314
IMAGE_UBUNTU_X86_64_FIPS: "platform-ingest-beats-ubuntu-2204-fips-1782878510"
15+
=======
16+
IMAGE_UBUNTU_X86_64: "platform-ingest-beats-ubuntu-2204-1781496166"
17+
>>>>>>> a829d56a0 (Use native Go for Linux FIPS builds (#51345))
1418
IMAGE_WIN_10: "platform-ingest-beats-windows-10-pro"
1519
IMAGE_WIN_11: "platform-ingest-beats-windows-11-pro"
1620
IMAGE_WIN_10_VERSION: "1.0.1782878510"
@@ -260,10 +264,8 @@ steps:
260264
context: "x-pack/metricbeat: Go fips140=only Integration Tests (Module)"
261265

262266
- label: ":ubuntu: x-pack/metricbeat: FIPS ECH Integration Tests"
263-
skip: "https://github.com/elastic/beats/issues/50269: ssh keys not FIPS"
264267
env:
265268
ASDF_TERRAFORM_VERSION: "1.9.3"
266-
ASDF_PYTHON_VERSION: "3.9.13" # Not needed by ECH tests, but needed by VM
267269
# We are temporarily using the Production CFT environment instead of the Staging GovCloud
268270
# one. This is being done until issues with creating deployments in Staging GovCloud are
269271
# fixed. Once this happens, uncomment the lines below and delete the gcp-us-west2 line.
@@ -277,9 +279,9 @@ steps:
277279
automatic:
278280
- limit: 1
279281
agents:
280-
provider: "aws"
281-
image: "${IMAGE_UBUNTU_X86_64_FIPS}"
282-
instanceType: "m5.xlarge"
282+
provider: "gcp"
283+
image: "${IMAGE_UBUNTU_X86_64}"
284+
machineType: "${GCP_DEFAULT_MACHINE_TYPE}"
283285
artifact_paths:
284286
- "x-pack/metricbeat/build/*.xml"
285287
- "x-pack/metricbeat/build/*.json"

‎NOTICE.txt‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -10732,11 +10732,11 @@ SOFTWARE
1073210732

1073310733
--------------------------------------------------------------------------------
1073410734
Dependency : github.com/elastic/elastic-agent-libs
10735-
Version: v0.44.0
10735+
Version: v0.45.0
1073610736
Licence type (autodetected): Apache-2.0
1073710737
--------------------------------------------------------------------------------
1073810738

10739-
Contents of probable licence file $GOMODCACHE/github.com/elastic/elastic-agent-libs@v0.44.0/LICENSE:
10739+
Contents of probable licence file $GOMODCACHE/github.com/elastic/elastic-agent-libs@v0.45.0/LICENSE:
1074010740

1074110741
Apache License
1074210742
Version 2.0, January 2004

‎changelog/fragments/1781782084-use-native-go-for-linux-fips-builds.yaml‎

Lines changed: 45 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,45 @@
1+
# REQUIRED
2+
# Kind can be one of:
3+
# - breaking-change: a change to previously-documented behavior
4+
# - deprecation: functionality that is being removed in a later release
5+
# - bug-fix: fixes a problem in a previous version
6+
# - enhancement: extends functionality but does not break or fix existing behavior
7+
# - feature: new functionality
8+
# - known-issue: problems that we are aware of in a given version
9+
# - security: impacts on the security of a product or a user’s deployment.
10+
# - upgrade: important information for someone upgrading from a prior version
11+
# - other: does not fit into any of the other categories
12+
kind: enhancement
13+
14+
# REQUIRED for all kinds
15+
# Change summary; a 80ish characters long description of the change.
16+
summary: Use native Go for Linux FIPS builds
17+
18+
# REQUIRED for breaking-change, deprecation, known-issue
19+
# Long description; in case the summary is not enough to describe the change
20+
# this field accommodate a description without length limits.
21+
# description:
22+
23+
# REQUIRED for breaking-change, deprecation, known-issue
24+
# impact:
25+
26+
# REQUIRED for breaking-change, deprecation, known-issue
27+
# action:
28+
29+
# REQUIRED for all kinds
30+
# Affected component; usually one of "elastic-agent", "fleet-server", "filebeat", "metricbeat", "auditbeat", "all", etc.
31+
component: all
32+
33+
# AUTOMATED
34+
# OPTIONAL to manually add other PR URLs
35+
# PR URL: A link the PR that added the changeset.
36+
# If not present is automatically filled by the tooling finding the PR where this changelog fragment has been added.
37+
# NOTE: the tooling supports backports, so it's able to fill the original PR number instead of the backport PR number.
38+
# Please provide it if you are adding a fragment for a different PR.
39+
# pr: https://github.com/owner/repo/1234
40+
41+
# AUTOMATED
42+
# OPTIONAL to manually add other issue URLs
43+
# Issue URL; optional; the GitHub issue related to this changeset (either closes or is part of).
44+
# If not present is automatically filled by the tooling with the issue linked to the PR number.
45+
# issue: https://github.com/owner/repo/1234

‎dev-tools/mage/build.go‎

Lines changed: 5 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -29,6 +29,8 @@ import (
2929

3030
"github.com/josephspurrier/goversioninfo"
3131
"github.com/magefile/mage/sh"
32+
"golang.org/x/text/cases"
33+
"golang.org/x/text/language"
3234
)
3335

3436
// BuildArgs are the arguments used for the "build" target and they define how
@@ -113,7 +115,6 @@ func DefaultBuildArgs() BuildArgs {
113115
for _, tag := range FIPSConfig.Compile.Tags {
114116
args.ExtraFlags = append(args.ExtraFlags, "-tags="+tag)
115117
}
116-
args.CGO = args.CGO || FIPSConfig.Compile.CGO
117118
for varName, value := range FIPSConfig.Compile.Env {
118119
args.Env[varName] = value
119120
}
@@ -168,8 +169,8 @@ func DefaultGolangCrossBuildArgs() BuildArgs {
168169
// environment.
169170
func GolangCrossBuild(params BuildArgs) error {
170171
if os.Getenv("GOLANG_CROSSBUILD") != "1" {
171-
return errors.New("Use the crossBuild target. golangCrossBuild can " +
172-
"only be executed within the golang-crossbuild docker environment.")
172+
return errors.New("use the crossBuild target; golangCrossBuild can " +
173+
"only be executed within the golang-crossbuild docker environment")
173174
}
174175

175176
defer DockerChown(filepath.Join(params.OutputDir, params.Name+binaryExtension(GOOS)))
@@ -292,7 +293,7 @@ func MakeWindowsSysoFile() (string, error) {
292293
},
293294
StringFileInfo: goversioninfo.StringFileInfo{
294295
CompanyName: BeatVendor,
295-
ProductName: strings.Title(BeatName),
296+
ProductName: cases.Title(language.English).String(BeatName),
296297
ProductVersion: version,
297298
FileVersion: version,
298299
FileDescription: BeatDescription,

‎dev-tools/mage/crossbuild.go‎

Lines changed: 10 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -193,6 +193,15 @@ func CrossBuild(options ...CrossBuildOption) error {
193193
// Make sure the module dependencies are downloaded on the host,
194194
// as they will be mounted into the container read-only.
195195
mg.Deps(func() error { return gotool.Mod.Download() })
196+
if FIPSBuild {
197+
// GOFIPS140=v1.0.0 unpacks golang.org/fips140 from GOROOT/lib/fips140
198+
// into the module cache on first use. Pre-populate it on the host (as
199+
// the host user) before the container mounts the cache read-only.
200+
// Any go command triggers fips140.Init(), so list -m is sufficient.
201+
mg.Deps(func() error {
202+
return sh.RunWith(FIPSConfig.Compile.Env, "go", "list", "-m")
203+
})
204+
}
196205
}
197206

198207
// Build the magefile for Linux, so we can run it inside the container.
@@ -278,9 +287,6 @@ func CrossBuildImage(platform string) (string, error) {
278287
if err != nil {
279288
return "", err
280289
}
281-
if FIPSBuild {
282-
tagSuffix += "-fips"
283-
}
284290

285291
return BeatsCrossBuildImage + ":" + goVersion + "-" + tagSuffix, nil
286292
}
@@ -584,7 +590,7 @@ func chownPaths(uid, gid int, path string) error {
584590
return nil
585591
}
586592

587-
if err := os.Chown(name, uid, gid); err != nil {
593+
if err := os.Chown(name, uid, gid); err != nil { //nolint:gosec // paths are controlled build artifacts inside a Docker container, not user input
588594
return fmt.Errorf("failed to chown path=%v: %w", name, err)
589595
}
590596
numFixed++

‎dev-tools/mage/fips-settings.yaml‎

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -4,10 +4,11 @@ beats:
44
- filebeat
55
- metricbeat
66
compile:
7-
cgo: true
87
env:
9-
GOEXPERIMENT: systemcrypto
10-
MS_GOTOOLCHAIN_TELEMETRY_ENABLED: "0"
8+
# See https://go.dev/doc/security/fips140#go-cryptographic-module-v100
9+
# CMVP Certificate #5247: https://csrc.nist.gov/projects/cryptographic-module-validation-program/certificate/5247
10+
# CAVP Certificate A6650: https://csrc.nist.gov/projects/cryptographic-algorithm-validation-program/details?product=19371
11+
GOFIPS140: v1.0.0
1112
tags:
1213
- requirefips
1314
platforms:

‎dev-tools/mage/gotest.go‎

Lines changed: 14 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -69,7 +69,7 @@ func makeGoTestArgs(name string) GoTestArgs {
6969
TestName: name,
7070
Race: RaceDetector,
7171
Packages: []string{"./..."},
72-
Env: make(map[string]string),
72+
Env: fipsTestEnv(),
7373
OutputFile: fileName + ".out",
7474
JUnitReportFile: fileName + ".xml",
7575
Tags: testTagsFromEnv(),
@@ -89,6 +89,7 @@ func makeGoTestArgsForPackage(name, pkg string) GoTestArgs {
8989
TestName: fmt.Sprintf("%s-%s", name, pkg),
9090
Race: RaceDetector,
9191
Packages: []string{fmt.Sprintf("./module/%s", pkg)},
92+
Env: fipsTestEnv(),
9293
OutputFile: fileName + ".out",
9394
JUnitReportFile: fileName + ".xml",
9495
Tags: testTagsFromEnv(),
@@ -141,6 +142,18 @@ func testTagsFromEnv() []string {
141142
return tags
142143
}
143144

145+
// fipsTestEnv returns the environment variables required to compile and run FIPS tests.
146+
func fipsTestEnv() map[string]string {
147+
env := make(map[string]string, len(FIPSConfig.Compile.Env))
148+
if !FIPSBuild {
149+
return env
150+
}
151+
for k, v := range FIPSConfig.Compile.Env {
152+
env[k] = v
153+
}
154+
return env
155+
}
156+
144157
// DefaultGoTestUnitArgs returns a default set of arguments for running
145158
// all unit tests. We tag unit test files with '!integration'.
146159
func DefaultGoTestUnitArgs() GoTestArgs { return makeGoTestArgs("Unit") }

0 commit comments

Comments
 (0)