Skip to content

Commit 0684f3a

Browse files
macdeweeclaude
authored andcommitted
Use native Go for Linux FIPS builds (#51345)
* feat: use native Go for Linux FIPS builds Switch Linux FIPS builds from Microsoft Go (GOEXPERIMENT=systemcrypto) to the upstream Go native FIPS module (GOFIPS140=v1.0.0, CMVP cert #5247). Drops CGO requirement and the Microsoft Go cross-build Docker image. Updates package tests to verify GOFIPS140 and DefaultGODEBUG buildinfo instead of GOEXPERIMENT and OpenSSL ELF symbols. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * chore: clean up MS Go FIPS artifacts - Remove GOEXPERIMENT=systemcrypto from CI pipelines and ECH test script - Switch FIPS unit/ECH test steps from AWS FIPS image to standard GCP image; unskip - Update VerifyFIPSBinary to check GOFIPS140+DefaultGODEBUG instead of GOEXPERIMENT+OpenSSL symbols - Strengthen GOFIPS140 assertion to require v1.0.0 version; improve error messages - Remove dead CGO field and OR from FIPSConfig/DefaultBuildArgs Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * chore: golangci-lint fixes Signed-off-by: Dominik Rosiek <dominik.rosiek@elastic.co> * fix: propagate FIPS env to unit tests and pre-populate module cache for crossbuild - Inject FIPS compile env vars (GOFIPS140=v1.0.0) into go test args so that test binaries actually enforce FIPS crypto at runtime, not just compile with the requirefips tag. - Pre-populate golang.org/fips140 in the host module cache before FIPS crossbuild containers mount it read-only, avoiding a write failure when the toolchain tries to unpack the module inside the container. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: update TestConnectionTLS for native Go FIPS Add verification_mode:strict so Go's native TLS stack performs chain verification (InsecureSkipVerify=false) and the FIPS key-size check triggers. Update the expected error string to match native Go's message. Simplify the test comment to remove implementation details. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * mod: pin elastic-agent-libs to main for native Go FIPS validation Picks up the unreleased FIPS 140-3 peer-cert key-type enforcement (elastic/elastic-agent-libs#426), which TestConnectionTLS depends on. Update the test's expected error string to match the actual message that check produces, rather than Go's own (unreachable, since a custom TLS callback is always installed) internal FIPS error. This is a temporary pin for CI validation ahead of a tagged elastic-agent-libs release containing the fix. * notice: regenerate for elastic-agent-libs pin bump * mod: bump elastic-agent-libs to released v0.45.0 Swaps the temporary pseudo-version pin (unreleased main commit) for the now-tagged release containing the FIPS 140-3 peer-cert key-type enforcement fix (elastic/elastic-agent-libs#426). Same commit content (d72ff2012311), now backed by a real release. * notice: regenerate for elastic-agent-libs v0.45.0 * fix: address reviewer findings on FIPS migration - gotest.go: wire fipsTestEnv() into makeGoTestArgsForPackage too, so per-module integration test builds (FIPS=true mage goIntegTest on a single module) get GOFIPS140 set, not just the requirefips tag. - crossbuild.go: drop the bogus "G122" gosec rule code from the nolint comment on os.Chown (gosec has no G122; nolint:gosec suppresses by linter name, so the code was misleading, not functional). - connection_test.go: cover both elastic-agent-libs FIPS callback paths (verification_mode strict and none use different callbacks) and assert the RSA-1024-specific error message instead of the more generic FIPS-rejection substring. * test: require GOFIPS140 value to start with v1.0.0, not just contain it Contains would tolerate v1.0.0 appearing anywhere in the buildinfo value; HasPrefix asserts it's actually the version identifier itself. * test: describe TestConnectionTLS modes by behavior, not libs internals The comment named elastic-agent-libs' unexported checkAllChainsFIPS and fipsVerifyNoneCallback, leaking implementation details of a dependency with no compatibility guarantee. Describe the observable strict-vs-none distinction instead. --------- Signed-off-by: Dominik Rosiek <dominik.rosiek@elastic.co> Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> (cherry picked from commit a829d56)
1 parent d325362 commit 0684f3a

16 files changed

Lines changed: 147 additions & 112 deletions

File tree

.buildkite/libbeat/pipeline.libbeat.yml

Lines changed: 3 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -16,8 +16,6 @@ env:
1616
IMAGE_WIN_2016: "platform-ingest-beats-windows-2016-1781496166"
1717
IMAGE_WIN_2019: "platform-ingest-beats-windows-2019-1781496166"
1818
IMAGE_WIN_2022: "platform-ingest-beats-windows-2022-1781496166"
19-
IMAGE_UBUNTU_X86_64_FIPS: "platform-ingest-beats-ubuntu-2204-fips-1781496166"
20-
2119
IMAGE_BEATS_WITH_HOOKS_LATEST: "docker.elastic.co/ci-agent-images/platform-ingest/buildkite-agent-beats-ci-with-hooks:latest"
2220

2321
#Deps
@@ -97,7 +95,6 @@ steps:
9795
context: "libbeat: Ubuntu x86_64 Unit Tests"
9896

9997
- label: ":ubuntu: Libbeat: Ubuntu x86_64 Go Unit Tests with fips provider and requirefips build tag"
100-
skip: "https://github.com/elastic/beats/issues/50269: ssh keys not FIPS"
10198
key: "mandatory-linux-unit-test-fips-tag"
10299
command: |
103100
cd libbeat
@@ -106,13 +103,11 @@ steps:
106103
automatic:
107104
- limit: 1
108105
agents:
109-
provider: "aws"
110-
image: "${IMAGE_UBUNTU_X86_64_FIPS}"
111-
instanceType: "m5.xlarge"
106+
provider: "gcp"
107+
image: "${IMAGE_UBUNTU_X86_64}"
108+
machineType: "${GCP_DEFAULT_MACHINE_TYPE}"
112109
env:
113110
FIPS: "true"
114-
GOEXPERIMENT: "systemcrypto"
115-
ASDF_PYTHON_VERSION: "3.9.13"
116111
artifact_paths:
117112
- "libbeat/build/*.xml"
118113
- "libbeat/build/*.json"

.buildkite/scripts/custom_fips_ech_test.sh

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -14,5 +14,5 @@ ech_up $STACK_VERSION
1414
echo "~~~ Running custom FIPS ECH tests"
1515

1616
pushd $BEAT_PATH
17-
GOEXPERIMENT=systemcrypto SNAPSHOT=true FIPS=true mage build fipsECHTest
17+
SNAPSHOT=true FIPS=true mage build fipsECHTest
1818
popd

.buildkite/x-pack/pipeline.xpack.filebeat.yml

Lines changed: 3 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -10,7 +10,6 @@ env:
1010
GCP_WIN_MACHINE_TYPE: "n2-standard-8"
1111

1212
IMAGE_UBUNTU_X86_64: "platform-ingest-beats-ubuntu-2204-1781496166"
13-
IMAGE_UBUNTU_X86_64_FIPS: "platform-ingest-beats-ubuntu-2204-fips-1781496166"
1413
IMAGE_WIN_10: "platform-ingest-beats-windows-10-pro"
1514
IMAGE_WIN_11: "platform-ingest-beats-windows-11-pro"
1615
IMAGE_WIN_10_VERSION: "1.0.1781496166"
@@ -221,10 +220,8 @@ steps:
221220
context: "x-pack/filebeat: Go fips140=only Integration Tests"
222221

223222
- label: ":ubuntu: x-pack/filebeat: FIPS ECH Integration Tests"
224-
skip: "https://github.com/elastic/beats/issues/50269: ssh keys not FIPS"
225223
env:
226224
ASDF_TERRAFORM_VERSION: "1.9.3"
227-
ASDF_PYTHON_VERSION: "3.9.13" # Not needed by ECH tests, but needed by VM
228225
# We are temporarily using the Production CFT environment instead of the Staging GovCloud
229226
# one. This is being done until issues with creating deployments in Staging GovCloud are
230227
# fixed. Once this happens, uncomment the lines below and delete the gcp-us-west2 line.
@@ -238,9 +235,9 @@ steps:
238235
automatic:
239236
- limit: 1
240237
agents:
241-
provider: "aws"
242-
image: "${IMAGE_UBUNTU_X86_64_FIPS}"
243-
instanceType: "m5.xlarge"
238+
provider: "gcp"
239+
image: "${IMAGE_UBUNTU_X86_64}"
240+
machineType: "${GCP_DEFAULT_MACHINE_TYPE}"
244241
artifact_paths:
245242
- "x-pack/filebeat/build/*.xml"
246243
- "x-pack/filebeat/build/*.json"

.buildkite/x-pack/pipeline.xpack.metricbeat.yml

Lines changed: 3 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -10,7 +10,6 @@ env:
1010
GCP_WIN_MACHINE_TYPE: "n2-standard-8"
1111

1212
IMAGE_UBUNTU_X86_64: "platform-ingest-beats-ubuntu-2204-1781496166"
13-
IMAGE_UBUNTU_X86_64_FIPS: "platform-ingest-beats-ubuntu-2204-fips-1781496166"
1413
IMAGE_WIN_10: "platform-ingest-beats-windows-10-pro"
1514
IMAGE_WIN_11: "platform-ingest-beats-windows-11-pro"
1615
IMAGE_WIN_10_VERSION: "1.0.1781496166"
@@ -260,10 +259,8 @@ steps:
260259
context: "x-pack/metricbeat: Go fips140=only Integration Tests (Module)"
261260

262261
- label: ":ubuntu: x-pack/metricbeat: FIPS ECH Integration Tests"
263-
skip: "https://github.com/elastic/beats/issues/50269: ssh keys not FIPS"
264262
env:
265263
ASDF_TERRAFORM_VERSION: "1.9.3"
266-
ASDF_PYTHON_VERSION: "3.9.13" # Not needed by ECH tests, but needed by VM
267264
# We are temporarily using the Production CFT environment instead of the Staging GovCloud
268265
# one. This is being done until issues with creating deployments in Staging GovCloud are
269266
# fixed. Once this happens, uncomment the lines below and delete the gcp-us-west2 line.
@@ -277,9 +274,9 @@ steps:
277274
automatic:
278275
- limit: 1
279276
agents:
280-
provider: "aws"
281-
image: "${IMAGE_UBUNTU_X86_64_FIPS}"
282-
instanceType: "m5.xlarge"
277+
provider: "gcp"
278+
image: "${IMAGE_UBUNTU_X86_64}"
279+
machineType: "${GCP_DEFAULT_MACHINE_TYPE}"
283280
artifact_paths:
284281
- "x-pack/metricbeat/build/*.xml"
285282
- "x-pack/metricbeat/build/*.json"

NOTICE.txt

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -10761,11 +10761,11 @@ SOFTWARE
1076110761

1076210762
--------------------------------------------------------------------------------
1076310763
Dependency : github.com/elastic/elastic-agent-libs
10764-
Version: v0.44.0
10764+
Version: v0.45.0
1076510765
Licence type (autodetected): Apache-2.0
1076610766
--------------------------------------------------------------------------------
1076710767

10768-
Contents of probable licence file $GOMODCACHE/github.com/elastic/elastic-agent-libs@v0.44.0/LICENSE:
10768+
Contents of probable licence file $GOMODCACHE/github.com/elastic/elastic-agent-libs@v0.45.0/LICENSE:
1076910769

1077010770
Apache License
1077110771
Version 2.0, January 2004
Lines changed: 45 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,45 @@
1+
# REQUIRED
2+
# Kind can be one of:
3+
# - breaking-change: a change to previously-documented behavior
4+
# - deprecation: functionality that is being removed in a later release
5+
# - bug-fix: fixes a problem in a previous version
6+
# - enhancement: extends functionality but does not break or fix existing behavior
7+
# - feature: new functionality
8+
# - known-issue: problems that we are aware of in a given version
9+
# - security: impacts on the security of a product or a user’s deployment.
10+
# - upgrade: important information for someone upgrading from a prior version
11+
# - other: does not fit into any of the other categories
12+
kind: enhancement
13+
14+
# REQUIRED for all kinds
15+
# Change summary; a 80ish characters long description of the change.
16+
summary: Use native Go for Linux FIPS builds
17+
18+
# REQUIRED for breaking-change, deprecation, known-issue
19+
# Long description; in case the summary is not enough to describe the change
20+
# this field accommodate a description without length limits.
21+
# description:
22+
23+
# REQUIRED for breaking-change, deprecation, known-issue
24+
# impact:
25+
26+
# REQUIRED for breaking-change, deprecation, known-issue
27+
# action:
28+
29+
# REQUIRED for all kinds
30+
# Affected component; usually one of "elastic-agent", "fleet-server", "filebeat", "metricbeat", "auditbeat", "all", etc.
31+
component: all
32+
33+
# AUTOMATED
34+
# OPTIONAL to manually add other PR URLs
35+
# PR URL: A link the PR that added the changeset.
36+
# If not present is automatically filled by the tooling finding the PR where this changelog fragment has been added.
37+
# NOTE: the tooling supports backports, so it's able to fill the original PR number instead of the backport PR number.
38+
# Please provide it if you are adding a fragment for a different PR.
39+
# pr: https://github.com/owner/repo/1234
40+
41+
# AUTOMATED
42+
# OPTIONAL to manually add other issue URLs
43+
# Issue URL; optional; the GitHub issue related to this changeset (either closes or is part of).
44+
# If not present is automatically filled by the tooling with the issue linked to the PR number.
45+
# issue: https://github.com/owner/repo/1234

dev-tools/mage/build.go

Lines changed: 5 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -29,6 +29,8 @@ import (
2929

3030
"github.com/josephspurrier/goversioninfo"
3131
"github.com/magefile/mage/sh"
32+
"golang.org/x/text/cases"
33+
"golang.org/x/text/language"
3234
)
3335

3436
// BuildArgs are the arguments used for the "build" target and they define how
@@ -113,7 +115,6 @@ func DefaultBuildArgs() BuildArgs {
113115
for _, tag := range FIPSConfig.Compile.Tags {
114116
args.ExtraFlags = append(args.ExtraFlags, "-tags="+tag)
115117
}
116-
args.CGO = args.CGO || FIPSConfig.Compile.CGO
117118
for varName, value := range FIPSConfig.Compile.Env {
118119
args.Env[varName] = value
119120
}
@@ -168,8 +169,8 @@ func DefaultGolangCrossBuildArgs() BuildArgs {
168169
// environment.
169170
func GolangCrossBuild(params BuildArgs) error {
170171
if os.Getenv("GOLANG_CROSSBUILD") != "1" {
171-
return errors.New("Use the crossBuild target. golangCrossBuild can " +
172-
"only be executed within the golang-crossbuild docker environment.")
172+
return errors.New("use the crossBuild target; golangCrossBuild can " +
173+
"only be executed within the golang-crossbuild docker environment")
173174
}
174175

175176
mountPoint, err := ElasticBeatsDir()
@@ -313,7 +314,7 @@ func MakeWindowsSysoFile() (string, error) {
313314
},
314315
StringFileInfo: goversioninfo.StringFileInfo{
315316
CompanyName: BeatVendor,
316-
ProductName: strings.Title(BeatName),
317+
ProductName: cases.Title(language.English).String(BeatName),
317318
ProductVersion: version,
318319
FileVersion: version,
319320
FileDescription: BeatDescription,

dev-tools/mage/crossbuild.go

Lines changed: 10 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -193,6 +193,15 @@ func CrossBuild(options ...CrossBuildOption) error {
193193
// Make sure the module dependencies are downloaded on the host,
194194
// as they will be mounted into the container read-only.
195195
mg.Deps(func() error { return gotool.Mod.Download() })
196+
if FIPSBuild {
197+
// GOFIPS140=v1.0.0 unpacks golang.org/fips140 from GOROOT/lib/fips140
198+
// into the module cache on first use. Pre-populate it on the host (as
199+
// the host user) before the container mounts the cache read-only.
200+
// Any go command triggers fips140.Init(), so list -m is sufficient.
201+
mg.Deps(func() error {
202+
return sh.RunWith(FIPSConfig.Compile.Env, "go", "list", "-m")
203+
})
204+
}
196205
}
197206

198207
// Build the magefile for Linux, so we can run it inside the container.
@@ -278,9 +287,6 @@ func CrossBuildImage(platform string) (string, error) {
278287
if err != nil {
279288
return "", err
280289
}
281-
if FIPSBuild {
282-
tagSuffix += "-fips"
283-
}
284290

285291
return BeatsCrossBuildImage + ":" + goVersion + "-" + tagSuffix, nil
286292
}
@@ -584,7 +590,7 @@ func chownPaths(uid, gid int, path string) error {
584590
return nil
585591
}
586592

587-
if err := os.Chown(name, uid, gid); err != nil {
593+
if err := os.Chown(name, uid, gid); err != nil { //nolint:gosec // paths are controlled build artifacts inside a Docker container, not user input
588594
return fmt.Errorf("failed to chown path=%v: %w", name, err)
589595
}
590596
numFixed++

dev-tools/mage/fips-settings.yaml

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -5,10 +5,11 @@ beats:
55
- metricbeat
66
- heartbeat
77
compile:
8-
cgo: true
98
env:
10-
GOEXPERIMENT: systemcrypto
11-
MS_GOTOOLCHAIN_TELEMETRY_ENABLED: "0"
9+
# See https://go.dev/doc/security/fips140#go-cryptographic-module-v100
10+
# CMVP Certificate #5247: https://csrc.nist.gov/projects/cryptographic-module-validation-program/certificate/5247
11+
# CAVP Certificate A6650: https://csrc.nist.gov/projects/cryptographic-algorithm-validation-program/details?product=19371
12+
GOFIPS140: v1.0.0
1213
tags:
1314
- requirefips
1415
platforms:

dev-tools/mage/gotest.go

Lines changed: 14 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -69,7 +69,7 @@ func makeGoTestArgs(name string) GoTestArgs {
6969
TestName: name,
7070
Race: RaceDetector,
7171
Packages: []string{"./..."},
72-
Env: make(map[string]string),
72+
Env: fipsTestEnv(),
7373
OutputFile: fileName + ".out",
7474
JUnitReportFile: fileName + ".xml",
7575
Tags: testTagsFromEnv(),
@@ -89,6 +89,7 @@ func makeGoTestArgsForPackage(name, pkg string) GoTestArgs {
8989
TestName: fmt.Sprintf("%s-%s", name, pkg),
9090
Race: RaceDetector,
9191
Packages: []string{fmt.Sprintf("./module/%s", pkg)},
92+
Env: fipsTestEnv(),
9293
OutputFile: fileName + ".out",
9394
JUnitReportFile: fileName + ".xml",
9495
Tags: testTagsFromEnv(),
@@ -141,6 +142,18 @@ func testTagsFromEnv() []string {
141142
return tags
142143
}
143144

145+
// fipsTestEnv returns the environment variables required to compile and run FIPS tests.
146+
func fipsTestEnv() map[string]string {
147+
env := make(map[string]string, len(FIPSConfig.Compile.Env))
148+
if !FIPSBuild {
149+
return env
150+
}
151+
for k, v := range FIPSConfig.Compile.Env {
152+
env[k] = v
153+
}
154+
return env
155+
}
156+
144157
// DefaultGoTestUnitArgs returns a default set of arguments for running
145158
// all unit tests. We tag unit test files with '!integration'.
146159
func DefaultGoTestUnitArgs() GoTestArgs { return makeGoTestArgs("Unit") }

0 commit comments

Comments
 (0)