Skip to content

Commit e1113b6

Browse files
emiliaFermailo-nrkb-newrelicotelcomm-botrenovate[bot]
authored
feat: fips distributions (#387)
* feat: create fips versions of distributions * feat: restructure so that both fips compliant and noncompliant versions of each distro is made * feat: separate build.sh and .goreleaser.yaml into default and fips versions * feat: change build-fips.sh so that instead of rebuilding the same thing over, just copy contents of _build into _build-fips * Revert "feat: change build-fips.sh so that instead of rebuilding the same thing over, just copy contents of _build into _build-fips" This reverts commit 600e08d. _build-fips can't just copy over the contents of _build because _build-fips has it's own manifest-fips.yaml * feat: edit configure.go to include fips * feat: data agreement * fix: fix bugs with Docker Images and Manifests * fix: ran make generate-goreleaser * feat: add fips version of files for new distro - nrdot-collector * feat: add fips flag * feat: add Dockerfile-fips files * move fips flag further down configure.go stack * use docker args * standardize fips in configure.go and manifest files * make build target dependent on build-fips target * add ci-fips workflow * add infrastructure to cross make ocb with cgo enabled * adjust fips workflow, doesn't need tf * bug squishing * remove unnecessary toolchain dir * chore: fips testing (#373) * test: fix nightly (#363) * feat: add e2e tests for core distro (#364) * feat: add e2e tests for core distro * chore: merge host config with core collector * feat: Bump otel component versions from v0.128.0 to v0.131.0 (#357) * chore: prep release 1.3.0 (#365) * chore: Add nrdot-collector to release workflows (#366) * chore: Add missing ec2 nightly deploy (#367) * chore: Address cancelled deploy step (#370) * feat: Add spanprocessor to nrdot-collector (#371) * chore(deps): update helm release nr-k8s-otel-collector to v0.8.40 (#369) Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> * feat: Bump otel component versions from v0.131.0 to v0.132.0 (#368) * chore: fips testing * chore: test fips docker * chore: cleaning up fips changes --------- Co-authored-by: kb-newrelic <121687305+kb-newrelic@users.noreply.github.com> Co-authored-by: otelcomm-bot <svc-otelcomm-bot@newrelic.com> Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> Co-authored-by: Emilia Ferreyra <110185663+emiliaFer@users.noreply.github.com> * build details overrides * git merged badly, fixing it * update fips workflow * add back fips configs * updated goreleaser.yaml files * add back fips go tags and ldflags * fix directory in goreleaser-fips * add boringcrypto validation * update boring crypto validation * add manifest-fips.yaml to .gitignore * remove existing manifest-fips.yaml files --------- Co-authored-by: mailo-nr <marsac@newrelic.com> Co-authored-by: kb-newrelic <121687305+kb-newrelic@users.noreply.github.com> Co-authored-by: otelcomm-bot <svc-otelcomm-bot@newrelic.com> Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
1 parent c77ca84 commit e1113b6

30 files changed

+818
-73
lines changed

‎.github/workflows/ci-base.yaml‎

Lines changed: 36 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,10 @@ on:
1010
required: false
1111
type: boolean
1212
default: false
13+
fips:
14+
required: false
15+
type: boolean
16+
default: false
1317
test_cluster_name:
1418
required: false
1519
type: string
@@ -70,6 +74,18 @@ jobs:
7074

7175
- uses: docker/setup-buildx-action@v2
7276

77+
- name: Install cross-compilation toolchain
78+
if: inputs.fips == true
79+
run: |
80+
sudo apt-get update
81+
sudo apt-get install -y \
82+
gcc-aarch64-linux-gnu \
83+
g++-aarch64-linux-gnu \
84+
gcc-x86-64-linux-gnu \
85+
g++-x86-64-linux-gnu \
86+
libc6-dev-arm64-cross \
87+
libc6-dev-amd64-cross
88+
7389
- name: Import GPG key
7490
if: ${{github.event.pull_request.user.login != 'dependabot[bot]' }}
7591
id: import_gpg
@@ -92,6 +108,8 @@ jobs:
92108
run: |
93109
if [ ${{ inputs.nightly }} = "true" ]; then
94110
echo "goreleaser_args=--snapshot --clean --skip=publish,validate --timeout 2h --config .goreleaser-nightly.yaml" >> $GITHUB_ENV
111+
elif [ ${{ inputs.fips }} = "true" ]; then
112+
echo "goreleaser_args=--snapshot --clean --skip=publish,validate --timeout 2h --config .goreleaser-fips.yaml" >> $GITHUB_ENV
95113
elif [ ${{github.event.pull_request.user.login == 'dependabot[bot]' }} ]; then
96114
echo "goreleaser_args=--snapshot --clean --skip=publish,validate,sign --timeout 2h" >> $GITHUB_ENV
97115
else
@@ -121,6 +139,8 @@ jobs:
121139
echo "arch=$ARCH" >> $GITHUB_ENV
122140
if [ ${{ inputs.nightly }} = "true" ]; then
123141
echo "image_tag=$VERSION-nightly-$ARCH" >> $GITHUB_ENV
142+
elif [ ${{ inputs.fips }} = "true" ]; then
143+
echo "image_tag=$VERSION-fips-$ARCH" >> $GITHUB_ENV
124144
else
125145
echo "image_tag=$VERSION-$ARCH" >> $GITHUB_ENV
126146
fi
@@ -138,6 +158,22 @@ jobs:
138158
139159
- uses: azure/setup-helm@v4.2.0
140160

161+
- name: Validate Usage of BoringCrypto
162+
if: inputs.fips == true
163+
run: |
164+
ARCH=$(echo '${{ runner.arch }}' | sed 's/X/amd/g')
165+
ARCH=${ARCH@L}
166+
OS=$(echo '${{ runner.os }}')
167+
OS=${OS@L}
168+
printf -v BINARY_PATTERN "distributions/%s/dist/%s-fips_%s_%s*/*" "${{ inputs.distribution}}" "${{ inputs.distribution }}" "${OS}" "${ARCH}"
169+
echo "Looking for binary pattern: ${BINARY_PATTERN}"
170+
BINARY=$(find ${BINARY_PATTERN} 2>/dev/null | head -1)
171+
if [[ -z "${BINARY}" ]]; then
172+
echo "Error"
173+
exit 1
174+
fi
175+
go tool nm ${BINARY} | grep '_Cfunc__goboringcrypto_' || (echo 'fips distro should not use standard crypto' && exit 1)
176+
141177
- name: Run local e2e tests
142178
if: ${{ hashFiles(format('distributions/{0}/test/spec-local.yaml', inputs.distribution)) != '' }}
143179
uses: newrelic/newrelic-integration-e2e-action@v1

‎.github/workflows/ci-fips.yaml‎

Lines changed: 33 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,33 @@
1+
name: 🔄 CI | FIPS
2+
3+
on:
4+
push:
5+
branches: [main]
6+
pull_request:
7+
branches: [main]
8+
9+
jobs:
10+
build:
11+
name: Build & Validate
12+
strategy:
13+
matrix:
14+
distribution:
15+
- nrdot-collector-host
16+
- nrdot-collector-k8s
17+
- nrdot-collector
18+
uses: ./.github/workflows/ci-base.yaml
19+
with:
20+
fips: true
21+
distribution: ${{ matrix.distribution }}
22+
# namespace by distro to avoid issues with cleanup (distro 1 still running tests while distro 2 cleans up cluster)
23+
test_cluster_name: '${{ matrix.distribution }}-${{ github.run_id }}-${{ github.run_attempt }}'
24+
secrets:
25+
docker_hub_username: ${{ secrets.OTELCOMM_DOCKER_HUB_USERNAME }}
26+
docker_hub_password: ${{ secrets.OTELCOMM_DOCKER_HUB_PASSWORD }}
27+
gpg_private_key: ${{ secrets.OTELCOMM_GPG_PRIVATE_KEY_BASE64 }}
28+
gpg_passphrase: ${{ secrets.OTELCOMM_GPG_PASSPHRASE }}
29+
registry: 'newrelic'
30+
nr_backend_url: ${{ secrets.NR_STAGING_BACKEND_URL }}
31+
nr_ingest_key: ${{ secrets.OTELCOMM_NR_INGEST_KEY }}
32+
nr_account_id: ${{ secrets.OTELCOMM_NR_TEST_ACCOUNT_ID }}
33+
nr_api_key: ${{ secrets.OTELCOMM_NR_API_KEY }}

‎.github/workflows/ci.yaml‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -21,6 +21,7 @@ jobs:
2121
- nrdot-collector
2222
uses: ./.github/workflows/ci-base.yaml
2323
with:
24+
fips: false
2425
distribution: ${{ matrix.distribution }}
2526
# namespace by distro to avoid issues with cleanup (distro 1 still running tests while distro 2 cleans up cluster)
2627
test_cluster_name: '${{ matrix.distribution }}-${{ github.run_id }}-${{ github.run_attempt }}'

‎.gitignore‎

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,6 @@
11
vendor
22
**/_build
3+
**/_build-fips
34
**/collections
45
**/roles
56
dist/
@@ -16,10 +17,13 @@ inventory
1617
*.auto.tfvars
1718
# rely on pinned provider versions for now to avoid having to deal with multi-platform lock file hashes
1819
.terraform.lock.hcl
20+
#fips manifest files
21+
distributions/**/manifest-fips.yaml
1922

2023
.env
2124
.secrets
2225
.input
2326
.tools
27+
.scratch
2428

25-
test/**/charts/*.tgz
29+
test/**/charts/*.tgz

‎Makefile‎

Lines changed: 46 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -15,13 +15,18 @@ TOOLS_PKG_NAMES := $(shell grep -E $(TOOLS_MOD_REGEX) < $(TOOLS_MOD_DIR)/tools.g
1515
TOOLS_BIN_NAMES := $(addprefix $(TOOLS_BIN_DIR)/, $(notdir $(shell echo $(TOOLS_PKG_NAMES))))
1616
GO_LICENCE_DETECTOR := $(TOOLS_BIN_DIR)/go-licence-detector
1717
GO_LICENCE_DETECTOR_CONFIG := $(SRC_ROOT)/internal/assets/license/rules.json
18+
CGO := 0
1819

1920
DISTRIBUTIONS ?= "nrdot-collector-host,nrdot-collector-k8s,nrdot-collector"
2021

2122
ci: check build version-check licenses-check
2223
check: ensure-goreleaser-up-to-date
2324

24-
build: go ocb
25+
build: build-fips
26+
@./scripts/build.sh -d "${DISTRIBUTIONS}" -b ${OTELCOL_BUILDER} -f false
27+
28+
build-fips: go
29+
@$(MAKE) ocb CGO=1
2530
@./scripts/build.sh -d "${DISTRIBUTIONS}" -b ${OTELCOL_BUILDER}
2631

2732
generate: generate-sources generate-goreleaser
@@ -53,13 +58,52 @@ ifeq (, $(shell command -v ocb 2>/dev/null))
5358
[ "$${machine}" != x86_64 ] || machine=amd64 ;\
5459
echo "Installing ocb ($${os}/$${machine}) at $(OTELCOL_BUILDER_DIR)";\
5560
mkdir -p $(OTELCOL_BUILDER_DIR) ;\
56-
CGO_ENABLED=0 go install -trimpath -ldflags="-s -w" go.opentelemetry.io/collector/cmd/builder@v$(OTELCOL_BUILDER_VERSION) ;\
61+
CGO_ENABLED=${CGO} go install -trimpath -ldflags="-s -w" go.opentelemetry.io/collector/cmd/builder@v$(OTELCOL_BUILDER_VERSION) ;\
5762
mv $$(go env GOPATH)/bin/builder $(OTELCOL_BUILDER) ;\
5863
}
5964
else
6065
OTELCOL_BUILDER=$(shell command -v ocb)
6166
endif
6267

68+
.PHONY: ocb-version-check
69+
ocb-version-check:
70+
@need_install=false; \
71+
if [ -x '$(OTELCOL_BUILDER)' ]; then \
72+
current_version=$$($(OTELCOL_BUILDER) version 2>&1 | grep -o 'v[0-9]\+\.[0-9]\+\.[0-9]\+' | sed 's/.*v//') ;\
73+
if [ "$${current_version}" != "$(OTELCOL_BUILDER_VERSION)" ]; then \
74+
echo "OCB version mismatch: found $${current_version} $(OTELCOL_BUILDER), expected $(OTELCOL_BUILDER_VERSION)" ;\
75+
rm -f $(OTELCOL_BUILDER) ;\
76+
need_install=true; \
77+
else \
78+
echo "OCB found correct version $${current_version}" ;\
79+
fi \
80+
elif command -v ocb 2>&1; then \
81+
current_version=$$(ocb version 2>&1 | grep -o 'v[0-9]\+\.[0-9]\+\.[0-9]\+' | sed 's/.*v//') ;\
82+
if [ "$${current_version}" != "$(OTELCOL_BUILDER_VERSION)" ]; then \
83+
echo "System OCB version mismatch: found $${current_version}, expected $(OTELCOL_BUILDER_VERSION)" ;\
84+
echo "Will install local version..." ;\
85+
need_install=true; \
86+
else \
87+
echo "OCB found correct version $${current_version}" ;\
88+
fi \
89+
else \
90+
echo "OCB not found, will install..." ;\
91+
need_install=true; \
92+
fi; \
93+
if [ "$$need_install" = "true" ]; then \
94+
set -e ;\
95+
os=$$(uname | tr A-Z a-z) ;\
96+
machine=$$(uname -m) ;\
97+
[ "$${machine}" != x86 ] || machine=386 ;\
98+
[ "$${machine}" != x86_64 ] || machine=amd64 ;\
99+
echo "Installing ocb ($${os}/$${machine}) at $(OTELCOL_BUILDER_DIR)";\
100+
mkdir -p $(OTELCOL_BUILDER_DIR) ;\
101+
go clean -modcache; \
102+
CGO_ENABLED=0 go install -trimpath -ldflags="-s -w" go.opentelemetry.io/collector/cmd/builder@v$(OTELCOL_BUILDER_VERSION) ;\
103+
mv $$(go env GOPATH)/bin/builder $(OTELCOL_BUILDER) ;\
104+
echo "OCB installed at $(OTELCOL_BUILDER)"; \
105+
fi
106+
63107
.PHONY: go
64108
go:
65109
@{ \

‎Makefile.dev‎

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,12 @@ ci_custom_matrix:
77
act push -W .github/workflows/ci.yaml \
88
--matrix distribution:nrdot-collector-k8s
99

10+
ci_custom_fips_matrix:
11+
@# repeat --matrix arg for multiple distros
12+
act push -W .github/workflows/ci-fips.yaml \
13+
--matrix distribution:nrdot-collector-host
14+
1015
ci_nightly_custom_matrix:
1116
@# repeat --matrix arg for multiple distros
1217
act schedule -W .github/workflows/ci-nightly.yaml \
13-
--matrix distribution:nrdot-collector-host
18+
--matrix distribution:nrdot-collector-host

0 commit comments

Comments
 (0)