Skip to content

Commit 3a08e0e

Browse files
kriscolemankriscoleman
authored
fix: eliminate critical/high vulnerabilities and add Grype/Scout scanning targets (#427)
fix: static binary, trixie-slim runtime, Grype/Scout targets, and CMX E2E tests - Build a fully static binary (CGO_ENABLED=0) so the runtime image no longer needs runtime library packages. - Switch runtime base from debian:bookworm-slim to debian:trixie-slim. Fresh scans (May 2026) show trixie carries fewer inherited CVEs: Grype: 2 critical / 12 high (vs 3 / 22 on bookworm) Scout: 0 critical / 0 high (vs 0 / 4 on bookworm) Also fixes CVE-2026-33845 (gnutls) and CVE-2023-45853 (zlib). - Add Makefile targets for vulnerability scanning: make grype-repo # Grype filesystem scan make grype-image # Grype image scan make docker-scout-image # Docker Scout image scan - Add automated E2E test automation via Replicated CMX: scripts/e2e-test.sh # VM lifecycle + kURL + health checks scripts/kurl-installer-minimal.yaml # Minimal kURL spec for speed make test-e2e / make test-e2e-ci # Local and CI entrypoints - Add .github/workflows/e2e-test.yaml triggered on push to main or workflow_dispatch. Auto-discovers accessible Replicated apps, validates the API token, and skips gracefully when unavailable. - Add AGENTS.md and CLAUDE.md project documentation. - Update README.md to document the automated E2E workflow.
1 parent ba093fd commit 3a08e0e

10 files changed

Lines changed: 581 additions & 9 deletions

File tree

‎.github/workflows/e2e-test.yaml‎

Lines changed: 114 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,114 @@
1+
name: e2e-test
2+
3+
on:
4+
push:
5+
branches:
6+
- main
7+
workflow_dispatch:
8+
inputs:
9+
ekco_image:
10+
description: 'EKCO image to test (e.g. ttl.sh/user/ekco:tag)'
11+
required: true
12+
type: string
13+
replicated_app:
14+
description: 'Replicated app slug to use for CMX (default: kurl-sandbox)'
15+
required: false
16+
type: string
17+
default: 'kurl-sandbox'
18+
19+
jobs:
20+
e2e-test:
21+
runs-on: ubuntu-22.04
22+
steps:
23+
- uses: actions/checkout@v6
24+
25+
- name: Setup Go
26+
uses: actions/setup-go@v6
27+
with:
28+
go-version-file: 'go.mod'
29+
30+
- name: Install replicated CLI
31+
run: |
32+
REPLICATED_VERSION=$(curl -s https://api.github.com/repos/replicatedhq/replicated/releases/latest | jq -r '.tag_name | ltrimstr("v")')
33+
curl -fsSL "https://github.com/replicatedhq/replicated/releases/download/v${REPLICATED_VERSION}/replicated_${REPLICATED_VERSION}_linux_amd64.tar.gz" | tar -xz -C /tmp
34+
sudo mv /tmp/replicated /usr/local/bin/replicated
35+
replicated version
36+
37+
- name: Setup SSH key
38+
run: |
39+
mkdir -p ~/.ssh
40+
ssh-keygen -t ed25519 -N '' -f ~/.ssh/id_ed25519
41+
cat ~/.ssh/id_ed25519.pub
42+
43+
- name: Build and push image (main)
44+
if: github.event_name != 'workflow_dispatch'
45+
run: |
46+
make docker-image DOCKER_IMAGE=ttl.sh/replicated/ekco:$GITHUB_SHA VERSION=$GITHUB_REF_NAME
47+
docker push ttl.sh/replicated/ekco:$GITHUB_SHA
48+
echo "EKCO_IMAGE=ttl.sh/replicated/ekco:$GITHUB_SHA" >> $GITHUB_ENV
49+
50+
- name: Build and push image (manual)
51+
if: github.event_name == 'workflow_dispatch'
52+
run: |
53+
make docker-image DOCKER_IMAGE=${{ inputs.ekco_image }} VERSION=$GITHUB_REF_NAME
54+
docker push ${{ inputs.ekco_image }}
55+
echo "EKCO_IMAGE=${{ inputs.ekco_image }}" >> $GITHUB_ENV
56+
57+
- name: Check Replicated API token
58+
id: check-token
59+
env:
60+
REPLICATED_API_TOKEN: ${{ secrets.REPLICATED_API_TOKEN }}
61+
INPUT_APP: ${{ inputs.replicated_app }}
62+
run: |
63+
if [ -z "$REPLICATED_API_TOKEN" ]; then
64+
echo "REPLICATED_API_TOKEN secret is not configured. Skipping E2E tests."
65+
echo "skipped=true" >> $GITHUB_OUTPUT
66+
exit 0
67+
fi
68+
69+
test_app() {
70+
local app="$1"
71+
replicated --token "$REPLICATED_API_TOKEN" --app "$app" installer ls --output json > /dev/null 2>&1
72+
}
73+
74+
CANDIDATE_APP="${INPUT_APP:-kurl-sandbox}"
75+
if test_app "$CANDIDATE_APP"; then
76+
echo "Using app: $CANDIDATE_APP"
77+
echo "replicated_app=$CANDIDATE_APP" >> $GITHUB_OUTPUT
78+
echo "skipped=false" >> $GITHUB_OUTPUT
79+
exit 0
80+
fi
81+
82+
echo "Cannot access app '$CANDIDATE_APP'. Searching for an accessible app with installers..."
83+
APP_LIST=$(replicated --token "$REPLICATED_API_TOKEN" app ls --output json 2>&1 || true)
84+
SLUGS=$(echo "$APP_LIST" | jq -r '.[].app.slug' 2>/dev/null || true)
85+
86+
for slug in $SLUGS; do
87+
if test_app "$slug"; then
88+
echo "Found accessible app with installers: $slug"
89+
echo "replicated_app=$slug" >> $GITHUB_OUTPUT
90+
echo "skipped=false" >> $GITHUB_OUTPUT
91+
exit 0
92+
fi
93+
done
94+
95+
echo "No accessible app with installers found. Skipping E2E tests."
96+
echo "skipped=true" >> $GITHUB_OUTPUT
97+
98+
- name: Run E2E tests
99+
if: steps.check-token.outputs.skipped != 'true'
100+
env:
101+
REPLICATED_APP: ${{ steps.check-token.outputs.replicated_app }}
102+
REPLICATED_API_TOKEN: ${{ secrets.REPLICATED_API_TOKEN }}
103+
SSH_KEY: ~/.ssh/id_ed25519
104+
VM_TTL: 2h
105+
run: |
106+
make test-e2e-ci
107+
108+
- name: Upload logs on failure
109+
if: failure() && steps.check-token.outputs.skipped != 'true'
110+
uses: actions/upload-artifact@v4
111+
with:
112+
name: e2e-logs
113+
path: /tmp/e2e-*.log
114+
retention-days: 7

‎.gitignore‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -17,4 +17,6 @@ vendor/
1717
# Additional
1818
*DS_Store
1919
/bin/
20-
.idea
20+
.idea
21+
22+
.envrc

‎AGENTS.md‎

Lines changed: 95 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,95 @@
1+
# EKCO — Agent Guide
2+
3+
Compact, repo-specific guidance for OpenCode sessions.
4+
5+
## What this is
6+
7+
EKCO (Embedded kURL cluster operator) is a single Go binary that runs as a Kubernetes operator to maintain the health of a kURL cluster. It is **not** a library or multi-service repo.
8+
9+
- **Language / toolchain:** Go (module `github.com/replicatedhq/ekco`)
10+
- **Entrypoint:** `cmd/ekco/main.go`
11+
- **CLI framework:** Cobra / Viper (commands live in `cmd/ekco/cli/`)
12+
- **Primary command:** `ekco operator` (long-running operator)
13+
- **Other commands:** `purge-node`, `rotate-certs`, `regen-cert`, `change-load-balancer`, `generate-haproxy-*`, `set-kubeconfig-server` (see `cmd/ekco/cli/`)
14+
15+
## Build, test, lint
16+
17+
Use the Makefile. Do not guess the Go version or linter config.
18+
19+
```bash
20+
# Install tooling (golangci-lint)
21+
make deps
22+
23+
# Full verification — order enforced by Makefile: lint -> vet -> test
24+
make test
25+
26+
# Build binary (outputs bin/ekco)
27+
make build
28+
29+
# Build Docker image
30+
make docker-image
31+
```
32+
33+
**Lint rules are non-default.** `.golangci.yaml` disables `errcheck` and `staticcheck`, and ignores all `*_test.go` files (`exclusions.paths`). Do not re-enable them locally.
34+
35+
**Test scope:** `go test ./pkg/... ./cmd/...` — only `pkg` and `cmd`; there is no root-level test target.
36+
37+
## Docker / deploy quirks
38+
39+
- The Dockerfile is at `deploy/Dockerfile`, not the repo root.
40+
- `ROOK_VERSION` defaults to `1.11.8` in the Makefile and Dockerfile.
41+
- The build stage downloads Helm and pulls `rook-ceph-cluster` chart into `pkg/helm/charts`.
42+
- Git SHA / version are injected via ldflags at build time (`pkg/version`).
43+
44+
## Manual integration testing
45+
46+
Do **not** use `make docker-image && kubectl apply -k deploy/` — the README notes this is broken and out of sync.
47+
48+
Current workflow (from README):
49+
50+
1. `make build-ttl.sh` — pushes a temporary image to `ttl.sh/${USER}/ekco:12h`.
51+
2. Deploy a kURL cluster that includes EKCO.
52+
3. Patch the deployment:
53+
```bash
54+
kubectl edit -n kurl deployment/ekc-operator
55+
# set image to ttl.sh/<user>/ekco:12h and imagePullPolicy: Always
56+
kubectl delete pod -l app=ekc-operator -n kurl
57+
```
58+
59+
## Mocks
60+
61+
Generated with `github.com/golang/mock`:
62+
63+
```bash
64+
make generate-mocks
65+
```
66+
67+
This currently only covers `pkg/k8s/exec.go` → `pkg/k8s/mock/mock_exec.go`.
68+
69+
## Monorepo boundaries
70+
71+
There are none. The repo is a single Go module with two top-level directories:
72+
73+
- `cmd/` — CLI commands and `main.go`
74+
- `pkg/` — operator logic, cluster controller, K8s clients, webhooks, cert rotation, object-store helpers, etc.
75+
- `deploy/` — Dockerfile, Kustomize manifests, and RBAC for the operator
76+
77+
No frontend, no separate API service, no nested Go modules.
78+
79+
## Key external dependencies & constraints
80+
81+
- Deep integration with **Rook Ceph**, **Kubernetes**, **Contour**, **Velero**, **etcd**, and **Prometheus Operator**.
82+
- `go.mod` carries a large set of `replace` and `exclude` blocks inherited from Rook. Do not prune them; they resolve transitive conflicts.
83+
- CI uses `go-version-file: 'go.mod'` so the declared Go version is the source of truth.
84+
85+
## CI / release conventions
86+
87+
- **PRs:** `make deps test build` + `make docker-image` (pushed to `ttl.sh`).
88+
- **Main branch:** `make docker-image` pushed to `replicated/ekco:alpha`.
89+
- **Releases:** Push a semver tag `v*.*.*` (e.g. `git tag -a v0.1.0 -m "Release v0.1.0" && git push origin v0.1.0`). Image is pushed to `replicated/ekco:<tag>`.
90+
- Scheduled vulnerability scans use Grype (`.grype.yaml`) against the repo and the built image.
91+
92+
## When in doubt
93+
94+
- Prefer `Makefile` targets over raw `go` commands — ldflags and build args matter.
95+
- Trust executable configs (`Makefile`, `.golangci.yaml`, `.github/workflows/`) over prose in README for build/test steps.

‎CLAUDE.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
@AGENTS.md

‎Makefile‎

Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -42,6 +42,8 @@ clean:
4242
.PHONY: deps
4343
deps:
4444
curl -sSfL https://raw.githubusercontent.com/golangci/golangci-lint/master/install.sh | sh -s -- -b $(shell go env GOPATH)/bin v2.11.4
45+
curl -sSfL https://raw.githubusercontent.com/anchore/grype/main/install.sh | sh -s -- -b $(shell go env GOPATH)/bin
46+
go install github.com/golang/mock/mockgen@v1.6.0
4547

4648
.PHONY: lint
4749
lint:
@@ -82,7 +84,35 @@ build-ttl.sh: ## Build the EKCO Docker container and deploy it to ttl.sh for use
8284
.
8385
docker push ttl.sh/${CURRENT_USER}/ekco:12h
8486

87+
.PHONY: install-grype
88+
install-grype:
89+
curl -sSfL https://raw.githubusercontent.com/anchore/grype/main/install.sh | sh -s -- -b $(shell go env GOPATH)/bin
90+
91+
.PHONY: grype-repo
92+
grype-repo: ## Scan the repo filesystem for vulnerabilities using Grype
93+
grype dir:.
94+
95+
.PHONY: grype-image
96+
grype-image: ## Scan the Docker image for vulnerabilities using Grype (image must exist locally)
97+
grype $(DOCKER_IMAGE)
98+
99+
.PHONY: docker-scout-image
100+
docker-scout-image: ## Scan the Docker image for vulnerabilities using Docker Scout (image must exist locally)
101+
docker scout cves $(DOCKER_IMAGE)
102+
85103
.PHONY: generate-mocks
86104
generate-mocks: ## Generate mocks tests for CLI and preflight. More info: https://github.com/golang/mock
87105
go install github.com/golang/mock/mockgen@v1.6.0
88106
mockgen -source=pkg/k8s/exec.go -destination=pkg/k8s/mock/mock_exec.go
107+
108+
.PHONY: test-e2e
109+
test-e2e: build-ttl.sh ## Run E2E tests on a kURL cluster in CMX.
110+
EKCO_IMAGE=ttl.sh/$(CURRENT_USER)/ekco:12h ./scripts/e2e-test.sh
111+
112+
.PHONY: test-e2e-ci
113+
test-e2e-ci: ## Run E2E tests in CI using an already-built image.
114+
@if [ -z "$(EKCO_IMAGE)" ]; then \
115+
echo "ERROR: EKCO_IMAGE must be set for CI e2e tests"; \
116+
exit 1; \
117+
fi
118+
./scripts/e2e-test.sh

‎README.md‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -23,6 +23,9 @@ Steps
2323
2. Replace .spec.imagePullPolicy with "Always"
2424
3. **kubectl delete pod -l app=ekc-operator -n kurl** - Delete the pod in the deployment to pull the new image
2525

26+
### Automated testing
27+
The `scripts/e2e-test.sh` script automates the above manual workflow using [Replicated CMX](https://docs.replicated.com/vendor/testing-about). It provisions a CMX VM, installs a kURL cluster, patches the EKCO deployment with a ttl.sh image, and runs health checks. Run `make test-e2e` after `make build-ttl.sh` to use it.
28+
2629

2730
## Release
2831

‎deploy/Dockerfile‎

Lines changed: 7 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -17,16 +17,15 @@ RUN helm repo add rook-release https://charts.rook.io/release
1717
RUN helm repo update
1818
RUN helm pull rook-release/rook-ceph-cluster --version $rook_version && mv rook-ceph-cluster-* pkg/helm/charts
1919

20-
RUN make build GIT_SHA=$git_sha VERSION=$version
20+
RUN CGO_ENABLED=0 make build GIT_SHA=$git_sha VERSION=$version
2121

2222

23-
FROM debian:bookworm-slim
23+
FROM debian:trixie-slim
2424

25-
RUN DEBIAN_FRONTEND=noninteractive apt-get update -qq && apt-get upgrade -qq && apt-get install -y --no-install-recommends \
26-
libgcrypt20 \
27-
libgnutls30 \
28-
liblz4-1 \
29-
&& rm -rf /var/lib/apt/lists/*
25+
# Even though the binary is fully static and we don't install new packages,
26+
# apt-get upgrade patches existing base-image packages so vulnerability scans
27+
# (Grype, Docker Scout) report fewer inherited CVEs.
28+
RUN DEBIAN_FRONTEND=noninteractive apt-get update -qq && apt-get upgrade -qq && rm -rf /var/lib/apt/lists/*
3029

3130
COPY --from=build /go/src/github.com/replicatedhq/ekco/bin/* /usr/bin/
3231

@@ -36,4 +35,4 @@ ARG version=alpha
3635
ENV GIT_SHA=$git_sha
3736
ENV VERSION=$version
3837

39-
ENTRYPOINT /usr/bin/ekco
38+
ENTRYPOINT ["/usr/bin/ekco"]

0 commit comments

Comments
 (0)