🧪 test-suite for registry mirroring and proxying

Author: slntopp

tests/e2e/README.md

@@ -6,8 +6,9 @@ End-to-end tests that deploy the Mondoo operator to a real GKE cluster and verif
 
 - **Fresh Deploy** (`run-fresh-deploy.sh`): Builds the operator from the current branch, deploys to a GKE cluster, configures scanning, and verifies everything works.
 - **Upgrade** (`run-upgrade.sh`): Installs a released baseline version first, verifies it, then upgrades to the current branch and verifies again.
+- **Registry Mirroring & Proxy** (`run-registry-mirroring.sh`): Deploys with an Artifact Registry mirror repo and optional Squid proxy. Verifies image references are rewritten, `imagePullSecrets` are propagated, and proxy env vars are set.
 
-Both tests pause for manual verification at each verify step (check Mondoo console for assets/scan results). Press Enter to continue or Ctrl+C to abort.
+All tests pause for manual verification at each verify step (check Mondoo console for assets/scan results). Press Enter to continue or Ctrl+C to abort.
 
 ## Prerequisites
 
@@ -17,6 +18,8 @@ Both tests pause for manual verification at each verify step (check Mondoo conso
 - `docker`
 - `kubectl`
 - Go toolchain (for building the operator)
+- `jq` (for registry mirroring test verification)
+- `crane` CLI (for registry mirroring test): `go install github.com/google/go-containerregistry/cmd/crane@latest`
 
 ### Mondoo credentials
 
@@ -57,6 +60,8 @@ terraform apply \
 | `mondoo_org_id` | yes | - | Mondoo organization ID |
 | `region` | no | `europe-west3` | GCP region |
 | `autopilot` | no | `true` | `true` for Autopilot, `false` for Standard cluster |
+| `enable_mirror_test` | no | `false` | Create a mirror AR repo for registry mirroring/imagePullSecrets tests |
+| `enable_proxy_test` | no | `false` | Provision a Squid proxy VM for proxy tests (requires `enable_mirror_test`) |
 
 You can also set these in a `terraform.tfvars` file.
 
@@ -94,6 +99,32 @@ What it does:
 6. Upgrades to the current branch image via local Helm chart
 7. Waits, verifies again, pauses for manual check
 
+### Registry Mirroring & Proxy
+
+```bash
+# Provision infrastructure with mirror AR repo (and optional proxy VM)
+cd tests/e2e/terraform
+terraform apply -var="project_id=MY_PROJECT" -var="mondoo_org_id=MY_ORG" \
+  -var="enable_mirror_test=true" -var="enable_proxy_test=true"
+
+# Run the test
+cd tests/e2e
+./run-registry-mirroring.sh
+```
+
+What it does:
+1. Loads Terraform outputs (including mirror AR repo and proxy IP if enabled)
+2. Builds the operator image and pushes to Artifact Registry
+3. Creates `imagePullSecret` from the mirror repo's GCP service account key
+4. Uses `crane` to copy cnspec image from ghcr.io into the mirror AR repo
+5. Deploys an nginx test workload
+6. Helm installs the operator with `registryMirrors`, `imagePullSecrets`, and proxy `--values`
+7. Applies MondooAuditConfig, waits 90s for reconciliation
+8. Runs standard verification checks
+9. Runs mirroring-specific checks: image refs rewritten, pull secrets propagated, proxy env vars set
+10. Checks Squid proxy access logs (if proxy enabled)
+11. Pauses for manual verification in the Mondoo console
+
 ## Cleanup
 
 Remove all test resources from the cluster (everything except Terraform infra):
@@ -115,23 +146,29 @@ terraform destroy -var="project_id=MY_PROJECT" -var="mondoo_org_id=MY_ORG"
 ```
 tests/e2e/
 ├── README.md
-├── run-fresh-deploy.sh          # Fresh deploy test orchestrator
-├── run-upgrade.sh               # Upgrade test orchestrator
+├── run-fresh-deploy.sh              # Fresh deploy test orchestrator
+├── run-upgrade.sh                   # Upgrade test orchestrator
+├── run-registry-mirroring.sh        # Registry mirroring & proxy test orchestrator
 ├── terraform/
-│   ├── versions.tf              # Provider requirements
-│   ├── variables.tf             # Input variables
-│   ├── main.tf                  # GKE cluster + Artifact Registry
-│   ├── mondoo.tf                # Mondoo space + service account
-│   └── outputs.tf               # Outputs consumed by scripts
+│   ├── versions.tf                  # Provider requirements
+│   ├── variables.tf                 # Input variables
+│   ├── main.tf                      # GKE cluster + Artifact Registry
+│   ├── mondoo.tf                    # Mondoo space + service account
+│   ├── proxy.tf                     # Squid proxy VM (optional)
+│   └── outputs.tf                   # Outputs consumed by scripts
 ├── scripts/
-│   ├── common.sh                # Logging, TF output loading, wait helpers
-│   ├── build-and-push.sh        # Build operator image, push to AR
-│   ├── deploy-operator.sh       # Helm install from local chart
-│   ├── deploy-baseline.sh       # Helm install released version
-│   ├── deploy-test-workload.sh  # Deploy nginx for scanning
-│   ├── apply-mondoo-config.sh   # Create secret + apply MondooAuditConfig
-│   ├── verify.sh                # Automated checks + manual verification pause
-│   └── cleanup.sh               # Remove all test resources from cluster
+│   ├── common.sh                    # Logging, TF output loading, wait helpers
+│   ├── build-and-push.sh            # Build operator image, push to AR
+│   ├── deploy-operator.sh           # Helm install from local chart
+│   ├── deploy-operator-mirroring.sh # Helm install with mirror/proxy values
+│   ├── deploy-baseline.sh           # Helm install released version
+│   ├── deploy-test-workload.sh      # Deploy nginx for scanning
+│   ├── apply-mondoo-config.sh       # Create secret + apply MondooAuditConfig
+│   ├── setup-mirror-registry.sh     # Create imagePullSecret for mirror AR repo
+│   ├── populate-mirror-registry.sh  # Copy cnspec image into mirror AR repo via crane
+│   ├── verify.sh                    # Automated checks + manual verification pause
+│   ├── verify-mirroring.sh          # Mirroring/proxy-specific verification
+│   └── cleanup.sh                   # Remove all test resources from cluster
 └── manifests/
     ├── mondoo-audit-config.yaml.tpl            # Standard cluster config (nodes enabled)
     ├── mondoo-audit-config-autopilot.yaml.tpl  # Autopilot config (nodes disabled)

tests/e2e/run-registry-mirroring.sh

@@ -0,0 +1,81 @@
+#!/usr/bin/env bash
+# Copyright Mondoo, Inc. 2026
+# SPDX-License-Identifier: BUSL-1.1
+
+# Test: Registry Mirroring, imagePullSecrets & Proxy
+# Builds operator, deploys with AR-based mirror registry and optional proxy,
+# verifies image references, pull secrets, and proxy configuration.
+#
+# Prerequisites:
+#   - Terraform infrastructure provisioned with enable_mirror_test=true
+#   - crane CLI installed (go install github.com/google/go-containerregistry/cmd/crane@latest)
+#   - gcloud authenticated, docker, helm, kubectl available
+#   - For proxy testing: also set enable_proxy_test=true in terraform
+#
+# Usage:
+#   ./run-registry-mirroring.sh
+
+set -euo pipefail
+
+E2E_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+source "${E2E_ROOT}/scripts/common.sh"
+
+info "=========================================="
+info "  Test: Registry Mirroring & Proxy"
+info "=========================================="
+
+# Step 1: Load Terraform outputs (includes mirror and proxy outputs)
+load_tf_outputs
+
+# Validate that mirror test infra is provisioned
+if [[ "${ENABLE_MIRROR_TEST}" != "true" ]]; then
+  die "This test requires enable_mirror_test=true in Terraform. Run: terraform apply -var='enable_mirror_test=true'"
+fi
+
+# Step 2: Build and push operator image
+info "--- Step: Build and Push ---"
+source "${E2E_ROOT}/scripts/build-and-push.sh"
+
+# Step 3: Create imagePullSecret for mirror AR repo
+info "--- Step: Setup Mirror Registry Credentials ---"
+source "${E2E_ROOT}/scripts/setup-mirror-registry.sh"
+
+# Step 4: Populate mirror AR repo with cnspec image
+info "--- Step: Populate Mirror Registry ---"
+source "${E2E_ROOT}/scripts/populate-mirror-registry.sh"
+
+# Step 5: Deploy test workload
+info "--- Step: Deploy Test Workload ---"
+source "${E2E_ROOT}/scripts/deploy-test-workload.sh"
+
+# Step 6: Set proxy env vars from Terraform if enabled
+if [[ "${ENABLE_PROXY_TEST}" == "true" ]]; then
+  info "Proxy testing enabled — Squid proxy at ${SQUID_PROXY_IP}"
+else
+  info "Proxy testing disabled — skipping proxy configuration"
+fi
+
+# Step 7: Deploy operator with mirroring/proxy configuration
+info "--- Step: Deploy Operator with Mirroring ---"
+source "${E2E_ROOT}/scripts/deploy-operator-mirroring.sh"
+
+# Step 8: Apply MondooAuditConfig
+info "--- Step: Apply Mondoo Config ---"
+source "${E2E_ROOT}/scripts/apply-mondoo-config.sh"
+
+# Step 9: Wait for operator to reconcile
+info "Waiting 90s for operator to reconcile with mirrored images..."
+sleep 90
+
+# Step 10: Standard verification
+info "--- Step: Standard Verify ---"
+source "${E2E_ROOT}/scripts/verify.sh"
+
+# Step 11: Mirroring-specific verification
+info "--- Step: Verify Mirroring & Proxy ---"
+source "${E2E_ROOT}/scripts/verify-mirroring.sh"
+
+info ""
+info "=========================================="
+info "  Test: Registry Mirroring & Proxy - COMPLETE"
+info "=========================================="

tests/e2e/scripts/cleanup.sh

@@ -66,4 +66,8 @@ fi
 info "Removing mondoo Helm repo..."
 helm repo remove mondoo 2>/dev/null || true
 
+# Mirror registry resources (from registry mirroring tests)
+info "Deleting mirror-registry-creds secret..."
+kubectl delete secret mirror-registry-creds -n "${NAMESPACE}" --ignore-not-found
+
 info "=== Cleanup complete ==="

tests/e2e/scripts/common.sh

@@ -43,6 +43,17 @@ load_tf_outputs() {
     export TARGET_KUBECONFIG_PATH="$(cd "${TF_DIR}" && realpath "$(terraform output -raw target_kubeconfig_path)")"
   fi
 
+  export ENABLE_MIRROR_TEST="$(terraform output -raw enable_mirror_test 2>/dev/null || echo "false")"
+  if [[ "${ENABLE_MIRROR_TEST}" == "true" ]]; then
+    export MIRROR_REGISTRY="$(terraform output -raw mirror_registry_repo)"
+    export MIRROR_SA_KEY_B64="$(terraform output -raw mirror_sa_key_b64)"
+  fi
+
+  export ENABLE_PROXY_TEST="$(terraform output -raw enable_proxy_test 2>/dev/null || echo "false")"
+  if [[ "${ENABLE_PROXY_TEST}" == "true" ]]; then
+    export SQUID_PROXY_IP="$(terraform output -raw squid_proxy_ip)"
+  fi
+
   cd ->/dev/null
 
   # Use gcloud to get cluster credentials (auto-refreshing auth)

tests/e2e/scripts/deploy-operator-mirroring.sh

@@ -0,0 +1,83 @@
+#!/usr/bin/env bash
+# Copyright Mondoo, Inc. 2026
+# SPDX-License-Identifier: BUSL-1.1
+
+# Deploy the operator from the local Helm chart with mirror/proxy configuration
+# Uses a values-override file to avoid Helm --set escaping issues with dots in keys
+
+set -euo pipefail
+SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+source "${SCRIPT_DIR}/common.sh"
+
+: "${IMAGE_REPO:?IMAGE_REPO must be set}"
+: "${IMAGE_TAG:?IMAGE_TAG must be set}"
+: "${NAMESPACE:?NAMESPACE must be set}"
+: "${MIRROR_REGISTRY:?MIRROR_REGISTRY must be set}"
+
+# Proxy settings (optional)
+SQUID_PROXY_IP="${SQUID_PROXY_IP:-}"
+
+info "Deploying operator with mirroring configuration..."
+info "  Mirror registry: ${MIRROR_REGISTRY}"
+if [[ -n "${SQUID_PROXY_IP}" ]]; then
+  info "  Proxy: http://${SQUID_PROXY_IP}:3128"
+fi
+
+# Build proxy values
+PROXY_HTTP=""
+PROXY_HTTPS=""
+PROXY_NO=""
+PROXY_CONTAINER=""
+if [[ -n "${SQUID_PROXY_IP}" ]]; then
+  PROXY_HTTP="http://${SQUID_PROXY_IP}:3128"
+  PROXY_HTTPS="http://${SQUID_PROXY_IP}:3128"
+  PROXY_CONTAINER="http://${SQUID_PROXY_IP}:3128"
+
+  # Get both the external and in-cluster Kubernetes API server IPs.
+  # External: from kubeconfig (used by kubectl from outside)
+  # Internal: the kubernetes.default ClusterIP (used by pods via KUBERNETES_SERVICE_HOST)
+  K8S_API_EXTERNAL=$(kubectl config view --minify -o jsonpath='{.clusters[0].cluster.server}' | sed -E 's|https?://||;s|:[0-9]+$||')
+  K8S_API_INTERNAL=$(kubectl get svc kubernetes -n default -o jsonpath='{.spec.clusterIP}')
+  info "Kubernetes API server IPs: external=${K8S_API_EXTERNAL}, internal=${K8S_API_INTERNAL}"
+  PROXY_NO="10.0.0.0/8,172.16.0.0/12,.cluster.local,.svc,localhost,127.0.0.1,${K8S_API_EXTERNAL},${K8S_API_INTERNAL}"
+fi
+
+# Generate values override file
+# Using a file avoids Helm --set escaping issues with dots in registry mirror keys
+VALUES_FILE=$(mktemp /tmp/mondoo-mirror-values-XXXXXX.yaml)
+trap "rm -f ${VALUES_FILE}" EXIT
+
+cat > "${VALUES_FILE}" <<EOF
+operator:
+  skipContainerResolution: true
+  registryMirrors:
+    ghcr.io: "${MIRROR_REGISTRY}"
+  imagePullSecrets:
+    - name: mirror-registry-creds
+  httpProxy: "${PROXY_HTTP}"
+  httpsProxy: "${PROXY_HTTPS}"
+  noProxy: "${PROXY_NO}"
+  containerProxy: "${PROXY_CONTAINER}"
+EOF
+
+info "Generated values override:"
+cat "${VALUES_FILE}"
+
+# Adopt any existing Mondoo CRDs so Helm can manage them
+for crd in $(kubectl get crds -o name 2>/dev/null | grep mondoo || true); do
+  info "Adopting existing CRD for Helm: ${crd}"
+  kubectl label "${crd}" app.kubernetes.io/managed-by=Helm --overwrite
+  kubectl annotate "${crd}" meta.helm.sh/release-name=mondoo-operator meta.helm.sh/release-namespace="${NAMESPACE}" --overwrite
+done
+
+helm upgrade --install mondoo-operator "${REPO_ROOT}/charts/mondoo-operator" \
+  --namespace "${NAMESPACE}" --create-namespace \
+  --set controllerManager.manager.image.repository="${IMAGE_REPO}" \
+  --set controllerManager.manager.image.tag="${IMAGE_TAG}" \
+  --set controllerManager.manager.imagePullPolicy=Always \
+  --values "${VALUES_FILE}" \
+  --wait --timeout 5m
+
+wait_for_deployment "${NAMESPACE}" "mondoo-operator-controller-manager"
+
+info "Operator deployed with mirroring configuration."

tests/e2e/scripts/populate-mirror-registry.sh

@@ -0,0 +1,44 @@
+#!/usr/bin/env bash
+# Copyright Mondoo, Inc. 2026
+# SPDX-License-Identifier: BUSL-1.1
+
+# Copy cnspec image from ghcr.io into the mirror Artifact Registry repo.
+# Uses crane locally with gcloud auth (caller is already authenticated to GCP).
+#
+# The operator image is pulled directly from the main AR repo (set via Helm
+# --set values) and does NOT go through the mirror. Only images the operator
+# spawns (cnspec) get resolved through registryMirrors.
+
+set -euo pipefail
+SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+source "${SCRIPT_DIR}/common.sh"
+
+: "${MIRROR_REGISTRY:?MIRROR_REGISTRY must be set}"
+: "${REGION:?REGION must be set}"
+
+# cnspec image constants (must match pkg/utils/mondoo/container_image_resolver.go)
+CNSPEC_IMAGE="ghcr.io/mondoohq/mondoo-operator/cnspec"
+CNSPEC_TAG="12-rootless"
+
+CNSPEC_SRC="${CNSPEC_IMAGE}:${CNSPEC_TAG}"
+# Mirror path preserves the ghcr.io path structure so registryMirrors mapping works:
+#   ghcr.io/mondoohq/mondoo-operator/cnspec:tag -> MIRROR_REGISTRY/mondoohq/mondoo-operator/cnspec:tag
+CNSPEC_MIRROR="${MIRROR_REGISTRY}/mondoohq/mondoo-operator/cnspec:${CNSPEC_TAG}"
+
+info "Populating mirror registry..."
+info "  cnspec: ${CNSPEC_SRC} -> ${CNSPEC_MIRROR}"
+
+# Verify crane is available
+if ! command -v crane &>/dev/null; then
+  die "crane CLI is required. Install with: go install github.com/google/go-containerregistry/cmd/crane@latest"
+fi
+
+# Configure crane auth for the mirror AR repo (uses gcloud credentials)
+info "Configuring crane auth for Artifact Registry..."
+crane auth login "${REGION}-docker.pkg.dev" -u oauth2accesstoken -p "$(gcloud auth print-access-token)"
+
+# Copy cnspec image from ghcr.io (public) to mirror AR repo
+info "Copying cnspec image to mirror..."
+crane copy "${CNSPEC_SRC}" "${CNSPEC_MIRROR}"
+
+info "Mirror registry populated successfully."

tests/e2e/scripts/setup-mirror-registry.sh

@@ -0,0 +1,35 @@
+#!/usr/bin/env bash
+# Copyright Mondoo, Inc. 2026
+# SPDX-License-Identifier: BUSL-1.1
+
+# Create the imagePullSecret for the mirror Artifact Registry repo.
+# The mirror AR repo and a read-only service account are provisioned by Terraform.
+
+set -euo pipefail
+SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+source "${SCRIPT_DIR}/common.sh"
+
+: "${NAMESPACE:?NAMESPACE must be set}"
+: "${MIRROR_REGISTRY:?MIRROR_REGISTRY must be set}"
+: "${MIRROR_SA_KEY_B64:?MIRROR_SA_KEY_B64 must be set}"
+: "${REGION:?REGION must be set}"
+
+info "Setting up mirror registry credentials..."
+
+# The mirror AR repo URL is REGION-docker.pkg.dev, extract the server
+MIRROR_SERVER="${REGION}-docker.pkg.dev"
+
+# Decode the GCP SA key (Terraform outputs it as base64)
+SA_KEY_JSON=$(echo "${MIRROR_SA_KEY_B64}" | base64 -d)
+
+# Create docker-registry secret in operator namespace for imagePullSecrets
+info "Creating mirror-registry-creds secret in ${NAMESPACE}..."
+kubectl create namespace "${NAMESPACE}" --dry-run=client -o yaml | kubectl apply -f -
+kubectl create secret docker-registry mirror-registry-creds \
+  --namespace "${NAMESPACE}" \
+  --docker-server="${MIRROR_SERVER}" \
+  --docker-username="_json_key" \
+  --docker-password="${SA_KEY_JSON}" \
+  --dry-run=client -o yaml | kubectl apply -f -
+
+info "Mirror registry credentials created for ${MIRROR_SERVER}"