Skip to content

🧪 Test suites for Vault and Mondoo Integration #1424

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
slntopp merged 3 commits into from
Mar 13, 2026
Merged

🧪 Test suites for Vault and Mondoo Integration #1424

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
755074d
fix: generate crds
slntopp Mar 13, 2026
87a473c
test-suites for external scanning with Vault and self-scanning with c…
slntopp Mar 13, 2026
997b05a
feat: implement Vault credential refresh logic and update user manual
slntopp Mar 13, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
90 changes: 86 additions & 4 deletions charts/mondoo-operator/crds/k8s.mondoo.com_mondooauditconfigs.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -393,7 +393,7 @@ spec:
description: |-
KubeconfigSecretRef references a Secret containing kubeconfig for the remote cluster.
The Secret must have a key "kubeconfig" with the kubeconfig content.
Mutually exclusive with ServiceAccountAuth and WorkloadIdentity.
Mutually exclusive with ServiceAccountAuth, WorkloadIdentity, SPIFFEAuth, and VaultAuth.
properties:
name:
default: ""
Expand Down Expand Up @@ -436,7 +436,7 @@ spec:
serviceAccountAuth:
description: |-
ServiceAccountAuth configures authentication using a service account token.
Mutually exclusive with KubeconfigSecretRef and WorkloadIdentity.
Mutually exclusive with KubeconfigSecretRef, WorkloadIdentity, SPIFFEAuth, and VaultAuth.
properties:
credentialsSecretRef:
description: |-
Expand Down Expand Up @@ -473,7 +473,7 @@ spec:
spiffeAuth:
description: |-
SPIFFEAuth configures SPIFFE/SPIRE-based authentication using X.509 SVIDs.
Mutually exclusive with KubeconfigSecretRef, ServiceAccountAuth, and WorkloadIdentity.
Mutually exclusive with KubeconfigSecretRef, ServiceAccountAuth, WorkloadIdentity, and VaultAuth.
properties:
audience:
description: |-
Expand Down Expand Up @@ -512,10 +512,92 @@ spec:
- server
- trustBundleSecretRef
type: object
vaultAuth:
description: |-
VaultAuth configures HashiCorp Vault Kubernetes secrets engine for dynamic credential generation.
The operator authenticates to Vault during reconciliation using its own service account token,
fetches short-lived credentials, and writes a kubeconfig Secret that the CronJob mounts.
Mutually exclusive with KubeconfigSecretRef, ServiceAccountAuth, WorkloadIdentity, and SPIFFEAuth.
properties:
authPath:
default: auth/kubernetes
description: AuthPath is the Vault Kubernetes auth method
mount path.
type: string
authRole:
description: AuthRole is the Vault role for authenticating
the pod's service account.
type: string
caCertSecretRef:
description: |-
CACertSecretRef references a Secret containing Vault's CA certificate
for TLS verification. The Secret must have a key "ca.crt".
properties:
name:
default: ""
description: |-
Name of the referent.
This field is effectively required, but due to backwards compatibility is
allowed to be empty. Instances of this type with an empty value here are
almost certainly wrong.
More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names
type: string
type: object
x-kubernetes-map-type: atomic
credsRole:
description: CredsRole is the Vault role for generating
target cluster credentials.
type: string
kubernetesNamespace:
description: KubernetesNamespace is the target namespace
for the generated service account token.
type: string
secretsPath:
default: kubernetes
description: SecretsPath is the Vault Kubernetes secrets
engine mount path.
type: string
server:
description: |-
Server is the URL of the target Kubernetes API server.
Example: "https://target-cluster.example.com:6443"
pattern: ^https://.*
type: string
targetCACertSecretRef:
description: |-
TargetCACertSecretRef references a Secret containing the target cluster's
CA certificate for TLS verification. The Secret must have a key "ca.crt".
properties:
name:
default: ""
description: |-
Name of the referent.
This field is effectively required, but due to backwards compatibility is
allowed to be empty. Instances of this type with an empty value here are
almost certainly wrong.
More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names
type: string
type: object
x-kubernetes-map-type: atomic
ttl:
description: TTL is the requested TTL for the generated
credentials (e.g. "1h", "30m").
type: string
vaultAddr:
description: |-
VaultAddr is the address of the Vault server.
Example: "https://vault.example.com:8200"
type: string
required:
- authRole
- credsRole
- server
- vaultAddr
type: object
workloadIdentity:
description: |-
WorkloadIdentity configures cloud-native Workload Identity Federation authentication.
Mutually exclusive with KubeconfigSecretRef, ServiceAccountAuth, and SPIFFEAuth.
Mutually exclusive with KubeconfigSecretRef, ServiceAccountAuth, SPIFFEAuth, and VaultAuth.
properties:
aks:
description: |-
Expand Down
6 changes: 3 additions & 3 deletions config/crd/bases/k8s.mondoo.com_mondooauditconfigs.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -393,7 +393,7 @@ spec:
description: |-
KubeconfigSecretRef references a Secret containing kubeconfig for the remote cluster.
The Secret must have a key "kubeconfig" with the kubeconfig content.
Mutually exclusive with ServiceAccountAuth and WorkloadIdentity.
Mutually exclusive with ServiceAccountAuth, WorkloadIdentity, SPIFFEAuth, and VaultAuth.
properties:
name:
default: ""
Expand Down Expand Up @@ -436,7 +436,7 @@ spec:
serviceAccountAuth:
description: |-
ServiceAccountAuth configures authentication using a service account token.
Mutually exclusive with KubeconfigSecretRef and WorkloadIdentity.
Mutually exclusive with KubeconfigSecretRef, WorkloadIdentity, SPIFFEAuth, and VaultAuth.
properties:
credentialsSecretRef:
description: |-
Expand Down Expand Up @@ -597,7 +597,7 @@ spec:
workloadIdentity:
description: |-
WorkloadIdentity configures cloud-native Workload Identity Federation authentication.
Mutually exclusive with KubeconfigSecretRef, ServiceAccountAuth, and SPIFFEAuth.
Mutually exclusive with KubeconfigSecretRef, ServiceAccountAuth, SPIFFEAuth, and VaultAuth.
properties:
aks:
description: |-
Expand Down
16 changes: 15 additions & 1 deletion controllers/k8s_scan/deployment_handler.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -178,7 +178,21 @@ func (n *DeploymentHandler) Reconcile(ctx context.Context) (ctrl.Result, error)
}
}

return ctrl.Result{}, nil
// If any external cluster uses VaultAuth, request an earlier requeue so the
// operator refreshes Vault credentials before they expire. Default to half
// the requested TTL, with a floor of 10 minutes and a ceiling of 1 hour.
result := ctrl.Result{}
for i := range n.Mondoo.Spec.KubernetesResources.ExternalClusters {
cluster := &n.Mondoo.Spec.KubernetesResources.ExternalClusters[i]
if cluster.VaultAuth != nil {
requeue := vaultRequeueInterval(cluster.VaultAuth.TTL)
if result.RequeueAfter == 0 || requeue < result.RequeueAfter {
result.RequeueAfter = requeue
}
}
}

return result, nil
}

func (n *DeploymentHandler) syncCronJob(ctx context.Context) error {
Expand Down
6 changes: 3 additions & 3 deletions controllers/k8s_scan/deployment_handler_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -500,7 +500,7 @@ func (s *DeploymentHandlerSuite) TestReconcile_ExternalCluster_VaultAuth() {

result, err := d.Reconcile(s.ctx)
s.NoError(err)
s.True(result.IsZero())
s.Equal(30*time.Minute, result.RequeueAfter, "should requeue at default interval when no TTL is set")

// Verify kubeconfig Secret was created
kubeconfigSecret := &corev1.Secret{}
Expand Down Expand Up @@ -599,7 +599,7 @@ func (s *DeploymentHandlerSuite) TestReconcile_ExternalCluster_VaultAuth_WithCAC

result, err := d.Reconcile(s.ctx)
s.NoError(err)
s.True(result.IsZero())
s.Equal(30*time.Minute, result.RequeueAfter, "should requeue at default interval when no TTL is set")

// Verify mock received the Vault CA cert bytes
s.Equal([]byte("-----BEGIN CERTIFICATE-----\nvault-ca\n-----END CERTIFICATE-----"), receivedVaultCA)
Expand Down Expand Up @@ -758,7 +758,7 @@ func (s *DeploymentHandlerSuite) createDeploymentHandlerWithVaultMock(vaultFetch
MondooOperatorConfig: &mondoov1alpha2.MondooOperatorConfig{},
MondooClientBuilder: mondooclient.NewClient,
VaultTokenFetcher: vaultFetcher,
SATokenPath: tmpFile.Name(),
SATokenPath: tmpName,
}
}

Expand Down
32 changes: 31 additions & 1 deletion controllers/k8s_scan/vault.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ import (
"encoding/base64"
"fmt"
"strings"
"time"

vault "github.com/hashicorp/vault-client-go"
"github.com/hashicorp/vault-client-go/schema"
Expand Down Expand Up @@ -93,6 +94,35 @@ func DefaultVaultTokenFetcher(ctx context.Context, saToken string, config v1alph
return token, nil
}

// vaultRequeueInterval calculates a reconcile requeue interval based on the Vault TTL.
// It targets half the TTL so credentials are refreshed well before expiry, with a
// floor of 10 minutes and a ceiling of 1 hour.
func vaultRequeueInterval(ttl string) time.Duration {
const (
minInterval = 10 * time.Minute
maxInterval = 1 * time.Hour
defaultInterval = 30 * time.Minute
)

if ttl == "" {
return defaultInterval
}

d, err := time.ParseDuration(ttl)
if err != nil {
return defaultInterval
}

half := d / 2
if half < minInterval {
return minInterval
}
if half > maxInterval {
return maxInterval
}
return half
}

// VaultKubeconfigSecretName returns the name for the Vault kubeconfig Secret.
func VaultKubeconfigSecretName(prefix, clusterName string) string {
return fmt.Sprintf("%s-vault-kubeconfig-%s", prefix, clusterName)
Expand Down Expand Up @@ -126,7 +156,7 @@ current-context: default
users:
- name: vault
user:
token: %s
token: "%s"
Copy link
Copy Markdown

@mondoo-code-review mondoo-code-review bot Mar 13, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 warning — Quoting the token with "%s" will embed literal double quotes into the YAML value. If the Vault-issued token ever contains a double quote or backslash, the kubeconfig will be malformed YAML. Consider using yaml.Marshal or at minimum escaping special characters within the token string. That said, Vault SA tokens are typically base64-encoded JWTs so this is unlikely to bite in practice.

`, clusterConfig, token)
}

Expand Down
2 changes: 1 addition & 1 deletion docs/user-manual.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -453,7 +453,7 @@ externalClusters:

> **Note: Credential refresh timing**
>
> Vault credentials are refreshed each time the operator reconciles (typically every 7 days, or when configuration changes). Ensure the requested TTL is longer than the operator's reconcile interval. If you need more frequent credential rotation, consider adjusting the operator's reconcile schedule.
> When VaultAuth is configured, the operator automatically reconciles at half the requested TTL (clamped between 10 minutes and 1 hour) to refresh credentials before they expire. For example, a `ttl: "1h"` results in a 30-minute refresh interval. If no TTL is specified, the operator defaults to refreshing every 30 minutes.

## Container Image Scanning

Expand Down
59 changes: 56 additions & 3 deletions tests/e2e/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,8 @@ End-to-end tests that deploy the Mondoo operator to a real GKE cluster and verif

- **Fresh Deploy** (`run-fresh-deploy.sh`): Builds the operator from the current branch, deploys to a GKE cluster, configures scanning, and verifies everything works.
- **Upgrade** (`run-upgrade.sh`): Installs a released baseline version first, verifies it, then upgrades to the current branch and verifies again.
- **External Cluster** (`run-external-cluster.sh`): Deploys the operator and configures external cluster scanning against a target GKE cluster using a static kubeconfig Secret.
- **Vault External Cluster** (`run-vault-external-cluster.sh`): Like External Cluster, but uses HashiCorp Vault's Kubernetes secrets engine to dynamically generate short-lived service account tokens instead of a static kubeconfig.
- **Registry Mirroring & Proxy** (`run-registry-mirroring.sh`): Deploys with an Artifact Registry mirror repo and optional Squid proxy. Verifies image references are rewritten, `imagePullSecrets` are propagated, and proxy env vars are set.

All tests pause for manual verification at each verify step (check Mondoo console for assets/scan results). Press Enter to continue or Ctrl+C to abort.
Expand Down Expand Up @@ -61,6 +63,7 @@ terraform apply \
| `region` | no | `europe-west3` | GCP region |
| `autopilot` | no | `true` | `true` for Autopilot, `false` for Standard cluster |
| `enable_mirror_test` | no | `false` | Create a mirror AR repo for registry mirroring/imagePullSecrets tests |
| `enable_target_cluster` | no | `false` | Create a second GKE cluster for external cluster / Vault scanning tests |
| `enable_proxy_test` | no | `false` | Provision a Squid proxy VM for proxy tests (requires `enable_mirror_test`) |

You can also set these in a `terraform.tfvars` file.
Expand Down Expand Up @@ -99,6 +102,45 @@ What it does:
6. Upgrades to the current branch image via local Helm chart
7. Waits, verifies again, pauses for manual check

### External Cluster (Static Kubeconfig)

```bash
# Provision infrastructure with a target cluster
cd tests/e2e/terraform
terraform apply -var="project_id=MY_PROJECT" -var="mondoo_org_id=MY_ORG" \
-var="enable_target_cluster=true"

# Run the test
cd tests/e2e
./run-external-cluster.sh
```

### Vault External Cluster

```bash
# Provision infrastructure with a target cluster
cd tests/e2e/terraform
terraform apply -var="project_id=MY_PROJECT" -var="mondoo_org_id=MY_ORG" \
-var="enable_target_cluster=true"

# Run the test
cd tests/e2e
./run-vault-external-cluster.sh
```

What it does:
1. Loads Terraform outputs (requires `enable_target_cluster=true`)
2. Builds the operator image and pushes to Artifact Registry
3. Deploys nginx test workloads to both clusters
4. Installs the operator via local Helm chart
5. Deploys HashiCorp Vault in dev mode to the scanner cluster
6. Configures Vault: Kubernetes auth (scanner cluster) + Kubernetes secrets engine (target cluster)
7. Creates target cluster CA cert Secret for TLS verification
8. Applies MondooAuditConfig with `vaultAuth` external cluster config
9. Waits 90s for operator to reconcile and fetch Vault credentials
10. Verifies: vault-kubeconfig Secret created, CronJob has no init containers, correct volume mounts
11. Pauses for manual verification in the Mondoo console

### Registry Mirroring & Proxy

```bash
Expand Down Expand Up @@ -148,6 +190,8 @@ tests/e2e/
├── README.md
├── run-fresh-deploy.sh # Fresh deploy test orchestrator
├── run-upgrade.sh # Upgrade test orchestrator
├── run-external-cluster.sh # External cluster scanning test orchestrator
├── run-vault-external-cluster.sh # Vault external cluster test orchestrator
├── run-registry-mirroring.sh # Registry mirroring & proxy test orchestrator
├── terraform/
│ ├── versions.tf # Provider requirements
Expand All @@ -163,16 +207,25 @@ tests/e2e/
│ ├── deploy-operator-mirroring.sh # Helm install with mirror/proxy values
│ ├── deploy-baseline.sh # Helm install released version
│ ├── deploy-test-workload.sh # Deploy nginx for scanning
│ ├── deploy-target-workload.sh # Deploy nginx + kubeconfig Secret for external cluster
│ ├── deploy-target-workload-only.sh # Deploy nginx to target (no kubeconfig Secret)
│ ├── deploy-vault.sh # Deploy + configure Vault for external cluster auth
│ ├── apply-mondoo-config.sh # Create secret + apply MondooAuditConfig
│ ├── setup-mirror-registry.sh # Create imagePullSecret for mirror AR repo
│ ├── populate-mirror-registry.sh # Copy cnspec image into mirror AR repo via crane
│ ├── verify.sh # Automated checks + manual verification pause
│ ├── verify-external.sh # External cluster verification
│ ├── verify-vault-external.sh # Vault external cluster verification
│ ├── verify-mirroring.sh # Mirroring/proxy-specific verification
│ └── cleanup.sh # Remove all test resources from cluster
└── manifests/
├── mondoo-audit-config.yaml.tpl # Standard cluster config (nodes enabled)
├── mondoo-audit-config-autopilot.yaml.tpl # Autopilot config (nodes disabled)
└── nginx-workload.yaml # Test workload
├── mondoo-audit-config.yaml.tpl # Standard config (nodes enabled)
├── mondoo-audit-config-autopilot.yaml.tpl # Autopilot config (nodes disabled)
├── mondoo-audit-config-external.yaml.tpl # External cluster (static kubeconfig)
├── mondoo-audit-config-external-autopilot.yaml.tpl # External cluster + Autopilot
├── mondoo-audit-config-vault-external.yaml.tpl # Vault external cluster
├── mondoo-audit-config-vault-external-autopilot.yaml.tpl # Vault external + Autopilot
└── nginx-workload.yaml # Test workload
```

## Notes
Expand Down
Loading
Loading