Skip to content

Commit 69e9e64

Browse files
chris-rockclaude
andcommitted
docs: simplify WIF testing documentation
Remove redundant local testing section that duplicates unit test coverage and add key implementation details to cloud provider sections. Changes: - Replace verbose local testing section with pointer to unit tests - Add key details (annotations, images, commands) to GKE/EKS/AKS sections - Fix init container name references (wif-init -> generate-kubeconfig) - Add WIF Testing Summary table - Rename test methods: AKS (8), EKS (7), GKE (6) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
1 parent 55c3305 commit 69e9e64

1 file changed

Lines changed: 42 additions & 247 deletions

File tree

docs/development.md

Lines changed: 42 additions & 247 deletions
Original file line numberDiff line numberDiff line change
@@ -548,256 +548,20 @@ This section describes how to test WIF for external cluster scanning. WIF suppor
548548
- **EKS** (Elastic Kubernetes Service) - uses AWS IAM Roles for Service Accounts (IRSA)
549549
- **AKS** (Azure Kubernetes Service) - uses Azure Workload Identity
550550

551-
### Local Testing Without Cloud Infrastructure
551+
### Local Testing
552552

553-
You can test most WIF functionality locally without setting up cloud infrastructure. This is useful for:
554-
- Verifying the operator creates correct resources (ServiceAccounts, CronJobs, init containers)
555-
- Testing validation logic
556-
- Verifying configuration handling
557-
558-
**What CAN be tested locally:**
559-
- Unit tests (ServiceAccount annotations, init container configuration, validation)
560-
- Resource creation (verify the operator creates the right Kubernetes resources)
561-
- Configuration validation (ensure invalid configs are rejected)
562-
- Init container structure (correct images, environment variables, volume mounts)
563-
564-
**What CANNOT be tested locally:**
565-
- Actual authentication to cloud providers (requires real GKE/EKS/AKS clusters with WIF enabled)
566-
- End-to-end scanning of external clusters via WIF
567-
568-
#### Running WIF Unit Tests
569-
570-
The unit tests comprehensively cover WIF functionality:
553+
WIF resource creation is covered by unit tests:
571554

572555
```bash
573-
# Run all unit tests
574-
make test
575-
576-
# Run only WIF-related tests
556+
# Run WIF-related unit tests
577557
go test -v ./controllers/k8s_scan/... -run "WIF|WorkloadIdentity"
578558
```
579559

580-
The tests cover:
581-
- `TestWIFServiceAccount` - Verifies ServiceAccount creation with correct annotations for GKE, EKS, and AKS
560+
These tests verify ServiceAccount annotations, init container configuration, and validation logic for all three providers (GKE, EKS, AKS). The tests include:
561+
- `TestWIFServiceAccount` - Verifies ServiceAccount creation with correct annotations
582562
- `TestWIFInitContainer` - Verifies init container configuration (images, env vars, volume mounts)
583563
- `TestValidateExternalClusterAuth` - Verifies validation rejects invalid configurations
584564

585-
#### Local Resource Verification Testing
586-
587-
You can verify the operator creates correct WIF resources using k3d without any cloud connectivity:
588-
589-
```bash
590-
# Create a local k3d cluster
591-
k3d cluster create wif-test
592-
593-
# Install CRDs and create namespace
594-
make install
595-
kubectl create namespace mondoo-operator
596-
597-
# Create RBAC
598-
cat <<EOF | kubectl apply -f -
599-
apiVersion: v1
600-
kind: ServiceAccount
601-
metadata:
602-
name: mondoo-operator-k8s-resources-scanning
603-
namespace: mondoo-operator
604-
---
605-
apiVersion: rbac.authorization.k8s.io/v1
606-
kind: ClusterRole
607-
metadata:
608-
name: mondoo-operator-k8s-resources-scanning
609-
rules:
610-
- apiGroups: ["*"]
611-
resources: ["*"]
612-
verbs: ["get", "watch", "list"]
613-
---
614-
apiVersion: rbac.authorization.k8s.io/v1
615-
kind: ClusterRoleBinding
616-
metadata:
617-
name: mondoo-operator-k8s-resources-scanning
618-
roleRef:
619-
apiGroup: rbac.authorization.k8s.io
620-
kind: ClusterRole
621-
name: mondoo-operator-k8s-resources-scanning
622-
subjects:
623-
- kind: ServiceAccount
624-
name: mondoo-operator-k8s-resources-scanning
625-
namespace: mondoo-operator
626-
EOF
627-
628-
# Create a dummy Mondoo credentials secret (required by operator)
629-
kubectl create secret generic mondoo-client \
630-
--namespace mondoo-operator \
631-
--from-literal=config='{"mrn":"//example","private_key":"test"}'
632-
```
633-
634-
**Test GKE WIF Resource Creation:**
635-
636-
```bash
637-
# Apply a GKE WIF configuration
638-
cat <<EOF | kubectl apply -f -
639-
apiVersion: k8s.mondoo.com/v1alpha2
640-
kind: MondooAuditConfig
641-
metadata:
642-
name: mondoo-client
643-
namespace: mondoo-operator
644-
spec:
645-
mondooCredsSecretRef:
646-
name: mondoo-client
647-
kubernetesResources:
648-
enable: true
649-
externalClusters:
650-
- name: gke-test
651-
workloadIdentity:
652-
provider: gke
653-
gke:
654-
projectId: test-project
655-
clusterName: test-cluster
656-
clusterLocation: us-central1-a
657-
googleServiceAccount: scanner@test-project.iam.gserviceaccount.com
658-
EOF
659-
660-
# Run the operator
661-
MONDOO_NAMESPACE_OVERRIDE=mondoo-operator go run ./cmd/mondoo-operator/main.go operator --metrics-bind-address=:9090
662-
663-
# In another terminal, verify resources were created correctly:
664-
665-
# Check ServiceAccount has GKE annotation
666-
kubectl get sa mondoo-client-wif-gke-test -n mondoo-operator -o yaml
667-
# Expected annotation: iam.gke.io/gcp-service-account: scanner@test-project.iam.gserviceaccount.com
668-
669-
# Check CronJob has correct init container
670-
kubectl get cronjob mondoo-client-k8s-scan-gke-test -n mondoo-operator -o yaml | grep -A20 "initContainers"
671-
# Expected: image contains "google-cloud-cli", env vars include CLUSTER_NAME, PROJECT_ID, CLUSTER_LOCATION
672-
```
673-
674-
**Test EKS WIF Resource Creation:**
675-
676-
```bash
677-
# Delete previous config
678-
kubectl delete mondooauditconfig mondoo-client -n mondoo-operator
679-
680-
# Apply an EKS WIF configuration
681-
cat <<EOF | kubectl apply -f -
682-
apiVersion: k8s.mondoo.com/v1alpha2
683-
kind: MondooAuditConfig
684-
metadata:
685-
name: mondoo-client
686-
namespace: mondoo-operator
687-
spec:
688-
mondooCredsSecretRef:
689-
name: mondoo-client
690-
kubernetesResources:
691-
enable: true
692-
externalClusters:
693-
- name: eks-test
694-
workloadIdentity:
695-
provider: eks
696-
eks:
697-
region: us-west-2
698-
clusterName: test-cluster
699-
roleArn: arn:aws:iam::123456789012:role/TestRole
700-
EOF
701-
702-
# Verify resources:
703-
kubectl get sa mondoo-client-wif-eks-test -n mondoo-operator -o yaml
704-
# Expected annotation: eks.amazonaws.com/role-arn: arn:aws:iam::123456789012:role/TestRole
705-
706-
kubectl get cronjob mondoo-client-k8s-scan-eks-test -n mondoo-operator -o yaml | grep -A20 "initContainers"
707-
# Expected: image contains "aws-cli", env vars include CLUSTER_NAME, AWS_REGION
708-
```
709-
710-
**Test AKS WIF Resource Creation:**
711-
712-
```bash
713-
# Delete previous config
714-
kubectl delete mondooauditconfig mondoo-client -n mondoo-operator
715-
716-
# Apply an AKS WIF configuration
717-
cat <<EOF | kubectl apply -f -
718-
apiVersion: k8s.mondoo.com/v1alpha2
719-
kind: MondooAuditConfig
720-
metadata:
721-
name: mondoo-client
722-
namespace: mondoo-operator
723-
spec:
724-
mondooCredsSecretRef:
725-
name: mondoo-client
726-
kubernetesResources:
727-
enable: true
728-
externalClusters:
729-
- name: aks-test
730-
workloadIdentity:
731-
provider: aks
732-
aks:
733-
subscriptionId: 12345678-1234-1234-1234-123456789012
734-
resourceGroup: test-rg
735-
clusterName: test-cluster
736-
clientId: abcdef12-3456-7890-abcd-ef1234567890
737-
tenantId: fedcba98-7654-3210-fedc-ba9876543210
738-
EOF
739-
740-
# Verify resources:
741-
kubectl get sa mondoo-client-wif-aks-test -n mondoo-operator -o yaml
742-
# Expected annotation: azure.workload.identity/client-id: abcdef12-3456-7890-abcd-ef1234567890
743-
# Expected label: azure.workload.identity/use: "true"
744-
745-
kubectl get cronjob mondoo-client-k8s-scan-aks-test -n mondoo-operator -o yaml | grep -A20 "initContainers"
746-
# Expected: image contains "azure-cli", env vars include CLUSTER_NAME, RESOURCE_GROUP, SUBSCRIPTION_ID
747-
```
748-
749-
**Test Validation (Invalid Configurations):**
750-
751-
```bash
752-
# Test: Missing provider-specific config
753-
cat <<EOF | kubectl apply -f -
754-
apiVersion: k8s.mondoo.com/v1alpha2
755-
kind: MondooAuditConfig
756-
metadata:
757-
name: mondoo-invalid
758-
namespace: mondoo-operator
759-
spec:
760-
mondooCredsSecretRef:
761-
name: mondoo-client
762-
kubernetesResources:
763-
enable: true
764-
externalClusters:
765-
- name: invalid-test
766-
workloadIdentity:
767-
provider: gke
768-
# Missing gke config - should fail validation
769-
EOF
770-
771-
# Check operator logs for validation error
772-
# Expected: "GKE workload identity requires GKE configuration"
773-
```
774-
775-
**Cleanup:**
776-
777-
```bash
778-
kubectl delete mondooauditconfig --all -n mondoo-operator
779-
k3d cluster delete wif-test
780-
```
781-
782-
### WIF Testing Checklist
783-
784-
Use this checklist when testing WIF functionality:
785-
786-
| Test | Local (k3d) | Cloud Required |
787-
|------|-------------|----------------|
788-
| Unit tests pass | ✅ | - |
789-
| ServiceAccount created with correct annotations | ✅ | - |
790-
| ServiceAccount has correct labels (AKS only) | ✅ | - |
791-
| CronJob created with init container | ✅ | - |
792-
| Init container uses correct cloud CLI image | ✅ | - |
793-
| Init container has correct environment variables | ✅ | - |
794-
| Init container has volume mounts (kubeconfig, temp) | ✅ | - |
795-
| Validation rejects invalid configs | ✅ | - |
796-
| Init container successfully authenticates | - | ✅ |
797-
| Kubeconfig is generated correctly | - | ✅ |
798-
| Scanner can read resources from target cluster | - | ✅ |
799-
| End-to-end scan completes successfully | - | ✅ |
800-
801565
## Testing External Cluster Scanning
802566

803567
This section describes how to test external cluster scanning locally using k3d.
@@ -1275,13 +1039,21 @@ kubectl exec -n spire-system $SPIRE_SERVER -- /opt/spire/bin/spire-server entry
12751039
docker logs k3d-target-server-0 2>&1 | grep -i "client certificate"
12761040
```
12771041

1278-
### Test Method 4: Azure AKS Workload Identity
1042+
### Test Method 8: AKS Azure Workload Identity
12791043

12801044
This method tests Workload Identity Federation (WIF) with Azure AKS. The management cluster
12811045
uses Azure AD Workload Identity to authenticate to a target AKS cluster without static credentials.
12821046

12831047
> **Note:** This requires Azure resources and two AKS clusters. It cannot be tested locally with k3d.
12841048

1049+
**Key AKS-specific details:**
1050+
- ServiceAccount annotation: `azure.workload.identity/client-id`
1051+
- ServiceAccount label: `azure.workload.identity/use: "true"` (required)
1052+
- Init container image: `mcr.microsoft.com/azure-cli:2.67.0`
1053+
- Kubeconfig command: `az aks get-credentials --resource-group $RESOURCE_GROUP --name $CLUSTER_NAME --subscription $SUBSCRIPTION_ID`
1054+
- Environment variables: `CLUSTER_NAME`, `RESOURCE_GROUP`, `SUBSCRIPTION_ID`
1055+
- API fields: `subscriptionId`, `resourceGroup`, `clusterName`, `clientId`, `tenantId`
1056+
12851057
#### Prerequisites
12861058

12871059
- Azure CLI (`az`) installed and authenticated
@@ -1510,7 +1282,7 @@ kubectl create job aks-wif-scan-test \
15101282
-n mondoo-operator
15111283
15121284
# Watch the init container (Azure CLI fetching credentials)
1513-
kubectl logs -n mondoo-operator job/aks-wif-scan-test -c wif-init -f
1285+
kubectl logs -n mondoo-operator job/aks-wif-scan-test -c generate-kubeconfig -f
15141286
15151287
# Then watch the main container
15161288
kubectl logs -n mondoo-operator job/aks-wif-scan-test -f
@@ -1540,7 +1312,7 @@ az role assignment list --assignee $WI_SP_OBJECT_ID --scope $TARGET_CLUSTER_ID -
15401312
**Azure CLI in init container fails:**
15411313
```bash
15421314
# Check the init container logs for detailed error
1543-
kubectl logs -n mondoo-operator job/aks-wif-scan-test -c wif-init
1315+
kubectl logs -n mondoo-operator job/aks-wif-scan-test -c generate-kubeconfig
15441316
15451317
# Verify the ServiceAccount token is being projected
15461318
kubectl get pod -n mondoo-operator -l job-name=aks-wif-scan-test -o yaml | grep -A20 "serviceAccountToken"
@@ -1557,13 +1329,20 @@ az aks delete --resource-group $AZURE_RESOURCE_GROUP --name $MGMT_CLUSTER_NAME -
15571329
az aks delete --resource-group $AZURE_RESOURCE_GROUP --name $TARGET_CLUSTER_NAME --yes --no-wait
15581330
```
15591331

1560-
### Test Method 5: AWS EKS IRSA (IAM Roles for Service Accounts)
1332+
### Test Method 7: EKS IAM Roles for Service Accounts (IRSA)
15611333

15621334
This method tests Workload Identity using AWS EKS IRSA. The management cluster uses IAM Roles
15631335
for Service Accounts to authenticate to a target EKS cluster without static credentials.
15641336

15651337
> **Note:** This requires AWS resources and two EKS clusters. It cannot be tested locally with k3d.
15661338

1339+
**Key EKS-specific details:**
1340+
- ServiceAccount annotation: `eks.amazonaws.com/role-arn`
1341+
- Init container image: `amazon/aws-cli:2.22.0`
1342+
- Kubeconfig command: `aws eks update-kubeconfig --name $CLUSTER_NAME --region $AWS_REGION`
1343+
- Environment variables: `CLUSTER_NAME`, `AWS_REGION`
1344+
- API fields: `region`, `clusterName`, `roleArn`
1345+
15671346
#### Prerequisites
15681347

15691348
- AWS CLI (`aws`) installed and configured
@@ -1840,7 +1619,7 @@ kubectl create job eks-irsa-scan-test \
18401619
-n mondoo-operator
18411620
18421621
# Watch the init container (AWS CLI fetching credentials)
1843-
kubectl logs -n mondoo-operator job/eks-irsa-scan-test -c wif-init -f
1622+
kubectl logs -n mondoo-operator job/eks-irsa-scan-test -c generate-kubeconfig -f
18441623
18451624
# Then watch the main container
18461625
kubectl logs -n mondoo-operator job/eks-irsa-scan-test -f
@@ -1874,7 +1653,7 @@ eksctl get iamidentitymapping --cluster $TARGET_CLUSTER_NAME --region $AWS_REGIO
18741653
**AWS CLI in init container fails:**
18751654
```bash
18761655
# Check the init container logs for detailed error
1877-
kubectl logs -n mondoo-operator job/eks-irsa-scan-test -c wif-init
1656+
kubectl logs -n mondoo-operator job/eks-irsa-scan-test -c generate-kubeconfig
18781657
18791658
# Verify the IRSA webhook is injecting the environment variables
18801659
kubectl get pod -n mondoo-operator -l job-name=eks-irsa-scan-test -o yaml | grep -A5 "AWS_"
@@ -1909,6 +1688,13 @@ uses GCP Workload Identity to authenticate to a target GKE cluster without stati
19091688

19101689
> **Note:** This requires GCP resources and two GKE clusters. It cannot be tested locally with k3d.
19111690

1691+
**Key GKE-specific details:**
1692+
- ServiceAccount annotation: `iam.gke.io/gcp-service-account`
1693+
- Init container image: `gcr.io/google.com/cloudsdktool/google-cloud-cli:499.0.0-slim`
1694+
- Kubeconfig command: `gcloud container clusters get-credentials $CLUSTER_NAME --project $PROJECT_ID --location $CLUSTER_LOCATION`
1695+
- Environment variables: `CLUSTER_NAME`, `PROJECT_ID`, `CLUSTER_LOCATION`
1696+
- API fields: `projectId`, `clusterName`, `clusterLocation`, `googleServiceAccount`
1697+
19121698
#### Prerequisites
19131699

19141700
- Google Cloud CLI (`gcloud`) installed and authenticated
@@ -2174,6 +1960,15 @@ gcloud container clusters delete $TARGET_CLUSTER_NAME \
21741960
gcloud iam service-accounts delete $GSA_EMAIL --project=$GCP_PROJECT_ID --quiet
21751961
```
21761962

1963+
### WIF Testing Summary
1964+
1965+
| Test Type | How to Run |
1966+
|-----------|-----------|
1967+
| Unit tests (all providers) | `go test -v ./controllers/k8s_scan/... -run "WIF\|WorkloadIdentity"` |
1968+
| GKE end-to-end | Follow Test Method 6 |
1969+
| EKS end-to-end | Follow Test Method 7 |
1970+
| AKS end-to-end | Follow Test Method 8 |
1971+
21771972
### Cleanup
21781973

21791974
```bash

0 commit comments

Comments
 (0)