cluster-operator - AI Agent Guide

Project Structure

Single-group layout (default):

cmd/main.go                    Manager entry (registers controllers/webhooks)
api/<version>/*_types.go       CRD schemas (+kubebuilder markers)
api/<version>/zz_generated.*   Auto-generated (DO NOT EDIT)
internal/controller/*          Reconciliation logic
internal/webhook/*             Validation/defaulting (if present)
config/crd/bases/*             Generated CRDs (DO NOT EDIT)
config/rbac/role.yaml          Generated RBAC (DO NOT EDIT)
config/samples/*               Example CRs (edit these)
Makefile                       Build/test/deploy commands
PROJECT                        Kubebuilder metadata Auto-generated (DO NOT EDIT)

Multi-group layout (for projects with multiple API groups):

api/<group>/<version>/*_types.go       CRD schemas by group
internal/controller/<group>/*          Controllers by group
internal/webhook/<group>/<version>/*   Webhooks by group and version (if present)

Multi-group layout organizes APIs by group name (e.g., batch, apps). Check the PROJECT file for multigroup: true.

To convert to multi-group layout:

Run: kubebuilder edit --multigroup=true
Move APIs: mkdir -p api/<group> && mv api/<version> api/<group>/
Move controllers: mkdir -p internal/controller/<group> && mv internal/controller/*.go internal/controller/<group>/
Move webhooks (if present): mkdir -p internal/webhook/<group> && mv internal/webhook/<version> internal/webhook/<group>/
Update import paths in all files
Fix path in PROJECT file for each resource
Update test suite CRD paths (add one more .. to relative paths)

Critical Rules

Never Edit These (Auto-Generated)

config/crd/bases/*.yaml - from make manifests
config/rbac/role.yaml - from make manifests
config/webhook/manifests.yaml - from make manifests
**/zz_generated.*.go - from make generate
PROJECT - from kubebuilder [OPTIONS]

Never Remove Scaffold Markers

Do NOT delete // +kubebuilder:scaffold:* comments. CLI injects code at these markers.

Keep Project Structure

Do not move files around. The CLI expects files in specific locations.

Always Use CLI Commands

Always use kubebuilder create api and kubebuilder create webhook to scaffold. Do NOT create files manually.

E2E Tests Require an Isolated Kind Cluster

The e2e tests are designed to validate the solution in an isolated environment (similar to GitHub Actions CI). Ensure you run them against a dedicated Kind cluster (not your “real” dev/prod cluster).

After Making Changes

After editing *_types.go or markers:

make manifests  # Regenerate CRDs/RBAC from markers
make generate   # Regenerate DeepCopy methods

After editing *.go files:

make fmt vet    # or: make checks  (adds govulncheck)
make just-unit-tests   # fast: ginkgo only, no codegen

CLI Commands Cheat Sheet

Create API (your own types)

kubebuilder create api --group <group> --version <version> --kind <Kind>

Deploy Image Plugin (scaffold to deploy/manage ANY container image)

Generate a controller that deploys and manages a container image (nginx, redis, memcached, your app, etc.):

# Example: deploying memcached
kubebuilder create api --group example.com --version v1alpha1 --kind Memcached \
  --image=memcached:alpine \
  --plugins=deploy-image.go.kubebuilder.io/v1-alpha

Scaffolds good-practice code: reconciliation logic, status conditions, finalizers, RBAC. Use as a reference implementation.

Create Webhooks

# Validation + defaulting
kubebuilder create webhook --group <group> --version <version> --kind <Kind> \
  --defaulting --programmatic-validation

# Conversion webhook (for multi-version APIs)
kubebuilder create webhook --group <group> --version v1 --kind <Kind> \
  --conversion --spoke v2

Controller for Core Kubernetes Types

# Watch Pods
kubebuilder create api --group core --version v1 --kind Pod \
  --controller=true --resource=false

# Watch Deployments
kubebuilder create api --group apps --version v1 --kind Deployment \
  --controller=true --resource=false

Controller for External Types (e.g., from other operators)

Watch resources from external APIs (cert-manager, Argo CD, Istio, etc.):

# Example: watching cert-manager Certificate resources
kubebuilder create api \
  --group cert-manager --version v1 --kind Certificate \
  --controller=true --resource=false \
  --external-api-path=github.com/cert-manager/cert-manager/pkg/apis/certmanager/v1 \
  --external-api-domain=io \
  --external-api-module=github.com/cert-manager/cert-manager

Note: Use --external-api-module=<module>@<version> only if you need a specific version. Otherwise, omit @<version> to use what's in go.mod.

Webhook for External Types

# Example: validating external resources
kubebuilder create webhook \
  --group cert-manager --version v1 --kind Issuer \
  --defaulting \
  --external-api-path=github.com/cert-manager/cert-manager/pkg/apis/certmanager/v1 \
  --external-api-domain=io \
  --external-api-module=github.com/cert-manager/cert-manager

Makefile (tests & tooling)

Run make help for all targets with descriptions. Tool binaries default to ./bin (LOCALBIN, override if needed).

Test targets

Target	What it does
`make unit-tests`	Installs tools, `controller-gen`, kubebuilder assets, `generate`/`fmt`/`vet`/`manifests`, then Ginkgo without `integration` label on `api/`, `internal/`, `pkg/`.
`make just-unit-tests`	Ginkgo only (no regen). Same suites/label as above.
`make integration-tests`	Same preamble as unit-tests, then controller tests with `integration` label under `internal/controller/`.
`make just-integration-tests`	Ginkgo integration only (still needs `kubebuilder-assets`).
`make tests`	Runs `unit-tests`, `integration-tests`, `system-tests`, `kubectl-plugin-tests` in sequence.
`make system-tests`	Ginkgo on `test/system/`; needs a real cluster (`~/.kube/config`).
`make kubectl-plugin-tests`	Bats tests for kubectl plugin (`./bin/kubectl-rabbitmq.bats`; expects `bin/kubectl-rabbitmq`).
`make test-e2e`	Creates/uses Kind cluster `KIND_CLUSTER`, runs `go test -tags=e2e ./test/e2e/ -v -ginkgo.v`, then deletes the cluster. E2e harness may skip cert-manager if env `CERT_MANAGER_INSTALL_SKIP=true` (see Makefile comment).
`make setup-test-e2e` / `make cleanup-test-e2e`	Create or delete the e2e Kind cluster only.

Envtest / kubebuilder: kubebuilder-assets and unit-tests/integration-tests download apiserver+etcd via setup-envtest; binaries live under testbin/ (LOCAL_TESTBIN). KUBEBUILDER_ASSETS is set by the Makefile from ENVTEST_K8S_VERSION (derived from k8s.io/api in go.mod). If asset setup fails, try make clean-testbin then retry.

Useful variables (override on CLI: `make VAR=value`)

GINKGO_PROCS (default 4) — Ginkgo parallelism for unit/integration/system.
GINKGO_EXTRA — Extra args appended to every Ginkgo invocation.
K8S_OPERATOR_NAMESPACE (default rabbitmq-system) — operator namespace for local run and system tests.
SYSTEM_TEST_NAMESPACE (default cluster-operator-system-tests) — namespace for system tests.
RABBITMQ_SERVICE_TYPE (default NodePort) — passed to system tests.
KIND_CLUSTER (default cluster-operator-e2e) — Kind cluster name for e2e.
IMG — image for deploy / deploy-secure-metrics (default ghcr.io/rabbitmq/cluster-operator:latest).
CONTAINER (default docker) — container CLI for image builds.

Other dev targets: make checks (fmt, vet, govulncheck), make install-tools, make manager, make run (regenerates, checks, installs CRDs, deploys namespace RBAC, then go run with K8S_OPERATOR_NAMESPACE; optional OPERATOR_ARGS for extra flags).

Tests use Ginkgo + Gomega. Check suite_test.go files for suite setup.

Deployment Workflow

# 1. Regenerate manifests
make manifests generate

# 2. Build & deploy
export IMG=<registry>/<project>:tag
make docker-build docker-push IMG=$IMG  # Or: kind load docker-image $IMG --name <cluster>
make deploy IMG=$IMG

# 3. Test
kubectl apply -k config/samples/

# 4. Debug
kubectl logs -n <project>-system deployment/<project>-controller-manager -c manager -f

API Design

Key markers for api/<version>/*_types.go:

// +kubebuilder:object:root=true
// +kubebuilder:subresource:status
// +kubebuilder:resource:scope=Namespaced
// +kubebuilder:printcolumn:name="Status",type=string,JSONPath=".status.conditions[?(@.type=='Ready')].status"

// On fields:
// +kubebuilder:validation:Required
// +kubebuilder:validation:Minimum=1
// +kubebuilder:validation:MaxLength=100
// +kubebuilder:validation:Pattern="^[a-z]+$"
// +kubebuilder:default="value"

Use metav1.Condition for status (not custom string fields)
Use predefined types: metav1.Time instead of string for dates
Follow K8s API conventions: Standard field names (spec, status, metadata)

Controller Design

RBAC markers in internal/controller/*_controller.go:

// +kubebuilder:rbac:groups=mygroup.example.com,resources=mykinds,verbs=get;list;watch;create;update;patch;delete
// +kubebuilder:rbac:groups=mygroup.example.com,resources=mykinds/status,verbs=get;update;patch
// +kubebuilder:rbac:groups=mygroup.example.com,resources=mykinds/finalizers,verbs=update
// +kubebuilder:rbac:groups=events.k8s.io,resources=events,verbs=create;patch
// +kubebuilder:rbac:groups=apps,resources=deployments,verbs=get;list;watch;create;update;patch;delete

Implementation rules:

Idempotent reconciliation: Safe to run multiple times
Re-fetch before updates: r.Get(ctx, req.NamespacedName, obj) before r.Update to avoid conflicts
Structured logging: log := log.FromContext(ctx); log.Info("msg", "key", val)
Owner references: Enable automatic garbage collection (SetControllerReference)
Watch secondary resources: Use .Owns() or .Watches(), not just RequeueAfter
Finalizers: Clean up external resources (buckets, VMs, DNS entries)

Webhooks

Create all types together: --defaulting --programmatic-validation --conversion
When--forceis used: Backup custom logic first, then restore after scaffolding
For multi-version APIs: Use hub-and-spoke pattern (--conversion --spoke v2)
- Hub version: Usually oldest stable version (v1)
- Spoke versions: Newer versions that convert to/from hub (v2, v3)
- Example: --group crew --version v1 --kind Captain --conversion --spoke v2 (v1 is hub, v2 is spoke)

Learning from Examples

The deploy-image plugin scaffolds a complete controller following good practices. Use it as a reference implementation:

kubebuilder create api --group example --version v1alpha1 --kind MyApp \
  --image=<your-image> --plugins=deploy-image.go.kubebuilder.io/v1-alpha

Generated code includes: status conditions (metav1.Condition), finalizers, owner references, events, idempotent reconciliation.

Distribution Options

Option 1: YAML Bundle (Kustomize)

# Generate release manifests under releases/ (see Makefile: generate-installation-manifest)
make generate-installation-manifest

Key points: This repo emits installation YAML under releases/ (not dist/install.yaml). Adjust URLs/docs to match your publishing layout.

Option 2: Helm Chart

kubebuilder edit --plugins=helm/v2-alpha  # One-time, generates dist/chart/
# Users install: helm install my-release ./dist/chart/ --namespace <ns> --create-namespace

Important: If you add webhooks or modify manifests after initial chart generation:

Backup any customizations in dist/chart/values.yaml and dist/chart/manager/manager.yaml
Re-run: kubebuilder edit --plugins=helm/v2-alpha --force
Manually restore your custom values from the backup

Publish Container Image

export IMG=<registry>/<project>:<version>
make docker-build docker-push IMG=$IMG

References

Essential Reading

Kubebuilder Book: https://book.kubebuilder.io (comprehensive guide)
controller-runtime FAQ: https://github.com/kubernetes-sigs/controller-runtime/blob/main/FAQ.md (common patterns and questions)
Good Practices: https://book.kubebuilder.io/reference/good-practices.html (why reconciliation is idempotent, status conditions, etc.)

API Design & Implementation

API Conventions: https://github.com/kubernetes/community/blob/master/contributors/devel/sig-architecture/api-conventions.md
Operator Pattern: https://kubernetes.io/docs/concepts/extend-kubernetes/operator/
Markers Reference: https://book.kubebuilder.io/reference/markers.html

Tools & Libraries

controller-runtime: https://github.com/kubernetes-sigs/controller-runtime
controller-tools: https://github.com/kubernetes-sigs/controller-tools
Kubebuilder Repo: https://github.com/kubernetes-sigs/kubebuilder

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

cluster-operator - AI Agent Guide

Project Structure

Critical Rules

Never Edit These (Auto-Generated)

Never Remove Scaffold Markers

Keep Project Structure

Always Use CLI Commands

E2E Tests Require an Isolated Kind Cluster

After Making Changes

CLI Commands Cheat Sheet

Create API (your own types)

Deploy Image Plugin (scaffold to deploy/manage ANY container image)

Create Webhooks

Controller for Core Kubernetes Types

Controller for External Types (e.g., from other operators)

Webhook for External Types

Makefile (tests & tooling)

Test targets

Useful variables (override on CLI: `make VAR=value`)

Deployment Workflow

API Design

Controller Design

Webhooks

Learning from Examples

Distribution Options

Option 1: YAML Bundle (Kustomize)

Option 2: Helm Chart

Publish Container Image

References

Essential Reading

API Design & Implementation

Tools & Libraries

FilesExpand file tree

AGENTS.md

Latest commit

History

AGENTS.md

File metadata and controls

cluster-operator - AI Agent Guide

Project Structure

Critical Rules

Never Edit These (Auto-Generated)

Never Remove Scaffold Markers

Keep Project Structure

Always Use CLI Commands

E2E Tests Require an Isolated Kind Cluster

After Making Changes

CLI Commands Cheat Sheet

Create API (your own types)

Deploy Image Plugin (scaffold to deploy/manage ANY container image)

Create Webhooks

Controller for Core Kubernetes Types

Controller for External Types (e.g., from other operators)

Webhook for External Types

Makefile (tests & tooling)

Test targets

Useful variables (override on CLI: make VAR=value)

Deployment Workflow

API Design

Controller Design

Webhooks

Learning from Examples

Distribution Options

Option 1: YAML Bundle (Kustomize)

Option 2: Helm Chart

Publish Container Image

References

Essential Reading

API Design & Implementation

Tools & Libraries

Useful variables (override on CLI: `make VAR=value`)