Complete Epic 4 retrospective and close Epic 4

- Fix ConditionsAware compile-time checks in policy_types.go and
  rabbitmqsecretenginerole_types.go (copy-paste bugs referencing wrong types)
- Add non-deletable type delete verification rule to project-context.md
- Add Epic 7.5 (CRD Field Annotation Refactor, 7 stories) to sprint status
- Mark epic-4 as done and epic-4-retrospective as done
- Save retrospective document with action items and Epic 5 preparation

Made-with: Cursor

Author: raffaelespazzoli

_bmad-output/implementation-artifacts/epic-4-retro-2026-04-23.md

@@ -0,0 +1,174 @@
+# Epic 4 Retrospective — Auth Engine Integration Tests
+
+**Date:** 2026-04-23
+**Facilitator:** Bob (Scrum Master)
+**Participants:** Raffa (Project Lead), Alice (Product Owner), Charlie (Senior Dev), Dana (QA Engineer), Amelia (Developer Agent)
+
+---
+
+## Epic Summary
+
+| Metric | Value |
+|--------|-------|
+| Epic | 4: Auth Engine Integration Tests |
+| Stories | 3 of 3 completed (100%) |
+| Duration | ~1 day (April 23, 2026) |
+| Scope | Integration tests for 6 auth engine types across 3 auth methods |
+| Debug failures | 7 total (2 + 2 + 3) — all environmental or external service quirks, zero operator code bugs |
+| Code review findings | 4 patches across 3 stories (0 + 3 + 1) |
+| New infrastructure | OpenLDAP wired into `make integration` (`deploy-ldap`), Keycloak 26.2 deployed to Kind (`deploy-keycloak`) |
+| Production code fixes | 1 (`ldapauthengineconfig_types.go` — `omitempty` on TLS version fields, later identified as incorrect workaround) |
+| New test patterns | Auth engine dependency chain (mount → config → role/group), non-deletable config persistence verification |
+| Technical debt documented | 1 new (Story 4.2 `omitempty` workaround violates CRD rules — tracked in Epic 7.5 Story 7.5.1) |
+| Regressions | 0 |
+| Coverage delta | 38.8% → 42.0% |
+
+### AI Models Used
+
+| Story | Model |
+|-------|-------|
+| 4.1 — KubernetesAuthEngineConfig/Role | Claude Opus 4.6 |
+| 4.2 — LDAPAuthEngineConfig/Group | Claude Opus 4.6 |
+| 4.3 — JWTOIDCAuthEngineConfig/Role | Claude Opus 4 |
+
+---
+
+## Epic 3 Retrospective Follow-Through
+
+| Action Item | Status |
+|-------------|--------|
+| Fix `CreateOrUpdateConfig` dual bug in PKI engine (carried from Epic 2) | ❌ Not addressed — downgraded from "must fix before Epic 5" to "fix in Epic 7" after confirming no Epic 5 types use VaultPKIEngineResource |
+| AC#4 extra-field handling → Story 7-4 | ⏳ In backlog — 6 more auth engine types confirmed with this pattern in Epic 4 |
+| Policy `ConditionsAware` copy-paste bug | ✅ Fixed during this retrospective (also found a second bug in `rabbitmqsecretenginerole_types.go`) |
+| Add checked type assertion rule to project-context.md | ✅ Applied — all 3 stories used checked type assertions consistently |
+| Continue using Opus 4.6 exclusively | ⚠️ Mostly — Stories 4.1/4.2 used Opus 4.6, Story 4.3 used Claude Opus 4 |
+| Story specs must flag infrastructure scope | ✅ Applied — all 3 stories explicitly classified infrastructure tier |
+
+Completed 2/6, in progress 1/6, not addressed 3/6. Key correction: PKI bug was incorrectly flagged as blocking Epic 5 in previous retros. The `VaultPKIEngineResource` reconciler is used exclusively by `PKISecretEngineConfig` — none of Epic 5's types (Database, RabbitMQ, GitHub, Azure, Quay, Kubernetes) use it.
+
+---
+
+## Successes
+
+1. **Real infrastructure over mocks.** All 3 stories deployed real services in Kind — Kubernetes API (already available), OpenLDAP (manifests wired into `make integration`), and Keycloak 26.2 (full OIDC provider with realm import). Vault actually validated these connections end-to-end.
+
+2. **Infrastructure template compounding.** The `deploy-*` Makefile target pattern (create namespace → apply manifests → wait for ready) was established in Story 4.2 and reused identically in Story 4.3. This template is directly reusable for RabbitMQ in Epic 5.
+
+3. **Story ordering by infrastructure complexity worked well.** 4.1 (no new infra) → 4.2 (LDAP — medium) → 4.3 (Keycloak — high). Each story built on patterns from the previous one.
+
+4. **Zero operator code bugs in debug failures.** All 7 debug failures were environmental (port conflicts, stale CRs, Keycloak filename conventions, health port, kubeconfig context) or Vault API surprises (`tls_min_version` empty string rejection, `policies` returned as `[]interface{}`). The operator code itself was solid.
+
+5. **Coverage grew 3.2 percentage points** (38.8% → 42.0%) — the largest single-epic coverage gain since Epic 2.
+
+---
+
+## Challenges
+
+1. **Non-deletable config persistence verification missed twice.** Both Stories 4.2 and 4.3 were caught in code review for the same gap: when deleting a `IsDeletable=false` type, the test verified K8s deletion but didn't prove the Vault config persisted. Now codified as a project-context rule.
+
+2. **Story 4.2 `omitempty` workaround violated CRD rules.** Vault 1.19.0 rejected empty `tls_min_version`/`tls_max_version` strings. The fix (adding `omitempty` to JSON tags) was a workaround that violates Rule 2 of the CRD Field Default & Validation Rules — non-zero defaults (`tls12`) should NOT have `omitempty`. The correct fix should be in the API interaction layer. Revert tracked in Epic 7.5 Story 7.5.1.
+
+3. **Keycloak external service quirks caused most debug time.** Story 4.3 had 3 debug failures, all Keycloak-specific: realm import filename convention (`test-realm-realm.json` not `test-realm.json`), health endpoint on management port 9000 not app port 8080, and wrong kubeconfig context. These are one-time learning costs that won't recur.
+
+---
+
+## Key Insights
+
+1. **Real infrastructure beats mocks for integration test confidence.** When Vault writes OIDC config and actually fetches Keycloak's discovery endpoint, or LDAP config and actually binds to OpenLDAP, that's proof the operator works — not just that it generates correct JSON.
+
+2. **Never change CRD annotations as a workaround for Vault API issues.** When Vault rejects a field value, fix the operator's API interaction (`toMap()`, `IsEquivalentToDesiredState`, or webhook `Default()`), not the CRD schema annotations. CRD annotations define the Kubernetes API contract; Vault compatibility is a separate concern.
+
+3. **PKI bug urgency was overstated for two retros.** The "must fix before Epic 5" claim from Epic 2 was never re-validated against Epic 5's actual scope. Lesson: always check whether a carried action item is truly blocking before escalating its priority.
+
+---
+
+## Action Items
+
+### Process Improvements
+
+1. **Non-deletable config persistence verification rule added to project-context.md**
+   - Status: ✅ Done during retrospective
+   - Description: When `IsDeletable=false`, delete tests must verify Vault config persists after CR deletion
+
+2. **CRD annotation workaround rule** (to be added)
+   - Owner: Bob (Scrum Master)
+   - Description: Never change CRD annotations (`omitempty`, `kubebuilder:default`) as a workaround for Vault API compatibility. Fix the API interaction layer instead.
+   - Success criteria: Rule added to project-context.md Critical Don't-Miss Rules section
+
+### Technical Debt
+
+3. **Fix `CreateOrUpdateConfig` dual bug in PKI engine** (CARRIED from Epic 2 — priority downgraded)
+   - Owner: Dev Agent (Epic 7)
+   - Priority: Medium — confirmed NOT blocking Epic 5
+   - Description: Two bugs in `vautlpkiengineobject.go:105-118`: (a) write path hardcoded to CRL, (b) equivalence check uses wrong payload
+
+4. **AC#4 extra-field handling → Story 7-4** (CARRIED from Epic 1)
+   - Owner: Story 7-4 (Epic 7)
+   - Priority: Medium — scope growing, all auth engine types confirmed with this pattern
+   - Status: Properly deferred
+
+5. **Revert Story 4.2 `omitempty` workaround on `TLSMinVersion`/`TLSMaxVersion`**
+   - Owner: Epic 7.5, Story 7.5.1 (Task 1.2)
+   - Priority: Low — functional workaround, proper fix tracked
+   - Status: Already planned
+
+### Items Resolved During Retrospective
+
+6. **Policy and RabbitMQSecretEngineRole `ConditionsAware` copy-paste bugs — FIXED**
+   - `policy_types.go:37`: `&PKISecretEngineRole{}` → `&Policy{}`
+   - `rabbitmqsecretenginerole_types.go:145`: `&RabbitMQSecretEngineConfig{}` → `&RabbitMQSecretEngineRole{}`
+
+7. **Epic 7.5 (CRD Field Annotation Refactor) added to sprint-status.yaml**
+   - 7 stories, positioned before Epic D1
+
+### Team Agreements
+
+- Continue using Opus-class models for integration test stories — validated across 12 consecutive stories (Epics 2–4)
+- Continue the code review process — catches real issues consistently
+- Infrastructure "install in Kind" template is now standard — reuse for RabbitMQ in Epic 5
+- Story ordering by ascending infrastructure complexity (proven in Epics 3 and 4)
+
+---
+
+## Epic 5 Preparation
+
+### Dependencies on Epic 4
+
+- VaultResource integration test pattern (established Epics 2–4) — direct reuse
+- `deploy-*` Makefile target template — reuse for RabbitMQ
+- Auth engine dependency chain pattern (mount → config → role) — analogous for secret engines
+- Non-deletable config persistence rule — ready for application
+
+### Infrastructure Requirements by Story
+
+| Story | External Dependency | Classification | Infrastructure Scope |
+|-------|-------------------|----------------|---------------------|
+| 5.1 — DatabaseSecretEngineConfig/Role | PostgreSQL | Already deployed (`deploy-postgresql`) | Low — no new infra |
+| 5.2 — RabbitMQ secret engine types | RabbitMQ | Install in Kind | Medium — new `deploy-rabbitmq` target |
+| 5.3 — Remaining secret engine types | Mixed | See below | Mixed |
+
+### Story 5.3 Type Classification
+
+| Type | Dependency | Classification | Rationale |
+|------|-----------|---------------|-----------|
+| KubernetesSecretEngineConfig/Role | Kubernetes API | Tier 1: Already available | Kind cluster IS the Kubernetes instance |
+| GitHubSecretEngineConfig/Role | GitHub App + custom Vault plugin | Tier 3: Skip | Cloud service, custom plugin already registered but no GitHub App |
+| AzureSecretEngineConfig/Role | Azure cloud | Tier 3: Skip | Cloud provider |
+| QuaySecretEngineConfig/Role/StaticRole | Quay registry + custom Vault plugin | Tier 3: Skip | Heavy deployment (PostgreSQL + Redis + Quay + custom Vault plugin not installed) |
+
+### Readiness Assessment
+
+- Testing & Quality: All tests passing, coverage at 42.0%
+- Technical Health: PKI bug confirmed non-blocking, copy-paste bugs fixed
+- Infrastructure: PostgreSQL already deployed, RabbitMQ needs new target
+- Unresolved Blockers: None
+
+### Verdict
+
+**Ready to proceed with Epic 5.** Story ordering: 5.1 (PostgreSQL — already there) → 5.2 (RabbitMQ — medium infra) → 5.3 (Kubernetes secret engine from Kind + skip cloud/Quay types).
+
+---
+
+## Team Performance
+
+Epic 4 delivered 3 stories covering integration tests for 6 auth engine types (KubernetesAuthEngineConfig, KubernetesAuthEngineRole, LDAPAuthEngineConfig, LDAPAuthEngineGroup, JWTOIDCAuthEngineConfig, JWTOIDCAuthEngineRole) with real infrastructure deployments (OpenLDAP, Keycloak), zero operator code bugs, zero regressions, and 3.2 percentage points of coverage growth. The retrospective fixed 2 copy-paste bugs, added a testing rule to project-context.md, tracked Epic 7.5 in sprint status, corrected the PKI bug priority, and identified the Story 4.2 CRD annotation workaround as a mistake to revert. The team is well-positioned for Epic 5.

_bmad-output/implementation-artifacts/sprint-status.yaml

@@ -36,6 +36,7 @@
 
 generated: 2026-04-11
 last_updated: 2026-04-23
+# Epic 4 retrospective completed 2026-04-23
 # Story 4.3 done 2026-04-23
 # Story 4.3 review 2026-04-23
 # Story 4.3 in-progress 2026-04-23
@@ -111,11 +112,11 @@ development_status:
   epic-3-retrospective: done
 
   # Epic 4: Auth Engine Integration Tests
-  epic-4: in-progress
+  epic-4: done
   4-1-integration-tests-for-kubernetesauthengineconfig-and-kubernetesauthenginerole: done
   4-2-integration-tests-for-ldapauthengineconfig-and-ldapauthenginegroup: done
   4-3-integration-tests-for-jwtoidcauthengineconfig-and-jwtoidcauthenginerole: done
-  epic-4-retrospective: optional
+  epic-4-retrospective: done
 
   # Epic 5: Secret Engine Integration Tests
   epic-5: backlog
@@ -141,6 +142,17 @@ development_status:
   7-5-drift-detection-integration-tests: backlog
   epic-7-retrospective: optional
 
+  # Epic 7.5: CRD Field Annotation Refactor
+  epic-7-5: backlog
+  7-5-1-ldap-auth-engine-types-annotation-refactor: backlog
+  7-5-2-jwtoidc-auth-engine-types-annotation-refactor: backlog
+  7-5-3-kubernetes-auth-and-secret-engine-types-annotation-refactor: backlog
+  7-5-4-azure-and-gcp-auth-secret-engine-types-annotation-refactor: backlog
+  7-5-5-pki-secret-engine-types-annotation-refactor: backlog
+  7-5-6-identity-and-remaining-types-annotation-refactor: backlog
+  7-5-7-validation-marker-sweep-enum-pattern-minlength-maxlength: backlog
+  epic-7-5-retrospective: optional
+
   # --- Phase 1.5: Documentation Improvement ---
 
   # Epic D1: Documentation Standards & Missing CertAuth Engine Docs

_bmad-output/project-context.md

@@ -140,6 +140,11 @@ Vault typically acts as an intermediary between vault-config-operator and a secu
 
 This rule applies to all current and future integration tests. When adding a new type, classify its external dependency into one of these three categories and document the decision in the story file.
 
+#### Non-Deletable Type Delete Verification
+- When a type has `IsDeletable() == false` (e.g., auth engine configs like KubernetesAuthEngineConfig, LDAPAuthEngineConfig, JWTOIDCAuthEngineConfig), the delete test **must** verify that the Vault configuration **persists** after the CR is deleted from Kubernetes — not just that K8s deletion succeeds.
+- After deleting the CR and confirming K8s `NotFound`, read the Vault path and assert the config data is still present (e.g., verify a key field like `url` or `kubernetes_host` still has the expected value).
+- This proves the `IsDeletable=false` contract: CR deletion removes the Kubernetes object but intentionally leaves the Vault state intact until the parent mount is deleted.
+
 #### Integration Test Pattern
 - Uses Ginkgo v2 `Describe`/`Context`/`It` BDD blocks with dot-imported `gomega` matchers.
 - Test fixtures are YAML files in `test/` directory, loaded via `controllertestutils.decoder.Get<TypeName>Instance("../test/<path>.yaml")`.

api/v1alpha1/policy_types.go

@@ -34,7 +34,7 @@ import (
 // NOTE: json tags are required.  Any new fields you add must have json tags for the fields to be serialized.
 
 var _ vaultutils.VaultObject = &Policy{}
-var _ vaultutils.ConditionsAware = &PKISecretEngineRole{}
+var _ vaultutils.ConditionsAware = &Policy{}
 
 func (d *Policy) GetVaultConnection() *vaultutils.VaultConnection {
 	return d.Spec.Connection

api/v1alpha1/rabbitmqsecretenginerole_types.go

@@ -142,7 +142,7 @@ type RabbitMQSecretEngineRoleList struct {
 	Items           []RabbitMQSecretEngineRole `json:"items"`
 }
 
-var _ vaultutils.ConditionsAware = &RabbitMQSecretEngineConfig{}
+var _ vaultutils.ConditionsAware = &RabbitMQSecretEngineRole{}
 
 func (d *RabbitMQSecretEngineRole) GetVaultConnection() *vaultutils.VaultConnection {
 	return d.Spec.Connection