Skip to content

Commit dab1e34

Browse files
authored
Fix events RBAC and automate Helm ClusterRole generation from markers (#292)
## Problem ### Two related issues: 1. Events permission used the wrong API group. The +kubebuilder:rbac marker for events specified groups=events.k8s.io (the newer structured events API), but controller-runtime records v1.Event objects against the core "" API group. This caused the controller to log Server rejected event (will not retry!) errors when deployed in a different namespace from the TemporalWorkerDeployment CRs it manages — common cluster-wide deployment pattern. 2. The Helm ClusterRole was hand-maintained and had drifted from the Go markers. The +kubebuilder:rbac markers in worker_controller.go were incomplete: workerresourcetemplates, workerresourcetemplates/status, and subjectaccessreviews were all present in the Helm chart but had no corresponding markers. This made it easy for RBAC rules to fall out of sync — which is exactly how the events bug happened in the first place. ## Changes Fix the events API group (worker_controller.go, config/rbac/role.yaml): correct groups=events.k8s.io → groups=core in the marker; the generated manifest now uses apiGroups: [""]. Add missing markers (worker_controller.go): add markers for workerresourcetemplates (get/list/watch/patch/update), workerresourcetemplates/status (get/patch/update), and authorization.k8s.io/subjectaccessreviews (create) to match what was already deployed by the Helm chart. Automate Helm ClusterRole sync (hack/sync-rbac-rules.py, Makefile, helm/.../rbac.yaml): make manifests now runs a script that reads the controller-gen-generated config/rbac/role.yaml and replaces the # GENERATED RULES BEGIN / # GENERATED RULES END section in the Helm template. The Helm-templated dynamic rules (the allowedResources range) are left untouched. Going forward, adding a +kubebuilder:rbac marker and running make manifests is all that's needed to update the deployed ClusterRole. Closes #277
1 parent 5f77eb8 commit dab1e34

5 files changed

Lines changed: 108 additions & 30 deletions

File tree

.github/workflows/helm-validate.yml

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -74,13 +74,27 @@ jobs:
7474
- name: Template CRDs chart
7575
run: helm template test-release helm/temporal-worker-controller-crds
7676

77+
helm-check-rbac:
78+
name: Check Helm RBAC wasn't manually updated
79+
runs-on: ubuntu-latest
80+
steps:
81+
- uses: actions/checkout@v4
82+
83+
- uses: actions/setup-go@v5
84+
with:
85+
go-version-file: go.mod
86+
87+
- name: Regenerate and diff
88+
run: make manifests && git diff --exit-code helm/temporal-worker-controller/templates/rbac.yaml
89+
7790
helm-validate-succeed:
7891
name: All Helm Validations Succeed
7992
needs:
8093
- helm-lint
8194
- helm-template
8295
- helm-lint-crds
8396
- helm-template-crds
97+
- helm-check-rbac
8498
runs-on: ubuntu-latest
8599
if: always()
86100
env:

Makefile

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -153,6 +153,7 @@ help: ## Display this help.
153153
manifests: controller-gen ## Generate ClusterRole and CustomResourceDefinition objects.
154154
GOWORK=off GO111MODULE=on $(CONTROLLER_GEN) rbac:roleName=manager-role crd:allowDangerousTypes=true,maxDescLen=0,generateEmbeddedObjectMeta=true paths=./api/... paths=./internal/... paths=./cmd/... \
155155
output:crd:artifacts:config=helm/temporal-worker-controller-crds/templates
156+
python3 hack/sync-rbac-rules.py
156157

157158
.PHONY: generate
158159
generate: controller-gen ## Generate code containing DeepCopy, DeepCopyInto, and DeepCopyObject method implementations.

hack/sync-rbac-rules.py

Lines changed: 65 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,65 @@
1+
#!/usr/bin/env python3
2+
"""Sync rules from config/rbac/role.yaml into the Helm ClusterRole template.
3+
4+
The Helm template must contain:
5+
# GENERATED RULES BEGIN
6+
...
7+
# GENERATED RULES END
8+
markers inside its ClusterRole rules section. Everything between the markers
9+
is replaced with the rules from config/rbac/role.yaml (indented by two spaces).
10+
"""
11+
import re
12+
import sys
13+
14+
ROLE_YAML = "config/rbac/role.yaml"
15+
HELM_RBAC = "helm/temporal-worker-controller/templates/rbac.yaml"
16+
BEGIN_MARKER = " # GENERATED RULES BEGIN"
17+
END_MARKER = " # GENERATED RULES END"
18+
19+
20+
def extract_rules_text(path):
21+
with open(path) as f:
22+
content = f.read()
23+
idx = content.find("\nrules:\n")
24+
if idx == -1:
25+
print(f"ERROR: 'rules:' not found in {path}", file=sys.stderr)
26+
sys.exit(1)
27+
rules_body = content[idx + len("\nrules:\n"):]
28+
# Indent lines relative to the `rules:` key in the Helm template.
29+
# controller-gen emits two indentation levels:
30+
# col 0: outer list items (e.g. "- apiGroups:") → add 2 spaces
31+
# col 2: inner list values (e.g. " - events") → add 4 spaces
32+
# col 2: mapping keys (e.g. " resources:") → add 2 spaces
33+
# The extra indent on inner list values matches the style used by the
34+
# hand-authored rules in the Helm template.
35+
lines = rules_body.splitlines(keepends=True)
36+
result = []
37+
for line in lines:
38+
if not line.strip():
39+
result.append(line)
40+
elif line.startswith(" - "):
41+
result.append(" " + line) # inner list value: 2 global + 2 extra
42+
else:
43+
result.append(" " + line) # outer list item or mapping key: 2 global
44+
return "".join(result)
45+
46+
47+
def update_helm(path, rules_text):
48+
with open(path) as f:
49+
content = f.read()
50+
pattern = re.compile(
51+
r"(" + re.escape(BEGIN_MARKER) + r"[^\n]*\n)(.*?)(" + re.escape(END_MARKER) + r")",
52+
re.DOTALL,
53+
)
54+
if not pattern.search(content):
55+
print(f"ERROR: markers not found in {path}", file=sys.stderr)
56+
sys.exit(1)
57+
updated = pattern.sub(r"\g<1>" + rules_text + r"\g<3>", content)
58+
with open(path, "w") as f:
59+
f.write(updated)
60+
print(f"Synced RBAC rules from {ROLE_YAML}{HELM_RBAC}")
61+
62+
63+
if __name__ == "__main__":
64+
rules = extract_rules_text(ROLE_YAML)
65+
update_helm(HELM_RBAC, rules)

helm/temporal-worker-controller/templates/rbac.yaml

Lines changed: 17 additions & 22 deletions
Original file line numberDiff line numberDiff line change
@@ -63,6 +63,14 @@ kind: ClusterRole
6363
metadata:
6464
name: {{ .Release.Name }}-{{ .Release.Namespace }}-manager-role
6565
rules:
66+
# GENERATED RULES BEGIN - do not edit; run 'make manifests' to update
67+
- apiGroups:
68+
- ""
69+
resources:
70+
- events
71+
verbs:
72+
- create
73+
- patch
6674
- apiGroups:
6775
- ""
6876
resources:
@@ -89,6 +97,12 @@ rules:
8997
- deployments/scale
9098
verbs:
9199
- update
100+
- apiGroups:
101+
- authorization.k8s.io
102+
resources:
103+
- subjectaccessreviews
104+
verbs:
105+
- create
92106
- apiGroups:
93107
- temporal.io
94108
resources:
@@ -119,6 +133,7 @@ rules:
119133
- temporal.io
120134
resources:
121135
- temporalworkerdeployments/status
136+
- workerresourcetemplates/status
122137
verbs:
123138
- get
124139
- patch
@@ -130,30 +145,10 @@ rules:
130145
verbs:
131146
- get
132147
- list
133-
- watch
134148
- patch
135149
- update
136-
- apiGroups:
137-
- temporal.io
138-
resources:
139-
- workerresourcetemplates/status
140-
verbs:
141-
- get
142-
- patch
143-
- update
144-
- apiGroups:
145-
- ""
146-
resources:
147-
- events
148-
verbs:
149-
- create
150-
- patch
151-
- apiGroups:
152-
- authorization.k8s.io
153-
resources:
154-
- subjectaccessreviews
155-
verbs:
156-
- create
150+
- watch
151+
# GENERATED RULES END
157152
# Rules for managing resources attached via WorkerResourceTemplate.
158153
# Driven entirely by workerResourceTemplate.allowedResources in values.yaml.
159154
{{- range .Values.workerResourceTemplate.allowedResources }}

internal/controller/worker_controller.go

Lines changed: 11 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -95,14 +95,17 @@ type TemporalWorkerDeploymentReconciler struct {
9595
MaxDeploymentVersionsIneligibleForDeletion int32
9696
}
9797

98-
//+kubebuilder:rbac:groups=temporal.io,resources=temporalworkerdeployments,verbs=get;list;watch;create;update;patch;delete
99-
//+kubebuilder:rbac:groups=temporal.io,resources=temporalworkerdeployments/status,verbs=get;update;patch
100-
//+kubebuilder:rbac:groups=temporal.io,resources=temporalworkerdeployments/finalizers,verbs=update
101-
//+kubebuilder:rbac:groups=temporal.io,resources=temporalconnections,verbs=get;list;watch
102-
//+kubebuilder:rbac:groups=core,resources=secrets,verbs=get;list;watch
103-
//+kubebuilder:rbac:groups=apps,resources=deployments,verbs=get;list;watch;create;update;patch;delete
104-
//+kubebuilder:rbac:groups=apps,resources=deployments/scale,verbs=update
105-
// +kubebuilder:rbac:groups=events.k8s.io,resources=events,verbs=create;patch
98+
// +kubebuilder:rbac:groups=temporal.io,resources=temporalworkerdeployments,verbs=get;list;watch;create;update;patch;delete
99+
// +kubebuilder:rbac:groups=temporal.io,resources=temporalworkerdeployments/status,verbs=get;update;patch
100+
// +kubebuilder:rbac:groups=temporal.io,resources=temporalworkerdeployments/finalizers,verbs=update
101+
// +kubebuilder:rbac:groups=temporal.io,resources=temporalconnections,verbs=get;list;watch
102+
// +kubebuilder:rbac:groups=core,resources=secrets,verbs=get;list;watch
103+
// +kubebuilder:rbac:groups=apps,resources=deployments,verbs=get;list;watch;create;update;patch;delete
104+
// +kubebuilder:rbac:groups=apps,resources=deployments/scale,verbs=update
105+
// +kubebuilder:rbac:groups=core,resources=events,verbs=create;patch
106+
// +kubebuilder:rbac:groups=temporal.io,resources=workerresourcetemplates,verbs=get;list;watch;patch;update
107+
// +kubebuilder:rbac:groups=temporal.io,resources=workerresourcetemplates/status,verbs=get;patch;update
108+
// +kubebuilder:rbac:groups=authorization.k8s.io,resources=subjectaccessreviews,verbs=create
106109

107110
// Reconcile is part of the main kubernetes reconciliation loop which aims to
108111
// move the current state of the cluster closer to the desired state.

0 commit comments

Comments
 (0)