Skip to content

Commit df96061

Browse files
dongjoon-hyundongjoon-hyun
committed
[SPARK-59504] Use enabled key convention in Helm values.yaml while honoring legacy enable
### What changes were proposed in this pull request? This PR aims to use the `enabled` key convention in the Helm chart `values.yaml` while still honoring the legacy `enable` keys. - `operatorDeployment.networkPolicy.enable` → `operatorDeployment.networkPolicy.enabled` - `operatorConfiguration.dynamicConfig.enable` → `operatorConfiguration.dynamicConfig.enabled` Two helpers in `_helpers.tpl` resolve each toggle to `true` when either the new or the legacy key is `true`, and all templates use them. The legacy keys stay valid in `values.schema.json`, marked as deprecated, and will be removed in chart `2.0.0`. ### Why are the changes needed? `<feature>.enabled` is the common Helm convention, and the operator's runtime property is already `spark.kubernetes.operator.dynamicConfig.enabled`. Resolving with `or` keeps existing `enable: true` users working because the chart default `enabled: false` is always merged and cannot be distinguished from a user-supplied value. ### Does this PR introduce _any_ user-facing change? Yes, compared to [v1.0.0 (2026-07-23)](https://github.com/apache/spark-kubernetes-operator/releases/tag/1.0.0). New users should use `enabled`. Existing values files with `enable` keep working without modification. ### How was this patch tested? - `helm lint --strict` passes. - Rendered with `helm template` for default, `enabled=true`, legacy `enable=true`, and legacy `enable=false` and confirmed the expected resources and properties. - Added a CI step in the `lint` job that renders the chart with the legacy `enable` keys and asserts they are still honored. - Existing Helm and E2E tests cover the new `enabled` keys. ### Was this patch authored or co-authored using generative AI tooling? Generated-by: Claude Fable 5.1 Closes #825 from dongjoon-hyun/SPARK-59504. Authored-by: Dongjoon Hyun <dongjoon@apache.org> Signed-off-by: Dongjoon Hyun <dongjoon@apache.org>
1 parent 20f0e19 commit df96061

13 files changed

Lines changed: 82 additions & 19 deletions

File tree

‎.github/workflows/build_and_test.yml‎

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -304,3 +304,27 @@ jobs:
304304
- name: Validate helm chart linting
305305
run: |
306306
helm lint --strict build-tools/helm/spark-kubernetes-operator
307+
- name: Validate deprecated helm values are still honored
308+
run: |
309+
helm template spark build-tools/helm/spark-kubernetes-operator \
310+
--set operatorDeployment.networkPolicy.enable=true \
311+
| grep -q 'kind: NetworkPolicy'
312+
helm template spark build-tools/helm/spark-kubernetes-operator \
313+
--set operatorConfiguration.dynamicConfig.enable=true \
314+
| grep -q 'spark.kubernetes.operator.dynamicConfig.enabled=true'
315+
- name: Validate helm values resolve to disabled
316+
run: |
317+
if helm template spark build-tools/helm/spark-kubernetes-operator \
318+
| grep -q 'kind: NetworkPolicy'; then
319+
echo "NetworkPolicy rendered with the default values"; exit 1
320+
fi
321+
if helm template spark build-tools/helm/spark-kubernetes-operator \
322+
--set operatorDeployment.networkPolicy.enable=false \
323+
| grep -q 'kind: NetworkPolicy'; then
324+
echo "NetworkPolicy rendered with enable=false"; exit 1
325+
fi
326+
if helm template spark build-tools/helm/spark-kubernetes-operator \
327+
--set operatorConfiguration.dynamicConfig.enable=false \
328+
| grep -q 'spark.kubernetes.operator.dynamicConfig.enabled=true'; then
329+
echo "dynamicConfig enabled with enable=false"; exit 1
330+
fi

‎build-tools/helm/spark-kubernetes-operator/templates/_helpers.tpl‎

Lines changed: 21 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -106,14 +106,34 @@ List of Spark workload namespaces. If not provied in values, use the same namesp
106106
{{- end }}
107107
{{- end }}
108108

109+
{{/*
110+
Whether the operator pod NetworkPolicy is enabled. The legacy key
111+
{operatorDeployment.networkPolicy.enable} is deprecated but still honored: the feature is
112+
enabled when either key is true.
113+
*/}}
114+
{{- define "spark-operator.networkPolicy.enabled" -}}
115+
{{- $np := .Values.operatorDeployment.networkPolicy -}}
116+
{{- if or $np.enabled $np.enable }}true{{ else }}false{{ end -}}
117+
{{- end }}
118+
119+
{{/*
120+
Whether dynamic config (hot properties loading) is enabled. The legacy key
121+
{operatorConfiguration.dynamicConfig.enable} is deprecated but still honored: the feature is
122+
enabled when either key is true.
123+
*/}}
124+
{{- define "spark-operator.dynamicConfig.enabled" -}}
125+
{{- $dc := .Values.operatorConfiguration.dynamicConfig -}}
126+
{{- if or $dc.enabled $dc.enable }}true{{ else }}false{{ end -}}
127+
{{- end }}
128+
109129
{{/*
110130
Default property overrides
111131
*/}}
112132
{{- define "spark-operator.defaultPropertyOverrides" -}}
113133
# Runtime resolved properties
114134
spark.kubernetes.operator.namespace={{ .Release.Namespace }}
115135
spark.kubernetes.operator.name={{- include "spark-operator.name" . }}
116-
spark.kubernetes.operator.dynamicConfig.enabled={{ .Values.operatorConfiguration.dynamicConfig.enable }}
136+
spark.kubernetes.operator.dynamicConfig.enabled={{ include "spark-operator.dynamicConfig.enabled" . }}
117137
spark.kubernetes.operator.dynamicConfig.source={{ .Values.operatorConfiguration.dynamicConfig.source }}
118138
spark.kubernetes.operator.metrics.port={{ include "spark-operator.metricsPort" . }}
119139
spark.kubernetes.operator.health.probePort={{ include "spark-operator.probePort" . }}

‎build-tools/helm/spark-kubernetes-operator/templates/network-policy.yaml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -13,7 +13,7 @@
1313
# See the License for the specific language governing permissions and
1414
# limitations under the License.
1515

16-
{{- if .Values.operatorDeployment.networkPolicy.enable }}
16+
{{- if eq (include "spark-operator.networkPolicy.enabled" .) "true" }}
1717
---
1818
apiVersion: networking.k8s.io/v1
1919
kind: NetworkPolicy

‎build-tools/helm/spark-kubernetes-operator/templates/operator-rbac.yaml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -173,7 +173,7 @@ metadata:
173173
{{- template "spark-operator.operatorRbacRules" $ }}
174174
---
175175
{{- end }}
176-
{{- if and .Values.operatorConfiguration.dynamicConfig.enable (eq .Values.operatorConfiguration.dynamicConfig.source "configMap") }}
176+
{{- if and (eq (include "spark-operator.dynamicConfig.enabled" .) "true") (eq .Values.operatorConfiguration.dynamicConfig.source "configMap") }}
177177
apiVersion: rbac.authorization.k8s.io/v1
178178
kind: Role
179179
metadata:

‎build-tools/helm/spark-kubernetes-operator/templates/spark-operator.yaml‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -162,7 +162,7 @@ spec:
162162
mountPath: /opt/spark-operator/logs
163163
- name: tmp-volume
164164
mountPath: /tmp
165-
{{- if and .Values.operatorConfiguration.dynamicConfig.enable (eq .Values.operatorConfiguration.dynamicConfig.source "file") }}
165+
{{- if and (eq (include "spark-operator.dynamicConfig.enabled" .) "true") (eq .Values.operatorConfiguration.dynamicConfig.source "file") }}
166166
- name: spark-operator-dynamic-config-volume
167167
mountPath: /opt/spark-operator/dynamic-conf
168168
readOnly: true
@@ -191,7 +191,7 @@ spec:
191191
emptyDir: { }
192192
- name: tmp-volume
193193
emptyDir: { }
194-
{{- if and .Values.operatorConfiguration.dynamicConfig.enable (eq .Values.operatorConfiguration.dynamicConfig.source "file") }}
194+
{{- if and (eq (include "spark-operator.dynamicConfig.enabled" .) "true") (eq .Values.operatorConfiguration.dynamicConfig.source "file") }}
195195
- name: spark-operator-dynamic-config-volume
196196
configMap:
197197
name: spark-kubernetes-operator-dynamic-configuration

‎build-tools/helm/spark-kubernetes-operator/templates/tests/test-network-policy.yaml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -13,7 +13,7 @@
1313
# See the License for the specific language governing permissions and
1414
# limitations under the License.
1515

16-
{{- if .Values.operatorDeployment.networkPolicy.enable }}
16+
{{- if eq (include "spark-operator.networkPolicy.enabled" .) "true" }}
1717
apiVersion: v1
1818
kind: Pod
1919
metadata:

‎build-tools/helm/spark-kubernetes-operator/values.schema.json‎

Lines changed: 12 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -396,13 +396,17 @@
396396
"type": "object",
397397
"description": "NetworkPolicy for the operator pod",
398398
"required": [
399-
"enable"
399+
"enabled"
400400
],
401401
"properties": {
402-
"enable": {
402+
"enabled": {
403403
"type": "boolean",
404404
"description": "Whether to create a NetworkPolicy for the operator pod"
405405
},
406+
"enable": {
407+
"type": "boolean",
408+
"description": "Deprecated, use 'enabled' instead; still honored (the NetworkPolicy is created when either key is true) and will be removed in chart 2.0.0 (SPARK-59533)"
409+
},
406410
"metricsIngress": {
407411
"type": ["array", "null"],
408412
"description": "List of NetworkPolicyPeer(s) allowed to reach the metrics port; when empty, ingress to the metrics port is denied",
@@ -818,16 +822,20 @@
818822
"type": "object",
819823
"description": "Dynamic configuration",
820824
"required": [
821-
"enable",
825+
"enabled",
822826
"create",
823827
"annotations",
824828
"data"
825829
],
826830
"properties": {
827-
"enable": {
831+
"enabled": {
828832
"type": "boolean",
829833
"description": "Enable dynamic configuration"
830834
},
835+
"enable": {
836+
"type": "boolean",
837+
"description": "Deprecated, use 'enabled' instead; still honored (dynamic configuration is enabled when either key is true) and will be removed in chart 2.0.0 (SPARK-59533)"
838+
},
831839
"create": {
832840
"type": "boolean",
833841
"description": "Create ConfigMap for dynamic configuration"

‎build-tools/helm/spark-kubernetes-operator/values.yaml‎

Lines changed: 7 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -94,7 +94,9 @@ operatorDeployment:
9494
# operator pod is denied. Note that this requires a CNI plugin with NetworkPolicy support,
9595
# and that egress traffic is not restricted.
9696
networkPolicy:
97-
enable: false
97+
# The legacy key `enable` is deprecated and will be removed in chart 2.0.0 (SPARK-59533).
98+
# It is still honored: the NetworkPolicy is created when either `enabled` or `enable` is true.
99+
enabled: false
98100
# List of NetworkPolicyPeer(s) allowed to reach the metrics port, e.g. the Prometheus
99101
# scraper. When empty, ingress to the metrics port is denied.
100102
metricsIngress: [ ]
@@ -218,8 +220,10 @@ operatorConfiguration:
218220
metrics.properties: |+
219221
# Metrics Properties Overrides
220222
dynamicConfig:
221-
# Enable this for hot properties loading.
222-
enable: false
223+
# Enable this for hot properties loading. The legacy key `enable` is deprecated and will be
224+
# removed in chart 2.0.0 (SPARK-59533). It is still honored: hot properties loading is
225+
# enabled when either `enabled` or `enable` is true.
226+
enabled: false
223227
# Source of the dynamic config overrides when enabled. Supported values:
224228
# configMap - (default) watch a ConfigMap via a Kubernetes informer. Requires the operator
225229
# to have RBAC to read ConfigMaps (created by this chart).

‎docs/operations.md‎

Lines changed: 9 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -133,7 +133,8 @@ following table:
133133
| operatorConfiguration.spark-operator.properties | The default operator configuration. | |
134134
| operatorConfiguration.metrics.properties | The default operator metrics (sink) configuration. | |
135135
| operatorConfiguration.dynamicConfig.create | If set to true, a config map would be created & watched by operator as source of truth for hot properties loading. | false |
136-
| operatorConfiguration.dynamicConfig.enable | If set to true, operator would honor the created config map as source of truth for hot properties loading. | false |
136+
| operatorConfiguration.dynamicConfig.enabled | If set to true, operator would honor the created config map as source of truth for hot properties loading. | false |
137+
| operatorConfiguration.dynamicConfig.enable | Deprecated, use `operatorConfiguration.dynamicConfig.enabled`. Still honored: `enable: true` wins over `enabled: false`. Removed in chart 2.0.0 (SPARK-59533). | |
137138
| operatorConfiguration.dynamicConfig.annotations | Annotations to be applied for the dynamicConfig resources. | `"helm.sh/resource-policy": keep` |
138139
| operatorConfiguration.dynamicConfig.data | Data field (key-value pairs) that acts as hot properties in the config map. | `spark.kubernetes.operator.reconciler.intervalSeconds: "60"` |
139140

@@ -154,7 +155,7 @@ for the operator pod:
154155
```yaml
155156
operatorDeployment:
156157
networkPolicy:
157-
enable: true
158+
enabled: true
158159
metricsIngress:
159160
- namespaceSelector:
160161
matchLabels:
@@ -174,6 +175,12 @@ Note that this requires a CNI plugin that enforces NetworkPolicy; on clusters wi
174175
a plugin the policy is silently ignored. Egress traffic of the operator (Kubernetes API
175176
server, DNS) is not restricted by this policy.
176177

178+
The legacy key `operatorDeployment.networkPolicy.enable` is deprecated in favor of `enabled`
179+
and will be removed in chart `2.0.0` ([SPARK-59533](https://issues.apache.org/jira/browse/SPARK-59533)).
180+
It is still honored: the NetworkPolicy is created when either key is `true`, so a stale
181+
`enable: true` in a base values file wins over `enabled: false` and must be removed to turn
182+
the feature off. The same rule applies to `operatorConfiguration.dynamicConfig.enable`.
183+
177184
## Operator Health(Liveness) Probe with Sentinel Resource
178185

179186
Learning

‎tests/e2e/helm/dynamic-config-values-2.yaml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -27,7 +27,7 @@ workloadResources:
2727

2828
operatorConfiguration:
2929
dynamicConfig:
30-
enable: true
30+
enabled: true
3131
create: true
3232
data:
3333
spark.kubernetes.operator.watchedNamespaces: "spark-3"

0 commit comments

Comments
 (0)