[SPARK-59596] Fix Helm chart configuration table in operations.md to match values.yaml

[dongjoon-hyun]

What changes were proposed in this pull request?

This PR aims to fix the Helm chart configuration table in docs/operations.md to be consistent with the Helm chart (values.yaml and templates/*.yaml).

Fix the wrong keys, descriptions and default values

Parameter	Before	After
Key	operatorDeployment.replica	operatorDeployment.replicas
Key	operatorDeployment.additionalContainers	operatorDeployment.operatorPod.additionalContainers
Default value of operatorDeployment.operatorPod.operatorContainer.jvmArgs	-Dfile.encoding=UTF8 -XX:+CrashOnOutOfMemoryError -XX:ErrorFile=/dev/stderr -XX:+UseParallelGC	-Dfile.encoding=UTF8 -XX:+CrashOnOutOfMemoryError -XX:ErrorFile=/dev/stderr -XX:+UseParallelGC -XX:InitialRAMPercentage=80 -XX:MaxRAMPercentage=80 -XX:+AlwaysPreTouch -XX:+UseCompactObjectHeaders
Description of operatorRbac.serviceAccount.name	Name of the operator Role.	Name of the operator ServiceAccount.
Description of operatorRbac.configManagement.create	... (hot property loading and leader election).	... (hot property loading from ConfigMap). Requires dynamicConfig with the configMap source.
Default value of operatorRbac.configManagement.roleName	spark-operator-config-role	"spark-operator-config-monitor"
Key	operatorRbac.configManagement.roleBinding	operatorRbac.configManagement.roleBindingName
Key	workloadResources.serviceAccounts.create	workloadResources.serviceAccount.create
Key	workloadResources.serviceAccounts.name	workloadResources.serviceAccount.name
Key	workloadResources.sparkApplicationSentinel.sentinelNamespaces	workloadResources.sparkApplicationSentinel.sentinelNamespaces.data
Key	workloadResources.sparkClusterSentinel.sentinelNamespaces	workloadResources.sparkClusterSentinel.sentinelNamespaces.data

Add the missing keys

Parameter	Default value
nameOverride
operatorDeployment.networkPolicy.enabled	false
operatorDeployment.networkPolicy.enable (deprecated)
operatorDeployment.networkPolicy.metricsIngress
operatorDeployment.operatorPod.affinity
operatorDeployment.operatorPod.tolerations
operatorDeployment.operatorPod.dnsPolicy
operatorDeployment.operatorPod.operatorContainer.volumeMounts
operatorDeployment.operatorPod.operatorContainer.metrics.port	19090
operatorRbac.annotations
operatorConfiguration.configMap.annotations
operatorConfiguration.configMap.labels
operatorConfiguration.dynamicConfig.source	configMap

Why are the changes needed?

The documentation is inconsistent with the actual Helm chart. Some documented keys do not exist in the chart, so users cannot configure the chart by following the documentation. In addition, several supported keys are not documented.

Does this PR introduce any user-facing change?

No behavior change. This is a documentation-only fix.

How was this patch tested?

Manual review.

Was this patch authored or co-authored using generative AI tooling?

Generated-by: Claude Opus 5

[peter-toth]

Thanks for the PR, @dongjoon-hyun!

I verified the table mechanically rather than by eye: flattening values.yaml and diffing both directions, every documented key now exists and every leaf in values.yaml has a documented ancestor, with the only three "missing" keys being the deliberate ones (image.digest, operatorRbac.annotations, dynamicConfig.enable). Every scalar default in the table matches too, including the long jvmArgs string. Diffing against values.schema.json instead of values.yaml is where the remaining gaps are - nameOverride is a live key with no row, and configManagement.create is documented as a toggle the chart ignores.

Non-blocking

• 1. nameOverride has no row, and fullnameOverride is dead: both are schema-declared and absent from values.yaml, the same category as the operatorRbac.annotations row you added. nameOverride changes 11 places in the rendered output; fullnameOverride changes nothing, because spark-operator.fullname has no consumer. [inline: operations.md:86]
• 2. configManagement.create is documented as a toggle no template reads: rendering with create=false and create=true is identical. The row now names the real gate, which is an improvement, but still presents create as the switch. [inline: operations.md:121]

Minor

• 3. The table documents one deprecated key but not its twin: dynamicConfig.enable has a row at :154, networkPolicy.enable does not, and this PR's new networkPolicy.enabled row is what makes that an in-table asymmetry. [inline: operations.md:88]

[peter-toth]

Finding 1. Anchoring here because a nameOverride row belongs just above, with the other top-level keys.

Diffing the table against values.yaml finds nothing left - I flattened both and every leaf has a documented ancestor. Diffing against values.schema.json finds two more, both top-level and both absent from values.yaml, which is exactly why they slipped through:

$ # schema properties with no documented ancestor, ignoring container nodes
nameOverride
fullnameOverride
operatorDeployment.networkPolicy.enable    # finding 3

They need opposite treatment.

nameOverride is live and should get a row. The schema describes it as "Override chart name", and _helpers.tpl:20 and :32 both read it:

$ helm template spark <chart> --set nameOverride=myop | grep -c myop
11

spark-operator.name is referenced from eight template files, so this key renames labels, the NetworkPolicy, the PDB and the helm-test pods. It is the same category as the operatorRbac.annotations row you added - schema-declared, no default in values.yaml, still settable.

fullnameOverride is dead and should not get a row. spark-operator.fullname is defined at _helpers.tpl:28 and referenced nowhere else in the chart:

$ grep -rn "spark-operator.fullname" build-tools/helm/spark-kubernetes-operator/
.../templates/_helpers.tpl:28:{{- define "spark-operator.fullname" -}}
$ helm template spark <chart> --set fullnameOverride=my-op | grep -c "my-op"
0

So documenting it would advertise a key that does nothing. Either drop the helper and its schema entry, or leave it out of the table - but the schema currently promises something the chart does not deliver, which is the same class of problem this PR is fixing from the other direction.

[peter-toth]

Finding 2. The second half of this description is the accurate part and a real improvement - operator-rbac.yaml:239 gates the Role on dynamicConfig.enabled and source == configMap. The first half is still wrong: nothing reads create.

$ grep -rn "configManagement" build-tools/helm/spark-kubernetes-operator/templates/
.../operator-rbac.yaml:242:  name: {{ .Values.operatorRbac.configManagement.roleName }}
.../operator-rbac.yaml:262:  name: {{ .Values.operatorRbac.configManagement.roleBindingName }}
.../operator-rbac.yaml:268:  name: {{ .Values.operatorRbac.configManagement.roleName }}

Only the two name fields. So the row promises a switch that does not exist, and I confirmed the render is identical either way:

$ helm template spark <chart> --set operatorRbac.configManagement.create=false \
    --set operatorConfiguration.dynamicConfig.enabled=true | grep -c spark-operator-config-monitor
3
$ # same command with create=true
3

A user following this table to suppress that Role and RoleBinding cannot, which is the failure mode the "Why" section describes. SPARK-59537 tracks wiring create in and is still Open, so until it lands the row could say so:

| operatorRbac.configManagement.create | Whether to create a Role for operator configuration management (hot property loading from ConfigMap). Currently not honored, see SPARK-59537: the Role is created whenever `dynamicConfig` is enabled with the `configMap` source. | true |

Or fix the chart here and keep the description as written - either resolves it, and the doc-only version is the smaller change.

[peter-toth]

Finding 3. operatorDeployment.networkPolicy.enable is still honored - _helpers.tpl resolves the toggle with or $np.enabled $np.enable - and it is removed in chart 2.0.0 under the same ticket, SPARK-59533. The table gives its twin a row:

:154 | operatorConfiguration.dynamicConfig.enable | Deprecated, use `operatorConfiguration.dynamicConfig.enabled`. Still honored: `enable: true` wins over `enabled: false`. Removed in chart 2.0.0 (SPARK-59533). |

Before this PR the table had no networkPolicy rows at all, so the legacy key living only in the prose at :196 was consistent. Now that enabled has a row, the pair reads as though only one of the two toggles was ever renamed. A matching row keeps the table self-contained:

| operatorDeployment.networkPolicy.enable | Deprecated, use `operatorDeployment.networkPolicy.enabled`. Still honored: `enable: true` wins over `enabled: false`. Removed in chart 2.0.0 (SPARK-59533). | |

[dongjoon-hyun]

Thank you for the review, @peter-toth. Addressed in 7bdd09d.

1. Added a nameOverride row. fullnameOverride is intentionally not documented because it has no consumer; I'll handle it in a separate JIRA.
2. SPARK-59537 has been merged to main (76761c2), so operatorRbac.configManagement.create is now honored and the current description is accurate.
3. Added a matching operatorDeployment.networkPolicy.enable deprecation row.

[dongjoon-hyun]

Merged to main