[SPARK-59540] Warn about deprecated Helm enable keys via NOTES.txt

[peter-toth]

What changes were proposed in this pull request?

This PR adds templates/NOTES.txt to the Helm chart. It prints a deprecation warning when a values file still carries operatorDeployment.networkPolicy.enable or operatorConfiguration.dynamicConfig.enable, the legacy keys that SPARK-59504 replaced with enabled.

hasKey is an exact presence signal for these two, because SPARK-59504 removed enable from the chart defaults. A key set to false still warns: the deprecation is about the key, not its value.

The ASF header sits inside a {{- /* ... */ -}} comment, so skywalking-eyes sees it and helm install does not print it.

Why are the changes needed?

The deprecation is announced in values.yaml, values.schema.json and docs/operations.md, but helm install and helm upgrade say nothing. Docs only reach people who go looking. When SPARK-59533 removes the keys in chart 2.0.0, a user still on networkPolicy.enable: true loses their NetworkPolicy with no error.

Does this PR introduce any user-facing change?

Yes. helm install and helm upgrade now print a NOTES section. Users on the current enabled keys see one line naming the installed version:

NOTES:
Apache Spark Kubernetes Operator 1.1.0-SNAPSHOT is installed.

Users still on a legacy key additionally get a warning naming the replacement and the removal target:

NOTES:
Apache Spark Kubernetes Operator 1.1.0-SNAPSHOT is installed.

WARNING: operatorDeployment.networkPolicy.enable is deprecated, use
         operatorDeployment.networkPolicy.enabled instead. The legacy key is still honored:
         the NetworkPolicy is created when either key is true. It will be removed in chart
         2.0.0, see SPARK-59533.

How was this patch tested?

• helm lint --strict passes with the default values and with each legacy key set.
• Rendered the notes for all six combinations and confirmed the warning fires on key presence only:

values	notes
defaults	no warning
networkPolicy.enable=true	networkPolicy warning
dynamicConfig.enable=true	dynamicConfig warning
both legacy keys set to false	both warnings
both new enabled keys set	no warning
networkPolicy=null	no warning, renders clean

• New Validate the deprecated helm key warning step in the lint job, next to the other deprecated-key assertions: renders with both legacy keys and asserts both warnings are printed, then renders with the default values and fails if any warning is printed. helm template skips NOTES.txt, so the step uses helm install --dry-run=client, which renders it without reaching a cluster.

Was this patch authored or co-authored using generative AI tooling?

Generated-by: Claude Opus 5

[dongjoon-hyun]

helm install --dry-run=client renders NOTES.txt without a cluster (verified locally with KUBECONFIG=/dev/null), so a real cluster is not required here. Could we move this check to the lint job, next to Validate deprecated helm values are still honored? That avoids coupling it to the unrelated configmap-metadata group and avoids applying a NetworkPolicy and dynamicConfig RBAC changes to the live release.

      - name: Validate the deprecated helm key warning
        run: |
          notes=$(helm install spark build-tools/helm/spark-kubernetes-operator --dry-run=client \
            --set operatorDeployment.networkPolicy.enable=true \
            --set operatorConfiguration.dynamicConfig.enable=true)
          echo "$notes" | grep -q 'operatorDeployment.networkPolicy.enable is deprecated'
          echo "$notes" | grep -q 'operatorConfiguration.dynamicConfig.enable is deprecated'
          if helm install spark build-tools/helm/spark-kubernetes-operator --dry-run=client \
            | grep -q 'is deprecated'; then
            echo "Deprecation warning printed without a legacy key"; exit 1
          fi

If so, please also update the PR description's "a real cluster is the only place this is observable".

[peter-toth]

Moved to the lint job, description updated. One wrinkle: on helm 3.18.6 that command still fails with Kubernetes cluster unreachable, even with KUBECONFIG=/dev/null. The runner has Helm 4.2.4 so CI is fine, but a local run on helm 3.x will report a false failure.

[dongjoon-hyun]

nit: hasKey fails on a nil map. With --set operatorDeployment.networkPolicy=null, the install now fails here:

NOTES.txt:18:21 ... wrong type for value; expected map[string]interface {}; got interface {}

The current chart renders fine with the same input because _helpers.tpl uses or $np.enabled $np.enable. It is an unrealistic input, but it could be guarded with hasKey (.Values.operatorDeployment.networkPolicy | default dict) "enable" (and likewise for dynamicConfig).

[peter-toth]

Reproduced and fixed with | default dict on both toggles. --set operatorConfiguration.dynamicConfig=null still fails, but identically on main: _helpers.tpl:137 reads .dynamicConfig.source unguarded.

[dongjoon-hyun]

+1, LGTM.

[dongjoon-hyun]

Thank you, @peter-toth!