Skip to content

Commit 8d5d8c4

Browse files
alimarediaclaude
andcommitted
website: add Kustomize installation docs for Spark Operator
Add Kustomize as a first-class installation path alongside Helm in the Spark Operator getting-started guide. This includes install, configuration, upgrade, uninstall, and RBAC setup instructions for Kustomize users. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> Signed-off-by: Ali Maredia <amaredia@redhat.com>
1 parent d28741b commit 8d5d8c4

1 file changed

Lines changed: 117 additions & 27 deletions

File tree

content/en/docs/components/spark-operator/getting-started.md

Lines changed: 117 additions & 27 deletions
Original file line numberDiff line numberDiff line change
@@ -9,12 +9,17 @@ User Guide. If you are running the Kubernetes Operator for Apache Spark on Googl
99

1010
## Prerequisites
1111

12-
- Helm >= 3
1312
- Kubernetes >= 1.16
1413

1514
## Installation
1615

17-
### Add Helm Repo
16+
The Spark Operator can be installed using [Helm](#install-with-helm) or [Kustomize](#install-with-kustomize).
17+
18+
### Install with Helm
19+
20+
Helm >= 3 is required to install the Spark Operator chart.
21+
22+
#### Add Helm Repo
1823

1924
```shell
2025
helm repo add spark-operator https://kubeflow.github.io/spark-operator
@@ -24,7 +29,7 @@ helm repo update
2429

2530
See [helm repo](https://helm.sh/docs/helm/helm_repo) for command documentation.
2631

27-
### Install the chart
32+
#### Install the chart
2833

2934
```shell
3035
helm install [RELEASE_NAME] spark-operator/spark-operator
@@ -61,15 +66,15 @@ Now you should see the operator running in the cluster by checking the status of
6166
helm status --namespace spark-operator my-release
6267
```
6368

64-
### Upgrade the Chart
69+
#### Upgrade the Chart
6570

6671
```shell
6772
helm upgrade [RELEASE_NAME] spark-operator/spark-operator [flags]
6873
```
6974

7075
See [helm upgrade](https://helm.sh/docs/helm/helm_upgrade) for command documentation.
7176

72-
### Uninstall the Chart
77+
#### Uninstall the Chart
7378

7479
```shell
7580
helm uninstall [RELEASE_NAME]
@@ -79,30 +84,119 @@ This removes all the Kubernetes resources associated with the chart and deletes
7984

8085
See [helm uninstall](https://helm.sh/docs/helm/helm_uninstall) for command documentation.
8186

82-
### Additional Steps to Integrate Jupyter Notebooks
87+
### Install with Kustomize
8388

84-
Integrating Jupyter Notebooks with the Spark Operator to run big data or distributed machine learning jobs with PySpark.
89+
Kustomize support is available starting from Spark Operator v2.5.1. You will need `kubectl` v1.14+ (which includes built-in Kustomize support) or a standalone [kustomize](https://kubectl.docs.kubernetes.io/installation/kustomize/) binary.
8590

86-
See [Integration with Notebooks](../user-guide/notebooks-spark-operator) for further details.
91+
To install using Kustomize, clone the release branch of the [Spark Operator repository](https://github.com/kubeflow/spark-operator) and apply the default configuration:
8792

88-
## Running the Examples
93+
```shell
94+
# Example: git clone --branch v2.5.1 https://github.com/kubeflow/spark-operator.git
95+
git clone --branch <release-tag> https://github.com/kubeflow/spark-operator.git
96+
cd spark-operator
97+
kubectl apply -k config/default --server-side --force-conflicts
98+
```
8999

90-
To run the Spark PI example, run the following command:
100+
By default, all resources are installed in the `spark-operator` namespace.
101+
102+
A successful `kubectl apply -k config/default --server-side --force-conflicts` creates:
103+
- The `spark-operator` Namespace
104+
- 3 CRDs (SparkApplication, ScheduledSparkApplication, SparkConnect)
105+
- Controller Deployment (1 replica) with ServiceAccount, ClusterRole, and leader-election Role
106+
- Webhook Deployment (1 replica) with ServiceAccount, ClusterRole, Role, Service, and self-signed TLS
107+
- MutatingWebhookConfiguration and ValidatingWebhookConfiguration
108+
109+
The webhook generates its own TLS certificates at startup (no cert-manager required by default).
110+
111+
Verify the operator is running:
91112

92113
```shell
93-
kubectl apply -f examples/spark-pi.yaml
114+
kubectl -n spark-operator get pods
94115
```
95116

96-
Note that `spark-pi.yaml` configures the driver pod to use the `spark` service account to communicate with the Kubernetes API server. You might need to replace it with the appropriate service account before submitting the job. If you installed the operator using the Helm chart and overrode `spark.jobNamespaces`, the service account name ends with `-spark` and starts with the Helm release name. For example, if you would like to run your Spark jobs to run in a namespace called `test-ns`, first make sure it already exists, and then install the chart with the command:
117+
#### Configuration with Kustomize
118+
119+
Unlike Helm values, kustomize parameters are edited directly in the manifest files or via overlays.
120+
121+
| What to change | Where to [edit](https://github.com/kubeflow/spark-operator) |
122+
|---|---|
123+
| Operator namespace | `namespace:` in `config/default/kustomization.yaml` |
124+
| Image tag | `images:` block in `config/default/kustomization.yaml`, or run `make kustomize-set-image` |
125+
| Controller flags (log level, workers, feature gates, etc.) | `args:` in `config/manager/manager.yaml` |
126+
| Webhook flags (port, log level, etc.) | `args:` in `config/webhook/deployment.yaml` |
127+
| Job namespaces (webhook selectors) | `values:` lists in `config/webhook/webhook-objectselector-patch.yaml` and `webhook-validating-selector-patch.yaml` |
128+
| Resource requests/limits | `resources:` in `config/manager/manager.yaml` and `config/webhook/deployment.yaml` |
129+
130+
131+
For larger customizations, create a [kustomize overlay](https://kubectl.docs.kubernetes.io/references/kustomize/kustomization/):
132+
133+
```bash
134+
mkdir -p my-overlay
135+
cat > my-overlay/kustomization.yaml <<EOF
136+
apiVersion: kustomize.config.k8s.io/v1beta1
137+
kind: Kustomization
138+
resources:
139+
- ../config/default
140+
# Add patches, images overrides, etc.
141+
EOF
142+
kubectl apply -k my-overlay --server-side --force-conflicts
143+
```
144+
145+
#### Upgrading the operator with Kustomize
146+
147+
To upgrade the operator using Kustomize manifests pull the latest manifests (or the desired release tag) and re-apply:
148+
149+
```
150+
git pull # or: git checkout v<new-version>
151+
kubectl apply -k config/default --server-side --force-conflicts
152+
```
153+
154+
#### Uninstall the operator with Kustomize
155+
156+
To uninstall, first delete the Spark RBAC resources from each application namespace, then delete the operator:
97157

98158
```shell
99-
helm install my-release spark-operator/spark-operator --namespace spark-operator --set "spark.jobNamespaces={test-ns}"
159+
kubectl delete -k config/spark-rbac -n <app-namespace>
160+
kubectl delete -k config/default
100161
```
101162

102-
Then the chart will set up a service account for your Spark jobs to use in that namespace.
163+
**Warning:** `kubectl delete -k config/default` will also remove the CRDs, which deletes all SparkApplication, ScheduledSparkApplication, and SparkConnect resources cluster-wide.
164+
165+
## Running the Examples
166+
167+
Spark driver and executor pods need a Spark-specific service account to communicate with the Kubernetes API server. The setup differs depending on your installation method.
168+
169+
### Helm RBAC setup
170+
171+
If you installed the operator using the Helm chart and overrode `spark.jobNamespaces`, the chart automatically creates a service account in the specified namespace. The service account name ends with `-spark` and starts with the Helm release name.
172+
173+
For example, if you would like your Spark jobs to run in a namespace called `test-ns`, first make sure it already exists, and then install the chart with the command:
174+
175+
```shell
176+
helm install my-release spark-operator/spark-operator --namespace spark-operator --set "spark.jobNamespaces={test-ns}"
177+
```
103178

104179
See the section on the [Spark Job Namespace](#about-spark-job-namespaces) for details on the behavior of the default Spark Job Namespace.
105180

181+
### Kustomize RBAC setup
182+
183+
If you installed the operator using Kustomize manifests, the service account and other RBAC resources must be created manually in each job namespace:
184+
185+
```shell
186+
# For jobs run in the default namespace
187+
kubectl -n default apply -k config/spark-rbac
188+
```
189+
190+
See the section on [Spark RBAC](#about-the-service-account-for-driver-pods) for details on the RBAC resources needed for Spark jobs.
191+
192+
### Run the Spark PI example
193+
194+
To run the Spark PI example, run the following command:
195+
196+
```shell
197+
kubectl apply -f examples/spark-pi.yaml
198+
```
199+
106200
Running the above command will create a `SparkApplication` object named `spark-pi`. Check the object by running the following command:
107201

108202
```shell
@@ -177,9 +271,15 @@ Events:
177271

178272
The operator submits the Spark Pi example to run once it receives an event indicating the `SparkApplication` object was added.
179273

274+
## Additional Steps to Integrate Jupyter Notebooks
275+
276+
Integrating Jupyter Notebooks with the Spark Operator to run big data or distributed machine learning jobs with PySpark.
277+
278+
See [Integration with Notebooks](../user-guide/notebooks-spark-operator) for further details.
279+
180280
## Configuration
181281

182-
The operator is typically deployed and run using the Helm chart. However, users can still run it outside a Kubernetes cluster and make it talk to the Kubernetes API server of a cluster by specifying path to `kubeconfig`, which can be done using the `-kubeconfig` flag.
282+
The operator is typically deployed and run using the Helm chart or Kustomize manifests. However, users can still run it outside a Kubernetes cluster and make it talk to the Kubernetes API server of a cluster by specifying path to `kubeconfig`, which can be done using the `-kubeconfig` flag.
183283

184284
The operator uses multiple workers in the `SparkApplication` controller. The number of worker threads are controlled using command-line flag `-controller-threads` which has a default value of 10.
185285

@@ -191,16 +291,6 @@ The mutating admission webhook is an **optional** component and can be enabled o
191291

192292
By default, the operator will manage custom resource objects of the managed CRD types for the whole cluster. It can be configured to manage only the custom resource objects in a specific namespace with the flag `-namespace=<namespace>`
193293

194-
## Upgrade
195-
196-
To upgrade the operator, e.g., to use a newer version container image with a new tag, run the following command with updated parameters for the Helm release:
197-
198-
```shell
199-
helm upgrade <YOUR-HELM-RELEASE-NAME> --set image.repository=org/image --set image.tag=newTag
200-
```
201-
202-
Refer to the Helm [documentation](https://helm.sh/docs/helm/helm_upgrade/) for more details on `helm upgrade`.
203-
204294
## About Spark Job Namespaces
205295

206296
The Spark Job Namespaces value defines the namespaces where `SparkApplications` can be deployed. The Helm chart value for the Spark Job Namespaces is `spark.jobNamespaces`, and its default value is `[]`. When the list of namespaces is empty the Helm chart will create a service account in the namespace where the spark-operator is deployed.
@@ -211,7 +301,7 @@ The Spark Operator uses the Spark Job Namespace to identify and filter relevant
211301

212302
## About the Service Account for Driver Pods
213303

214-
A Spark driver pod need a Kubernetes service account in the pod's namespace that has permissions to create, get, list, and delete executor pods, and create a Kubernetes headless service for the driver. The driver will fail and exit without the service account, unless the default service account in the pod's namespace has the needed permissions. To submit and run a `SparkApplication` in a namespace, please make sure there is a service account with the permissions in the namespace and set `.spec.driver.serviceAccount` to the name of the service account. Please refer to [spark-rbac.yaml](https://github.com/kubeflow/spark-operator/blob/master/config/rbac/spark-application-rbac.yaml) for an example RBAC setup that creates a driver service account named `spark-operator-spark` in the `default` namespace, with a RBAC role binding giving the service account the needed permissions.
304+
A Spark driver pod need a Kubernetes service account in the pod's namespace that has permissions to create, get, list, and delete executor pods, and create a Kubernetes headless service for the driver. The driver will fail and exit without the service account, unless the default service account in the pod's namespace has the needed permissions. To submit and run a `SparkApplication` in a namespace, please make sure there is a service account with the permissions in the namespace and set `.spec.driver.serviceAccount` to the name of the service account. Please refer to [spark-application-rbac.yaml](https://github.com/kubeflow/spark-operator/blob/master/config/spark-rbac/spark-application-rbac.yaml) for an example RBAC setup that creates a driver service account named `spark-operator-spark` in the `default` namespace, with a RBAC role binding giving the service account the needed permissions.
215305

216306
## About the Service Account for Executor Pods
217307

@@ -324,4 +414,4 @@ helm install my-release spark-operator/spark-operator \
324414
--set "spark.jobNamespaces={spark}" \
325415
--set webhook.enable=true \
326416
--set webhook.port=443
327-
```
417+
```

0 commit comments

Comments
 (0)