docs(affinity): updates the pod scheduling section to use node pools (#11803)

Signed-off-by: prmellor <pmellor@redhat.com>

Author: PaulRMellor

documentation/assemblies/configuring/assembly-scheduling.adoc

@@ -6,8 +6,12 @@
 = Configuring pod scheduling
 
 [role="_abstract"]
-To avoid performance degradation caused by resource conflicts between applications scheduled on the same Kubernetes node, you can schedule Kafka pods separately from critical workloads. 
-This can be achieved by either selecting specific nodes or dedicating a set of nodes exclusively for Kafka.
+To optimize the resilience and performance of your Kafka cluster, you can control how its pods are scheduled across Kubernetes nodes.
+Pod scheduling strategies can help you to achieve the following:
+
+* Increase fault tolerance by spreading Kafka pods across different nodes.
+* Avoid resource contention by separating pods from critical workloads.
+* Maintain resource availability by assigning Kafka pods to nodes with sufficient capacity.
 
 include::../../modules/configuring/ref-affinity.adoc[leveloffset=+1]
 

documentation/assemblies/configuring/assembly-storage.adoc

@@ -37,7 +37,8 @@ include::../../modules/configuring/con-considerations-for-data-storage.adoc[leve
 
 //KRaft storage
 include::../../modules/configuring/con-config-storage-kraft.adoc[leveloffset=+1]
-include::../../modules/configuring/proc-managing-storage-node-pools.adoc[leveloffset=+2]
-include::../../modules/configuring/proc-managing-storage-affinity-node-pools.adoc[leveloffset=+2]
+include::../../modules/configuring/con-config-storage-kraft-metadata.adoc[leveloffset=+1]
+include::../../modules/configuring/proc-managing-storage-node-pools.adoc[leveloffset=+1]
+include::../../modules/configuring/proc-managing-storage-affinity-node-pools.adoc[leveloffset=+1]
 //tiered storage
 include::../../modules/configuring/ref-storage-tiered.adoc[leveloffset=+1]

documentation/modules/configuring/con-config-storage-kraft-metadata.adoc

@@ -0,0 +1,42 @@
+// Module included in the following assemblies:
+//
+// assembly-storage.adoc
+
+[id='con-storing-metadata-log-{context}']
+= Configuring KRaft metadata log storage
+
+In KRaft mode, each node (including brokers and controllers) stores a copy of the Kafka cluster's metadata log on one of its data volumes. 
+By default, the log is stored on the volume with the lowest ID, but you can specify a different volume using the `kraftMetadata` property.
+
+For controller-only nodes, storage is exclusively for the metadata log. 
+Since the log is always stored on a single volume, using JBOD storage with multiple volumes does not improve performance or increase available disk space.
+
+In contrast, broker nodes or nodes that combine broker and controller roles can share the same volume for both the metadata log and partition replica data, optimizing disk utilization. 
+They can also use JBOD storage, where one volume is shared for the metadata log and partition replica data, while additional volumes are used solely for partition replica data.
+
+Changing the volume that stores the metadata log triggers a rolling update of the cluster nodes, involving the deletion of the old log and the creation of a new one in the specified location. 
+If `kraftMetadata` isn't specified, adding a new volume with a lower ID also prompts an update and relocation of the metadata log.
+
+.Example JBOD storage configuration using volume with ID 1 to store the KRaft metadata
+[source,yaml,subs="attributes+"]
+----
+apiVersion: {KafkaApiVersion}
+kind: KafkaNodePool
+metadata:
+  name: pool-a
+  # ...
+spec:
+  storage:
+    type: jbod
+    volumes:
+    - id: 0
+      type: persistent-claim
+      size: 100Gi
+      deleteClaim: false
+    - id: 1
+      type: persistent-claim
+      size: 100Gi
+      kraftMetadata: shared
+      deleteClaim: false
+  # ...
+----

documentation/modules/configuring/con-config-storage-kraft.adoc

@@ -3,7 +3,7 @@
 // assembly-storage.adoc
 
 [id='con-config-storage-kraft-{context}']
-= Configuring Kafka storage in KRaft mode
+= Configuring storage types
 
 [role="_abstract"]
 Use the `storage` properties of the `KafkaNodePool` custom resource to configure storage for a deployment of Kafka in KRaft mode.
@@ -208,43 +208,4 @@ Volume IDs cannot be changed once JBOD volumes are created, though you can add o
 When adding a new volume to the to the `volumes` array under an `id` which was already used in the past and removed, make sure that the previously used `PersistentVolumeClaims` have been deleted.
 
 Use Cruise Control to reassign partitions when adding or removing volumes. 
-For information on intra-broker disk balancing, see xref:con-rebalance-{context}[].
-
-[id='con-storing-metadata-log-{context}']
-== Configuring KRaft metadata log storage
-
-In KRaft mode, each node (including brokers and controllers) stores a copy of the Kafka cluster's metadata log on one of its data volumes. 
-By default, the log is stored on the volume with the lowest ID, but you can specify a different volume using the `kraftMetadata` property.
-
-For controller-only nodes, storage is exclusively for the metadata log. 
-Since the log is always stored on a single volume, using JBOD storage with multiple volumes does not improve performance or increase available disk space.
-
-In contrast, broker nodes or nodes that combine broker and controller roles can share the same volume for both the metadata log and partition replica data, optimizing disk utilization. 
-They can also use JBOD storage, where one volume is shared for the metadata log and partition replica data, while additional volumes are used solely for partition replica data.
-
-Changing the volume that stores the metadata log triggers a rolling update of the cluster nodes, involving the deletion of the old log and the creation of a new one in the specified location. 
-If `kraftMetadata` isn't specified, adding a new volume with a lower ID also prompts an update and relocation of the metadata log.
-
-.Example JBOD storage configuration using volume with ID 1 to store the KRaft metadata
-[source,yaml,subs="attributes+"]
-----
-apiVersion: {KafkaApiVersion}
-kind: KafkaNodePool
-metadata:
-  name: pool-a
-  # ...
-spec:
-  storage:
-    type: jbod
-    volumes:
-    - id: 0
-      type: persistent-claim
-      size: 100Gi
-      deleteClaim: false
-    - id: 1
-      type: persistent-claim
-      size: 100Gi
-      kraftMetadata: shared
-      deleteClaim: false
-  # ...
-----
+For information on intra-broker disk balancing, see xref:con-rebalance-{context}[].

documentation/modules/configuring/proc-dedicated-nodes.adoc

@@ -3,37 +3,56 @@
 // assembly-scheduling.adoc
 
 [id='proc-dedicated-nodes-{context}']
-= Setting up dedicated nodes and scheduling pods on them
+= Configuring pod scheduling for dedicated nodes
+
+[role="_abstract"]
+You can dedicate a set of worker nodes exclusively to your Kafka brokers so that no other applications can compete with Kafka for resources on those nodes.
+
+To configure dedicated worker nodes for Kafka pods in a specific pool, combine the following:
+
+Taints:: Apply taints to worker nodes to prevent other pods from being scheduled on them.
+Tolerations:: Apply tolerations to Kafka pods to allow them to be scheduled on tainted nodes.
+Affinity:: Apply affinity to Kafka pods to schedule them on specifically labeled nodes.
+
+This procedure provides configuration examples for the cluster-wide `Kafka` resource and the pool-specific `KafkaNodePool` resource.
+
+NOTE: If you configure `KafkaNodePool.spec.template`, its settings replace `Kafka.spec.kafka.template` for that node pool.
+Properties are not merged. 
+For more information, see xref:affinity-{context}[Scheduling strategies].
 
 .Prerequisites
 
-* A Kubernetes cluster
-* A running Cluster Operator
+* xref:deploying-cluster-operator-str[The Cluster Operator must be deployed.] 
+* Dedicated worker nodes without scheduled workloads
 
 .Procedure
 
-. Select the nodes which should be used as dedicated.
-. Make sure there are no workloads scheduled on these nodes.
-. Set the taints on the selected nodes:
+. Taint and label the dedicated worker nodes to prevent other pods from being scheduled on them and to identify them when scheduling the Kafka pods:
 +
-This can be done using `kubectl taint`:
-[source,shell,subs=+quotes]
-kubectl taint node _NAME-OF-NODE_ dedicated=Kafka:NoSchedule
-+
-. Additionally, add a label to the selected nodes as well.
+[source,shell]
+----
+kubectl taint node <name_of_node> dedicated=kafka:NoSchedule
+kubectl label node <name_of_node> dedicated=kafka
+----
+
+. Configure `tolerations` and `nodeAffinity` in either your `Kafka` or `KafkaNodePool` custom resource to match the taint and label.
 +
-This can be done using `kubectl label`:
-[source,shell,subs=+quotes]
-kubectl label node _NAME-OF-NODE_ dedicated=Kafka
+--
+* To set a cluster-wide rule, edit the `affinity` property in `spec.kafka.template.pod` of your `Kafka` resource.
+* To set a pool-specific rule, edit the `affinity` property in `spec.template.pod` of your `KafkaNodePool` resource.
+--
 +
-. Edit the `affinity` and `tolerations` properties in the resource specifying the cluster deployment.
+In both cases, use `nodeSelectorTerms` with `matchExpressions` to specify the key-value label of the nodes you want to schedule pods on.
 +
-For example:
+This example applies a rule to a `Kafka` resource that assigns all its broker pods to run only on nodes that have been tainted and labeled with `dedicated=kafka`.
 +
+.Example cluster-wide affinity configuration for dedicated nodes
 [source,yaml,subs=attributes+]
 ----
-apiVersion: {KafkaApiVersion}
+apiVersion: kafka.strimzi.io/v1beta2
 kind: Kafka
+metadata:
+  name: my-cluster
 spec:
   kafka:
     # ...
@@ -42,7 +61,7 @@ spec:
         tolerations:
           - key: "dedicated"
             operator: "Equal"
-            value: "Kafka"
+            value: "kafka"
             effect: "NoSchedule"
         affinity:
           nodeAffinity:
@@ -52,12 +71,42 @@ spec:
                 - key: dedicated
                   operator: In
                   values:
-                  - Kafka
-    # ...
+                  - kafka
+  # ...
 ----
-
-. Create or update the resource.
 +
-This can be done using `kubectl apply`:
-[source,shell,subs=+quotes]
-kubectl apply -f _<kafka_configuration_file>_
+This example applies a rule to a `KafkaNodePool` resource named `broker` that assigns its pods to run on dedicated nodes marked with `dedicated=broker-kafka`.
++
+.Example node pool-specific affinity configuration for dedicated nodes
+[source,yaml,subs=attributes+]
+----
+apiVersion: {KafkaNodePoolApiVersion}
+kind: KafkaNodePool
+metadata:
+  name: broker
+  labels:
+    strimzi.io/cluster: my-cluster
+spec:
+  replicas: 3
+  roles:
+    - broker
+  template:
+    pod:
+      tolerations:
+        - key: "dedicated"
+          operator: "Equal"
+          value: "kafka"
+          effect: "NoSchedule"
+      affinity:
+        nodeAffinity:
+          requiredDuringSchedulingIgnoredDuringExecution:
+            nodeSelectorTerms:
+            - matchExpressions:
+              - key: dedicated
+                operator: In
+                values:
+                - broker-kafka
+  # ...
+----
+
+. Apply the changes to your custom resource configuration.

documentation/modules/configuring/proc-scheduling-based-on-other-pods.adoc

@@ -3,27 +3,43 @@
 // assembly-scheduling.adoc
 
 [id='configuring-pod-anti-affinity-in-kafka-components-{context}']
-= Configuring pod anti-affinity in Kafka components
+= Configuring pod anti-affinity against other workloads
 
-Pod anti-affinity configuration helps with the stability and performance of Kafka brokers. By using `podAntiAffinity`, Kubernetes will not schedule Kafka brokers on the same nodes as other workloads. 
-Typically, you want to avoid Kafka running on the same worker node as other network or storage intensive applications such as databases, storage or other messaging platforms.
+[role="_abstract"]
+To improve stability and performance, you can prevent Kafka pods from running on the same worker nodes as other resource-intensive applications, such as databases. 
+Configure `podAntiAffinity` so that these workloads are scheduled on separate nodes.
+
+This procedure provides configuration examples for the cluster-wide `Kafka` resource and the pool-specific `KafkaNodePool` resource.
+
+NOTE: If you configure `KafkaNodePool.spec.template`, its settings replace `Kafka.spec.kafka.template` for that node pool.
+Properties are not merged. 
+For more information, see xref:affinity-{context}[Scheduling strategies].
 
 .Prerequisites
 
-* A Kubernetes cluster
-* A running Cluster Operator
+* xref:deploying-cluster-operator-str[The Cluster Operator must be deployed.] 
+* The other workloads in your cluster use consistent labels.
 
 .Procedure
 
-. Edit the `affinity` property in the resource specifying the cluster deployment.
-Use labels to specify the pods which should not be scheduled on the same nodes.
-The `topologyKey` should be set to `kubernetes.io/hostname` to specify that the selected pods should not be scheduled on nodes with the same hostname.
-For example:
+. Configure `podAntiAffinity` in either the `Kafka` or `KafkaNodePool` resource.
 +
-[source,yaml,subs=attributes+]
+--
+* To set a cluster-wide rule, edit the `affinity` property in `spec.kafka.template.pod` of your `Kafka` resource. 
+* To set a pool-specific rule, edit the `affinity` property in `spec.template.pod` of your `KafkaNodePool` resource. 
+--
++
+In both cases, use a `labelSelector` to identify the application pods you want to keep separate from your Kafka pods and set the `topologyKey` to `"kubernetes.io/hostname"` to prevent pods from being placed on the same host.
++
+This example applies a rule to a `Kafka` resource named `my-cluster` that prevents any of its broker pods from running on the same node as pods labeled `postgresql` and `mongodb`.
++
+.Example cluster-wide anti-affinity configuration
+[source,yaml,subs="+attributes"]
 ----
 apiVersion: {KafkaApiVersion}
 kind: Kafka
+metadata:
+  name: my-cluster
 spec:
   kafka:
     # ...
@@ -37,14 +53,41 @@ spec:
                     - key: application
                       operator: In
                       values:
-                        - postgresql
-                        - mongodb
+                      - postgresql
+                      - mongodb
                 topologyKey: "kubernetes.io/hostname"
-    # ...
+  # ...
 ----
-
-. Create or update the resource.
 +
-This can be done using `kubectl apply`:
-[source,shell,subs=+quotes]
-kubectl apply -f _<kafka_configuration_file>_
+This example applies a rule to a `KafkaNodePool` resource named `broker` that prevents pods from that pool from running on the same node as pods labeled `postgresql` and `mongodb`.
++
+.Example node pool-specific anti-affinity configuration
+[source,yaml,subs=attributes+]
+----
+apiVersion: {KafkaNodePoolApiVersion}
+kind: KafkaNodePool
+metadata:
+  name: broker
+  labels:
+    strimzi.io/cluster: my-cluster
+spec:
+  replicas: 3
+  roles:
+    - broker
+  template:
+    pod:
+      affinity:
+        podAntiAffinity:
+          requiredDuringSchedulingIgnoredDuringExecution:
+            - labelSelector:
+                matchExpressions:
+                  - key: application
+                    operator: In
+                    values:
+                      - postgresql
+                      - mongodb
+              topologyKey: "kubernetes.io/hostname"
+  # ...
+----
+
+. Apply the changes to your custom resource configuration.