docs(kafka): refactor of kafka configuration content (#12298)

Signed-off-by: prmellor <pmellor@redhat.com>
Signed-off-by: PaulRMellor <47596553+PaulRMellor@users.noreply.github.com>

Author: PaulRMellor

documentation/assemblies/configuring/assembly-config.adoc

@@ -48,11 +48,24 @@ This provides a convenient mechanism for resources to be labeled as required.
 //How to access examples
 include::../../modules/configuring/con-config-examples.adoc[leveloffset=+1]
 
-//`Kafka` resource config
+//`Kafka` resource core config
 include::../../modules/configuring/con-config-kafka-kraft.adoc[leveloffset=+1]
-//setting static broker limits
+// Securing client connections
+include::../../modules/configuring/con-config-kafka-securing-client-connections.adoc[leveloffset=+2]
+// Configuring authorization
+include::../../modules/configuring/con-config-kafka-authorization.adoc[leveloffset=+2]
+// Configuring Kafka broker behavior
+include::../../modules/configuring/con-config-kafka-broker-configuration.adoc[leveloffset=+2]
+// Automating Kafka resource distribution and storage management
+include::../../modules/configuring/con-kafka-cruise-control.adoc[leveloffset=+2]
+// Capturing consumer lag metrics
+include::../../modules/configuring/con-kafka-exporter.adoc[leveloffset=+2]
+// Controlling throughput and storage usage
+include::../../modules/managing/con-choosing-a-quota-plugin.adoc[leveloffset=+2]
 include::../../modules/managing/proc-setting-broker-limits.adoc[leveloffset=+2]
-//Delete kafka nodes using annotations
+// Recovering from broker storage issues
+include::../../modules/configuring/con-kafka-recovering-brokers.adoc[leveloffset=+2]
+// Deleting Kafka nodes using annotations
 include::../../modules/configuring/proc-manual-delete-pod-pvc-kafka.adoc[leveloffset=+2]
 
 //configuring node pools

documentation/assemblies/configuring/assembly-logging-configuration.adoc

@@ -5,7 +5,7 @@
 // assembly-config.adoc
 
 :_mod-docs-content-type: ASSEMBLY
-[id='external-logging_{context}']
+[id='assembly-logging-configuration-{context}']
 = Configuring logging
 
 [role="_abstract"]

documentation/modules/configuring/con-config-kafka-authorization.adoc

@@ -0,0 +1,92 @@
+:_mod-docs-content-type: CONCEPT
+
+// Module included in the following assemblies:
+//
+// assembly-config.adoc
+
+[id='con-kafka-authorization-{context}']
+= Configuring authorization
+
+[role="_abstract"]
+Control what authenticated clients can do in a Kafka cluster by enabling authorization on Kafka brokers.
+Authorization is configured in the `Kafka` custom resource and enforced by the Kafka broker.
+
+Authorization is separate from listener authentication.
+Authentication verifies a client identity.
+Authorization controls which operations the client is allowed to perform.
+
+Kafka supports the following authorization types:
+
+* `simple`
+* `custom`
+
+The `simple` authorization type uses the Kafka `StandardAuthorizer` plugin.
+
+The following examples show how to enable authorization on Kafka brokers.
+
+== Enabling simple authorization
+
+Simple authorization enables ACL-based access control using Kafka’s built-in authorizer.
+
+[source,yaml,subs="+attributes"]
+----
+spec:
+  kafka:
+    # Authorization (optional)
+    authorization:
+      type: simple
+----
+
+* `spec.kafka.authorization.type: simple` enables ACL-based authorization using the Kafka `StandardAuthorizer`.
+* Access to Kafka resources is controlled using ACL rules.
+
+== Configuring super users
+
+You can define super users that are allowed to perform all operations, regardless of ACL rules.
+Use super users for Kafka administrator identities and internal components that must always be able to operate.
+
+[source,yaml,subs="+attributes"]
+----
+spec:
+  kafka:
+    # Authorization (optional)
+    authorization:
+      type: simple
+      superUsers:
+        - CN=kafka-admin
+        - my-team-admin
+----
+
+* `spec.kafka.authorization.superUsers` defines super user identities.
+* The identity format must match the principal used by your authentication mechanism.
+
+The `CN=` prefix shown in the example applies to mTLS authentication, where the principal is derived from the certificate subject.
+Other authentication mechanisms, such as SASL-based authentication, use different principal formats.
+
+== Enabling custom authorization
+
+Custom authorization allows the use of a custom authorizer implementation.
+Use custom authorization when you need to integrate Kafka with an external authorization system.
+
+[source,yaml,subs="+attributes"]
+----
+spec:
+  kafka:
+    # Authorization (optional)
+    authorization:
+      type: custom
+      authorizerClass: com.example.CustomAuthorizer
+----
+
+* `spec.kafka.authorization.type: custom` enables a custom authorization implementation.
+* `spec.kafka.authorization.authorizerClass` specifies the fully qualified class name of the custom authorizer.
+
+Authorization configuration enables enforcement of access control on the Kafka broker.
+User identities, credentials, and ACL rules are configured separately.
+
+For step-by-step procedures and examples for configuring authorized access, see
+xref:assembly-securing-access-{context}[Securing access to a Kafka cluster].
+
+For details of authorization configuration options, see the
+link:{BookURLConfiguring}#type-Kafka-authorization-simple-reference[`KafkaAuthorizationSimple` schema reference^] and
+link:{BookURLConfiguring}#type-Kafka-authorization-custom-reference[`KafkaAuthorizationCustom` schema reference^].

documentation/modules/configuring/con-config-kafka-broker-configuration.adoc

@@ -0,0 +1,63 @@
+:_mod-docs-content-type: CONCEPT
+
+// Module included in the following assemblies:
+//
+// assembly-config.adoc
+
+[id='con-kafka-broker-configuration-{context}']
+= Configuring Kafka broker operation
+
+[role="_abstract"]
+Control how Kafka brokers operate by configuring broker properties in the `Kafka` custom resource.
+Broker configuration affects availability and performance of the cluster.
+
+Broker configuration is applied using the `spec.kafka.config` property.
+Only Kafka configuration properties that are not managed directly by Strimzi can be set.
+Strimzi validates and applies these properties during reconciliation.
+
+Use broker configuration to:
+
+* Replicate topics for high availability
+* Improve throughput or request handling
+* Optimize performance for high-latency or high-bandwidth networks
+* Control disk usage and log retention behavior
+* Reduce unnecessary rebalances or leadership changes
+
+Broker configuration options must be used in the correct combinations to meet your availability and fault-tolerance requirements.
+Kafka provides many additional broker properties for more granular configuration.
+However, certain properties are managed directly by Strimzi and cannot be set within this config property.
+For more information, see the link:{BookURLConfiguring}#type-KafkaClusterSpec-reference[`KafkaClusterSpec` schema reference^].
+
+Broker tuning is workload-dependent.
+Test changes in a non-production environment before applying them to a production cluster.
+A practical approach is to adjust configuration incrementally and monitor the impact using broker and client metrics.
+For detailed guidance on selecting and tuning broker configuration properties, see
+xref:con-broker-config-properties-{context}[Kafka broker configuration tuning].
+
+== Adding broker configuration
+
+Kafka broker configuration is defined as key-value pairs under `spec.kafka.config`.
+
+[source,yaml,subs="+attributes"]
+----
+spec:
+  kafka:
+    # Kafka configuration (recommended)
+    config:
+      offsets.topic.replication.factor: 3
+      transaction.state.log.replication.factor: 3
+      transaction.state.log.min.isr: 2
+      default.replication.factor: 3
+      min.insync.replicas: 2
+----
+
+* `spec.kafka.config` defines Kafka broker properties as key-value pairs.
+* `offsets.topic.replication.factor` sets the replication factor for the consumer offsets topic.
+* `transaction.state.log.replication.factor` sets the replication factor for the transaction state log.
+* `transaction.state.log.min.isr` sets the minimum number of in-sync replicas (ISR) required for the transaction state log.
+* `default.replication.factor` sets the default replication factor for topics that do not specify one.
+* `min.insync.replicas` sets the minimum number of in-sync replicas (ISR) required to acknowledge writes.
+
+WARNING: If you remove the `min.insync.replicas` property from `spec.kafka.config`,
+the Cluster Operator forces Kafka to fall back to the default value of `1`.
+To ensure fault tolerance, explicitly set `min.insync.replicas` to a value greater than `1`.