Upgrade v1beta2 to v1 - Kafka Listener Authentication #12361

user222635 · 2026-01-28T14:42:24Z

user222635
Jan 28, 2026

Hi! I encounter an error when I'm trying to upgrade the Kafka CR from v1beta2 to v1.

I'm currently using Strimzi Kafka Operator 0.49.0 and Kafka 4.1.0, with the current broker configuration:

apiVersion: kafka.strimzi.io/v1beta2
kind: Kafka
metadata:
  name: kafka
  namespace: kafka
spec:
  entityOperator:
    template:
      topicOperatorContainer:
        env:
          - name: STRIMZI_USE_FINALIZERS
            value: 'false'
    topicOperator:
      jvmOptions:
        '-XX':
          MaxRAMPercentage: '60'
      livenessProbe:
        failureThreshold: 5
        initialDelaySeconds: 30
        timeoutSeconds: 10
      logging:
        type: external
        valueFrom:
          configMapKeyRef:
            key: log4j2.properties
            name: kafka-log4j-config
      readinessProbe:
        failureThreshold: 5
        initialDelaySeconds: 30
        timeoutSeconds: 10
      resources:
        limits:
          cpu: 1000m
          memory: 384Mi
        requests:
          cpu: 100m
          memory: 256Mi
      startupProbe:
        timeoutSeconds: 10
    userOperator:
      livenessProbe:
        failureThreshold: 5
        initialDelaySeconds: 30
        timeoutSeconds: 10
      logging:
        type: external
        valueFrom:
          configMapKeyRef:
            key: log4j2.properties
            name: kafka-log4j-config
      readinessProbe:
        failureThreshold: 5
        initialDelaySeconds: 30
        timeoutSeconds: 10
      resources:
        limits:
          cpu: 1000m
          memory: 384Mi
        requests:
          cpu: 100m
          memory: 256Mi
  kafka:
    config:
      auto.create.topics.enable: 'false'
      compression.type: snappy
      default.replication.factor: 1
      log.segment.bytes: 104857600
      min.insync.replicas: 1
      num.io.threads: 25
      num.network.threads: 10
      num.partitions: 1
      offsets.topic.replication.factor: 1
      transaction.state.log.min.isr: 1
      transaction.state.log.replication.factor: 1
    image: quay.io/strimzi/kafka:0.49.0-kafka-4.1.0
    jvmOptions:
      '-XX':
        MaxRAMPercentage: '60'
    listeners:
      - configuration:
          useServiceDnsDomain: true
        name: tls
        port: 9093
        tls: true
        type: internal
      - authentication:
          enableOauthBearer: true
          jwksEndpointUri: https://url/protocol/openid-connect/certs
          jwksExpirySeconds: 360
          jwksMinRefreshPauseSeconds: 1
          jwksRefreshSeconds: 300
          maxSecondsWithoutReauthentication: 3600
          type: oauth
          userNameClaim: preferred_username
          validIssuerUri: https://url
        configuration:
          useServiceDnsDomain: true
        name: iam
        port: 9098
        tls: false
        type: internal
    livenessProbe:
      initialDelaySeconds: 60
      timeoutSeconds: 5
    logging:
      type: external
      valueFrom:
        configMapKeyRef:
          key: log4j2.properties
          name: kafka-log4j-config
    metricsConfig:
      type: jmxPrometheusExporter
      valueFrom:
        configMapKeyRef:
          key: kafka-metrics-config.yml
          name: kafka-metrics
    readinessProbe:
      initialDelaySeconds: 60
      timeoutSeconds: 5
    version: 4.1.0
  kafkaExporter:
    groupRegex: .*
    image: quay.io/strimzi/kafka:0.49.0-kafka-4.1.0
    topicRegex: .*

Using this configuration the Kafka broker and Strimzi Operator are working just fine, also without any errors in logs.

Now I'm trying to upgrade the Kafka CR to the v1, following this example from the documentation. Here is how my broker configuration looks:

apiVersion: kafka.strimzi.io/v1
kind: Kafka
metadata:
  name: kafka
  namespace: kafka
spec:
  entityOperator:
    template:
      topicOperatorContainer:
        env:
          - name: STRIMZI_USE_FINALIZERS
            value: 'false'
    topicOperator:
      jvmOptions:
        '-XX':
          MaxRAMPercentage: '60'
      livenessProbe:
        failureThreshold: 5
        initialDelaySeconds: 30
        timeoutSeconds: 10
      logging:
        type: external
        valueFrom:
          configMapKeyRef:
            key: log4j2.properties
            name: kafka-log4j-config
      readinessProbe:
        failureThreshold: 5
        initialDelaySeconds: 30
        timeoutSeconds: 10
      resources:
        limits:
          cpu: 1000m
          memory: 384Mi
        requests:
          cpu: 100m
          memory: 256Mi
      startupProbe:
        timeoutSeconds: 10
    userOperator:
      livenessProbe:
        failureThreshold: 5
        initialDelaySeconds: 30
        timeoutSeconds: 10
      logging:
        type: external
        valueFrom:
          configMapKeyRef:
            key: log4j2.properties
            name: kafka-log4j-config
      readinessProbe:
        failureThreshold: 5
        initialDelaySeconds: 30
        timeoutSeconds: 10
  kafka:
    config:
      auto.create.topics.enable: 'false'
      compression.type: snappy
      default.replication.factor: 1
      log.segment.bytes: 104857600
      min.insync.replicas: 1
      num.io.threads: 25
      num.network.threads: 10
      num.partitions: 1
      offsets.topic.replication.factor: 1
      principal.builder.class: io.strimzi.kafka.oauth.server.OAuthKafkaPrincipalBuilder
      transaction.state.log.min.isr: 1
      transaction.state.log.replication.factor: 1
    image: quay.io/strimzi/kafka:0.49.0-kafka-4.1.0
    jvmOptions:
      '-XX':
        MaxRAMPercentage: '60'
    listeners:
      - configuration:
          useServiceDnsDomain: true
        name: tls
        port: 9093
        tls: true
        type: internal
      - authentication:
          listenerConfig:
            oauthbearer.sasl.jaas.config: |
              org.apache.kafka.common.security.oauthbearer.OAuthBearerLoginModule
              required
              oauth.valid.issuer.uri="https://url/auth/realms/fts"
              oauth.jwks.endpoint.uri="https://url/protocol/openid-connect/certs"
              oauth.username.claim="preferred_username"
              oauth.ssl.truststore.location="/mnt/oauth-certs/ca.crt"
              oauth.ssl.truststore.type="PEM";
            oauthbearer.sasl.server.callback.handler.class: io.strimzi.kafka.oauth.server.JaasServerOauthValidatorCallbackHandler
            sasl.enabled.mechanisms: OAUTHBEARER
          sasl: true
          type: custom
        configuration:
          useServiceDnsDomain: true
        name: iam
        port: 9098
        tls: false
        type: internal
    livenessProbe:
      initialDelaySeconds: 60
      timeoutSeconds: 5
    logging:
      type: external
      valueFrom:
        configMapKeyRef:
          key: log4j2.properties
          name: kafka-log4j-config
    metricsConfig:
      type: jmxPrometheusExporter
      valueFrom:
        configMapKeyRef:
          key: kafka-metrics-config.yml
          name: kafka-metrics
    readinessProbe:
      initialDelaySeconds: 60
      timeoutSeconds: 5
    template:
      kafkaContainer:
        volumeMounts:
          - mountPath: /mnt/oauth-certs
            name: certificates
      pod:
        volumes:
          - name: certificates
            secret:
              secretName: kafka-cluster-ca-cert
    version: 4.1.0
  kafkaExporter:
    groupRegex: .*
    image: quay.io/strimzi/kafka:0.49.0-kafka-4.1.0
    topicRegex: .*
status:
  autoRebalance:
    lastTransitionTime: '2026-01-26T12:08:47.807736762Z'
    state: Idle
  clusterId: 3bV6k2TNQjqbODi-A9mt_w
  conditions:
    - lastTransitionTime: '2026-01-28T12:39:10.474023516Z'
      message: Pod is unschedulable or is not starting
      reason: FatalProblem
      status: 'True'
      type: NotReady
  kafkaMetadataVersion: 4.1-IV1
  kafkaNodePools:
    - name: controller-kafka
    - name: kafka
  kafkaVersion: 4.1.0
  observedGeneration: 14
  operatorLastSuccessfulVersion: 0.49.0

I tried the following methods:

By specifying the trustore from a strimzi generated certificate

oauth.ssl.truststore.location="/mnt/oauth-certs/ca.crt"
oauth.ssl.truststore.type="PEM";

By specifying the trustore from a generated certificate by us

oauth.ssl.truststore.location="/mnt/oauth-certs/tls.crt"
oauth.ssl.truststore.type="PEM";

Without specifying the trustore at all

In every case I have the following error in the Kafka Broker logs:

2026-01-28 13:39:22 INFO  [SECURITY] [main] SocketServer:69 - [SocketServer listenerType=BROKER, nodeId=1] Created data-plane acceptor and processors for endpoint : ListenerName(REPLICATION-9091)
    2026-01-28 13:39:22 INFO  [SECURITY] [main] ConnectionQuotas:69 - Updated connection-accept-rate max connection creation rate to 2147483647
    2026-01-28 13:39:22 INFO  [SECURITY] [main] SocketServer:69 - [SocketServer listenerType=BROKER, nodeId=1] Created data-plane acceptor and processors for endpoint : ListenerName(TLS-9093)
    2026-01-28 13:39:22 INFO  [SECURITY] [main] ConnectionQuotas:69 - Updated connection-accept-rate max connection creation rate to 2147483647
    2026-01-28 13:39:23 INFO  [main] JWTSignatureValidator:410 - JWKS keys change detected. Keys updated.
2026-01-28 13:39:23 ERROR [SECURITY] [main] OAuthBearerLoginModule:319 - No principal name in JWT claim: sub
    java.io.IOException: No principal name in JWT claim: sub
	at org.apache.kafka.common.security.oauthbearer.internals.unsecured.OAuthBearerUnsecuredLoginCallbackHandler.handle(OAuthBearerUnsecuredLoginCallbackHandler.java:163) ~[kafka-clients-4.1.0.jar:?]
	at org.apache.kafka.common.security.oauthbearer.OAuthBearerLoginModule.identifyToken(OAuthBearerLoginModule.java:317) ~[kafka-clients-4.1.0.jar:?]
	at org.apache.kafka.common.security.oauthbearer.OAuthBearerLoginModule.login(OAuthBearerLoginModule.java:302) ~[kafka-clients-4.1.0.jar:?]
	at java.base/javax.security.auth.login.LoginContext.invoke(Unknown Source) ~[?:?]
	at java.base/javax.security.auth.login.LoginContext$4.run(Unknown Source) ~[?:?]
	at java.base/javax.security.auth.login.LoginContext$4.run(Unknown Source) ~[?:?]
	at java.base/java.security.AccessController.doPrivileged(Unknown Source) ~[?:?]
	at java.base/javax.security.auth.login.LoginContext.invokePriv(Unknown Source) ~[?:?]
	at java.base/javax.security.auth.login.LoginContext.login(Unknown Source) ~[?:?]
	at org.apache.kafka.common.security.authenticator.AbstractLogin.login(AbstractLogin.java:61) ~[kafka-clients-4.1.0.jar:?]
	at org.apache.kafka.common.security.authenticator.LoginManager.<init>(LoginManager.java:69) ~[kafka-clients-4.1.0.jar:?]
	at org.apache.kafka.common.security.authenticator.LoginManager.acquireLoginManager(LoginManager.java:116) ~[kafka-clients-4.1.0.jar:?]
	at org.apache.kafka.common.network.SaslChannelBuilder.configure(SaslChannelBuilder.java:170) ~[kafka-clients-4.1.0.jar:?]
	at org.apache.kafka.common.network.ChannelBuilders.create(ChannelBuilders.java:188) ~[kafka-clients-4.1.0.jar:?]
	at org.apache.kafka.common.network.ChannelBuilders.serverChannelBuilder(ChannelBuilders.java:105) ~[kafka-clients-4.1.0.jar:?]
	at kafka.network.Processor.<init>(SocketServer.scala:876) ~[kafka_2.13-4.1.0.jar:?]
	at kafka.network.Acceptor.newProcessor(SocketServer.scala:784) ~[kafka_2.13-4.1.0.jar:?]
	at kafka.network.Acceptor.$anonfun$addProcessors$1(SocketServer.scala:750) ~[kafka_2.13-4.1.0.jar:?]
	at scala.collection.immutable.Range.foreach$mVc$sp(Range.scala:192) ~[scala-library-2.13.16.jar:?]
	at kafka.network.Acceptor.addProcessors(SocketServer.scala:749) ~[kafka_2.13-4.1.0.jar:?]
	at kafka.network.DataPlaneAcceptor.configure(SocketServer.scala:467) ~[kafka_2.13-4.1.0.jar:?]
	at kafka.network.SocketServer.createDataPlaneAcceptorAndProcessors(SocketServer.scala:222) ~[kafka_2.13-4.1.0.jar:?]
	at kafka.network.SocketServer.$anonfun$new$16(SocketServer.scala:148) ~[kafka_2.13-4.1.0.jar:?]
	at kafka.network.SocketServer.$anonfun$new$16$adapted(SocketServer.scala:148) ~[kafka_2.13-4.1.0.jar:?]
	at scala.collection.IterableOnceOps.foreach(IterableOnce.scala:619) ~[scala-library-2.13.16.jar:?]
	at scala.collection.IterableOnceOps.foreach$(IterableOnce.scala:617) ~[scala-library-2.13.16.jar:?]
	at scala.collection.AbstractIterable.foreach(Iterable.scala:935) ~[scala-library-2.13.16.jar:?]
	at kafka.network.SocketServer.<init>(SocketServer.scala:148) ~[kafka_2.13-4.1.0.jar:?]
	at kafka.server.BrokerServer.startup(BrokerServer.scala:281) ~[kafka_2.13-4.1.0.jar:?]
	at kafka.server.KafkaRaftServer.$anonfun$startup$2(KafkaRaftServer.scala:95) ~[kafka_2.13-4.1.0.jar:?]
	at kafka.server.KafkaRaftServer.$anonfun$startup$2$adapted(KafkaRaftServer.scala:95) ~[kafka_2.13-4.1.0.jar:?]
	at scala.Option.foreach(Option.scala:437) [scala-library-2.13.16.jar:?]
	at kafka.server.KafkaRaftServer.startup(KafkaRaftServer.scala:95) [kafka_2.13-4.1.0.jar:?]
	at kafka.Kafka$.main(Kafka.scala:97) [kafka_2.13-4.1.0.jar:?]
	at kafka.Kafka.main(Kafka.scala) [kafka_2.13-4.1.0.jar:?]
Caused by: org.apache.kafka.common.security.oauthbearer.internals.unsecured.OAuthBearerConfigException: No principal name in JWT claim: sub
	at org.apache.kafka.common.security.oauthbearer.internals.unsecured.OAuthBearerUnsecuredLoginCallbackHandler.handleTokenCallback(OAuthBearerUnsecuredLoginCallbackHandler.java:217) ~[kafka-clients-4.1.0.jar:?]
	at org.apache.kafka.common.security.oauthbearer.internals.unsecured.OAuthBearerUnsecuredLoginCallbackHandler.handle(OAuthBearerUnsecuredLoginCallbackHandler.java:161) ~[kafka-clients-4.1.0.jar:?]
	... 34 more
Caused by: org.apache.kafka.common.security.oauthbearer.internals.unsecured.OAuthBearerIllegalTokenException: No principal name in JWT claim: sub
	at org.apache.kafka.common.security.oauthbearer.internals.unsecured.OAuthBearerUnsecuredJws.<init>(OAuthBearerUnsecuredJws.java:108) ~[kafka-clients-4.1.0.jar:?]
	at org.apache.kafka.common.security.oauthbearer.internals.unsecured.OAuthBearerUnsecuredLoginCallbackHandler.handleTokenCallback(OAuthBearerUnsecuredLoginCallbackHandler.java:209) ~[kafka-clients-4.1.0.jar:?]
	at org.apache.kafka.common.security.oauthbearer.internals.unsecured.OAuthBearerUnsecuredLoginCallbackHandler.handle(OAuthBearerUnsecuredLoginCallbackHandler.java:161) ~[kafka-clients-4.1.0.jar:?]
	... 34 more
2026-01-28 13:39:23 INFO  [main] BrokerServer:69 - [BrokerServer id=1] Transition from STARTING to STARTED
2026-01-28 13:39:23 ERROR [main] BrokerServer:85 - [BrokerServer id=1] Fatal error during broker startup. Prepare to shutdown
org.apache.kafka.common.KafkaException: javax.security.auth.login.LoginException: An internal error occurred while retrieving token from callback handler
	at org.apache.kafka.common.network.SaslChannelBuilder.configure(SaslChannelBuilder.java:184) ~[kafka-clients-4.1.0.jar:?]
	at org.apache.kafka.common.network.ChannelBuilders.create(ChannelBuilders.java:188) ~[kafka-clients-4.1.0.jar:?]
	at org.apache.kafka.common.network.ChannelBuilders.serverChannelBuilder(ChannelBuilders.java:105) ~[kafka-clients-4.1.0.jar:?]
	at kafka.network.Processor.<init>(SocketServer.scala:876) ~[kafka_2.13-4.1.0.jar:?]
	at kafka.network.Acceptor.newProcessor(SocketServer.scala:784) ~[kafka_2.13-4.1.0.jar:?]
	at kafka.network.Acceptor.$anonfun$addProcessors$1(SocketServer.scala:750) ~[kafka_2.13-4.1.0.jar:?]
	at scala.collection.immutable.Range.foreach$mVc$sp(Range.scala:192) ~[scala-library-2.13.16.jar:?]
	at kafka.network.Acceptor.addProcessors(SocketServer.scala:749) ~[kafka_2.13-4.1.0.jar:?]
	at kafka.network.DataPlaneAcceptor.configure(SocketServer.scala:467) ~[kafka_2.13-4.1.0.jar:?]
	at kafka.network.SocketServer.createDataPlaneAcceptorAndProcessors(SocketServer.scala:222) ~[kafka_2.13-4.1.0.jar:?]
	at kafka.network.SocketServer.$anonfun$new$16(SocketServer.scala:148) ~[kafka_2.13-4.1.0.jar:?]
	at kafka.network.SocketServer.$anonfun$new$16$adapted(SocketServer.scala:148) ~[kafka_2.13-4.1.0.jar:?]
	at scala.collection.IterableOnceOps.foreach(IterableOnce.scala:619) ~[scala-library-2.13.16.jar:?]
	at scala.collection.IterableOnceOps.foreach$(IterableOnce.scala:617) ~[scala-library-2.13.16.jar:?]
	at scala.collection.AbstractIterable.foreach(Iterable.scala:935) ~[scala-library-2.13.16.jar:?]
	at kafka.network.SocketServer.<init>(SocketServer.scala:148) ~[kafka_2.13-4.1.0.jar:?]
	at kafka.server.BrokerServer.startup(BrokerServer.scala:281) ~[kafka_2.13-4.1.0.jar:?]
	at kafka.server.KafkaRaftServer.$anonfun$startup$2(KafkaRaftServer.scala:95) ~[kafka_2.13-4.1.0.jar:?]
	at kafka.server.KafkaRaftServer.$anonfun$startup$2$adapted(KafkaRaftServer.scala:95) ~[kafka_2.13-4.1.0.jar:?]
	at scala.Option.foreach(Option.scala:437) [scala-library-2.13.16.jar:?]
	at kafka.server.KafkaRaftServer.startup(KafkaRaftServer.scala:95) [kafka_2.13-4.1.0.jar:?]
	at kafka.Kafka$.main(Kafka.scala:97) [kafka_2.13-4.1.0.jar:?]
	at kafka.Kafka.main(Kafka.scala) [kafka_2.13-4.1.0.jar:?]
Caused by: javax.security.auth.login.LoginException: An internal error occurred while retrieving token from callback handler
	at org.apache.kafka.common.security.oauthbearer.OAuthBearerLoginModule.identifyToken(OAuthBearerLoginModule.java:320) ~[kafka-clients-4.1.0.jar:?]
	at org.apache.kafka.common.security.oauthbearer.OAuthBearerLoginModule.login(OAuthBearerLoginModule.java:302) ~[kafka-clients-4.1.0.jar:?]
	at java.base/javax.security.auth.login.LoginContext.invoke(Unknown Source) ~[?:?]
	at java.base/javax.security.auth.login.LoginContext$4.run(Unknown Source) ~[?:?]
	at java.base/javax.security.auth.login.LoginContext$4.run(Unknown Source) ~[?:?]
	at java.base/java.security.AccessController.doPrivileged(Unknown Source) ~[?:?]
	at java.base/javax.security.auth.login.LoginContext.invokePriv(Unknown Source) ~[?:?]
	at java.base/javax.security.auth.login.LoginContext.login(Unknown Source) ~[?:?]
	at org.apache.kafka.common.security.authenticator.AbstractLogin.login(AbstractLogin.java:61) ~[kafka-clients-4.1.0.jar:?]
	at org.apache.kafka.common.security.authenticator.LoginManager.<init>(LoginManager.java:69) ~[kafka-clients-4.1.0.jar:?]
	at org.apache.kafka.common.security.authenticator.LoginManager.acquireLoginManager(LoginManager.java:116) ~[kafka-clients-4.1.0.jar:?]
	at org.apache.kafka.common.network.SaslChannelBuilder.configure(SaslChannelBuilder.java:170) ~[kafka-clients-4.1.0.jar:?]
	... 22 more
2026-01-28 13:39:23 INFO  [main] BrokerServer:69 - [BrokerServer id=1] Transition from STARTED to SHUTTING_DOWN
2026-01-28 13:39:23 INFO  [main] BrokerServer:69 - [BrokerServer id=1] shutting down
2026-01-28 13:39:23 INFO  [broker-1-lifecycle-manager-event-handler] BrokerLifecycleManager:69 - [BrokerLifecycleManager id=1] Skipping controlled shutdown because we are in state NOT_RUNNING.
2026-01-28 13:39:23 INFO  [broker-1-lifecycle-manager-event-handler] KafkaEventQueue:498 - [BrokerLifecycleManager id=1] beginShutdown: shutting down event queue.
2026-01-28 13:39:23 INFO  [broker-1-lifecycle-manager-event-handler] BrokerLifecycleManager:69 - [BrokerLifecycleManager id=1] Transitioning from NOT_RUNNING to SHUTTING_DOWN.
2026-01-28 13:39:23 INFO  [main] NodeToControllerRequestThread:91 - [broker-1-to-controller-forwarding-channel-manager]: Shutting down
2026-01-28 13:39:23 INFO  [broker-1-to-controller-forwarding-channel-manager] NodeToControllerRequestThread:148 - [broker-1-to-controller-forwarding-channel-manager]: Stopped
2026-01-28 13:39:23 INFO  [main] NodeToControllerRequestThread:110 - [broker-1-to-controller-forwarding-channel-manager]: Shutdown completed
2026-01-28 13:39:23 INFO  [main] NodeToControllerChannelManagerImpl:69 - Node to controller channel manager for forwarding shutdown
2026-01-28 13:39:23 INFO  [main] LogManager:69 - Shutting down.
2026-01-28 13:39:23 INFO  [main] LogManager:69 - Shutdown complete.
2026-01-28 13:39:23 INFO  [main] ClientQuotaManager$ThrottledChannelReaper:91 - [broker-1-ThrottledChannelReaper-Fetch]: Shutting down
2026-01-28 13:39:23 INFO  [broker-1-ThrottledChannelReaper-Fetch] ClientQuotaManager$ThrottledChannelReaper:148 - [broker-1-ThrottledChannelReaper-Fetch]: Stopped
2026-01-28 13:39:23 INFO  [main] ClientQuotaManager$ThrottledChannelReaper:110 - [broker-1-ThrottledChannelReaper-Fetch]: Shutdown completed
2026-01-28 13:39:23 INFO  [main] ClientQuotaManager$ThrottledChannelReaper:91 - [broker-1-ThrottledChannelReaper-Produce]: Shutting down
2026-01-28 13:39:23 INFO  [broker-1-ThrottledChannelReaper-Produce] ClientQuotaManager$ThrottledChannelReaper:148 - [broker-1-ThrottledChannelReaper-Produce]: Stopped
2026-01-28 13:39:23 INFO  [main] ClientQuotaManager$ThrottledChannelReaper:110 - [broker-1-ThrottledChannelReaper-Produce]: Shutdown completed
2026-01-28 13:39:23 INFO  [main] ClientQuotaManager$ThrottledChannelReaper:91 - [broker-1-ThrottledChannelReaper-Request]: Shutting down
2026-01-28 13:39:23 INFO  [broker-1-ThrottledChannelReaper-Request] ClientQuotaManager$ThrottledChannelReaper:148 - [broker-1-ThrottledChannelReaper-Request]: Stopped
2026-01-28 13:39:23 INFO  [main] ClientQuotaManager$ThrottledChannelReaper:110 - [broker-1-ThrottledChannelReaper-Request]: Shutdown completed
2026-01-28 13:39:23 INFO  [main] ClientQuotaManager$ThrottledChannelReaper:91 - [broker-1-ThrottledChannelReaper-ControllerMutation]: Shutting down
2026-01-28 13:39:23 INFO  [broker-1-ThrottledChannelReaper-ControllerMutation] ClientQuotaManager$ThrottledChannelReaper:148 - [broker-1-ThrottledChannelReaper-ControllerMutation]: Stopped
2026-01-28 13:39:23 INFO  [main] ClientQuotaManager$ThrottledChannelReaper:110 - [broker-1-ThrottledChannelReaper-ControllerMutation]: Shutdown completed
2026-01-28 13:39:23 INFO  [main] BrokerTopicStats:260 - Broker and topic stats closed
2026-01-28 13:39:23 INFO  [main] KafkaEventQueue:520 - [BrokerLifecycleManager id=1] closed event queue.
2026-01-28 13:39:23 INFO  [main] SystemTimerReaper$Reaper:91 - [client-metrics-reaper]: Shutting down
2026-01-28 13:39:23 INFO  [client-metrics-reaper] SystemTimerReaper$Reaper:148 - [client-metrics-reaper]: Stopped
2026-01-28 13:39:23 INFO  [main] SystemTimerReaper$Reaper:110 - [client-metrics-reaper]: Shutdown completed
2026-01-28 13:39:23 INFO  [main] SharedServer:69 - [SharedServer id=1] Stopping SharedServer
2026-01-28 13:39:23 INFO  [main] KafkaEventQueue:498 - [MetadataLoader id=1] beginShutdown: shutting down event queue.
2026-01-28 13:39:23 INFO  [kafka-1-metadata-loader-event-handler] KafkaEventQueue:498 - [SnapshotGenerator id=1] close: shutting down event queue.
2026-01-28 13:39:23 INFO  [kafka-1-metadata-loader-event-handler] KafkaEventQueue:520 - [SnapshotGenerator id=1] closed event queue.
2026-01-28 13:39:23 INFO  [main] KafkaEventQueue:520 - [MetadataLoader id=1] closed event queue.
2026-01-28 13:39:23 INFO  [main] KafkaEventQueue:520 - [SnapshotGenerator id=1] closed event queue.
2026-01-28 13:39:23 INFO  [main] TimingWheelExpirationService$ExpiredOperationReaper:91 - [raft-expiration-reaper]: Shutting down
2026-01-28 13:39:23 INFO  [main] TimingWheelExpirationService$ExpiredOperationReaper:110 - [raft-expiration-reaper]: Shutdown completed
2026-01-28 13:39:23 INFO  [raft-expiration-reaper] TimingWheelExpirationService$ExpiredOperationReaper:148 - [raft-expiration-reaper]: Stopped
2026-01-28 13:39:23 INFO  [main] KafkaRaftClientDriver:91 - [kafka-1-raft-io-thread]: Shutting down
2026-01-28 13:39:23 INFO  [main] KafkaRaftClient:3594 - [RaftManager id=1] Beginning graceful shutdown
2026-01-28 13:39:23 INFO  [kafka-1-raft-io-thread] KafkaRaftClient:3786 - [RaftManager id=1] Graceful shutdown completed
2026-01-28 13:39:23 INFO  [kafka-1-raft-io-thread] KafkaRaftClientDriver:77 - [RaftManager id=1] Completed graceful shutdown of RaftClient
2026-01-28 13:39:23 INFO  [kafka-1-raft-io-thread] KafkaRaftClientDriver:148 - [kafka-1-raft-io-thread]: Stopped
2026-01-28 13:39:23 INFO  [main] KafkaRaftClientDriver:110 - [kafka-1-raft-io-thread]: Shutdown completed
2026-01-28 13:39:23 INFO  [main] KafkaNetworkChannel$SendThread:91 - [kafka-1-raft-outbound-request-thread]: Shutting down
2026-01-28 13:39:23 INFO  [main] KafkaNetworkChannel$SendThread:110 - [kafka-1-raft-outbound-request-thread]: Shutdown completed
2026-01-28 13:39:23 INFO  [kafka-1-raft-outbound-request-thread] KafkaNetworkChannel$SendThread:148 - [kafka-1-raft-outbound-request-thread]: Stopped
2026-01-28 13:39:23 INFO  [main] ProducerStateManager:441 - [ProducerStateManager partition=__cluster_metadata-0] Wrote producer snapshot at offset 50701 with 0 producer ids in 3 ms.
2026-01-28 13:39:23 INFO  [main] Metrics:684 - Metrics scheduler closed
2026-01-28 13:39:23 INFO  [main] Metrics:694 - Metrics reporters closed
2026-01-28 13:39:23 INFO  [main] AppInfoParser:89 - App info kafka.server for 1 unregistered
2026-01-28 13:39:23 INFO  [main] BrokerServer:69 - [BrokerServer id=1] shut down completed
2026-01-28 13:39:23 INFO  [main] BrokerServer:69 - [BrokerServer id=1] Transition from SHUTTING_DOWN to SHUTDOWN
2026-01-28 13:39:23 ERROR [main] Kafka$:28 - Exiting Kafka due to fatal exception during startup.
org.apache.kafka.common.KafkaException: javax.security.auth.login.LoginException: An internal error occurred while retrieving token from callback handler
	at org.apache.kafka.common.network.SaslChannelBuilder.configure(SaslChannelBuilder.java:184) ~[kafka-clients-4.1.0.jar:?]
	at org.apache.kafka.common.network.ChannelBuilders.create(ChannelBuilders.java:188) ~[kafka-clients-4.1.0.jar:?]
	at org.apache.kafka.common.network.ChannelBuilders.serverChannelBuilder(ChannelBuilders.java:105) ~[kafka-clients-4.1.0.jar:?]
	at kafka.network.Processor.<init>(SocketServer.scala:876) ~[kafka_2.13-4.1.0.jar:?]
	at kafka.network.Acceptor.newProcessor(SocketServer.scala:784) ~[kafka_2.13-4.1.0.jar:?]
	at kafka.network.Acceptor.$anonfun$addProcessors$1(SocketServer.scala:750) ~[kafka_2.13-4.1.0.jar:?]
	at scala.collection.immutable.Range.foreach$mVc$sp(Range.scala:192) ~[scala-library-2.13.16.jar:?]
	at kafka.network.Acceptor.addProcessors(SocketServer.scala:749) ~[kafka_2.13-4.1.0.jar:?]
	at kafka.network.DataPlaneAcceptor.configure(SocketServer.scala:467) ~[kafka_2.13-4.1.0.jar:?]
	at kafka.network.SocketServer.createDataPlaneAcceptorAndProcessors(SocketServer.scala:222) ~[kafka_2.13-4.1.0.jar:?]
	at kafka.network.SocketServer.$anonfun$new$16(SocketServer.scala:148) ~[kafka_2.13-4.1.0.jar:?]
	at kafka.network.SocketServer.$anonfun$new$16$adapted(SocketServer.scala:148) ~[kafka_2.13-4.1.0.jar:?]
	at scala.collection.IterableOnceOps.foreach(IterableOnce.scala:619) ~[scala-library-2.13.16.jar:?]
	at scala.collection.IterableOnceOps.foreach$(IterableOnce.scala:617) ~[scala-library-2.13.16.jar:?]
	at scala.collection.AbstractIterable.foreach(Iterable.scala:935) ~[scala-library-2.13.16.jar:?]
	at kafka.network.SocketServer.<init>(SocketServer.scala:148) ~[kafka_2.13-4.1.0.jar:?]
	at kafka.server.BrokerServer.startup(BrokerServer.scala:281) ~[kafka_2.13-4.1.0.jar:?]
	at kafka.server.KafkaRaftServer.$anonfun$startup$2(KafkaRaftServer.scala:95) ~[kafka_2.13-4.1.0.jar:?]
	at kafka.server.KafkaRaftServer.$anonfun$startup$2$adapted(KafkaRaftServer.scala:95) ~[kafka_2.13-4.1.0.jar:?]
	at scala.Option.foreach(Option.scala:437) ~[scala-library-2.13.16.jar:?]
	at kafka.server.KafkaRaftServer.startup(KafkaRaftServer.scala:95) ~[kafka_2.13-4.1.0.jar:?]
	at kafka.Kafka$.main(Kafka.scala:97) [kafka_2.13-4.1.0.jar:?]
	at kafka.Kafka.main(Kafka.scala) [kafka_2.13-4.1.0.jar:?]
Caused by: javax.security.auth.login.LoginException: An internal error occurred while retrieving token from callback handler
	at org.apache.kafka.common.security.oauthbearer.OAuthBearerLoginModule.identifyToken(OAuthBearerLoginModule.java:320) ~[kafka-clients-4.1.0.jar:?]
	at org.apache.kafka.common.security.oauthbearer.OAuthBearerLoginModule.login(OAuthBearerLoginModule.java:302) ~[kafka-clients-4.1.0.jar:?]
	at java.base/javax.security.auth.login.LoginContext.invoke(Unknown Source) ~[?:?]
	at java.base/javax.security.auth.login.LoginContext$4.run(Unknown Source) ~[?:?]
	at java.base/javax.security.auth.login.LoginContext$4.run(Unknown Source) ~[?:?]
	at java.base/java.security.AccessController.doPrivileged(Unknown Source) ~[?:?]
	at java.base/javax.security.auth.login.LoginContext.invokePriv(Unknown Source) ~[?:?]
	at java.base/javax.security.auth.login.LoginContext.login(Unknown Source) ~[?:?]
	at org.apache.kafka.common.security.authenticator.AbstractLogin.login(AbstractLogin.java:61) ~[kafka-clients-4.1.0.jar:?]
	at org.apache.kafka.common.security.authenticator.LoginManager.<init>(LoginManager.java:69) ~[kafka-clients-4.1.0.jar:?]
	at org.apache.kafka.common.security.authenticator.LoginManager.acquireLoginManager(LoginManager.java:116) ~[kafka-clients-4.1.0.jar:?]
	at org.apache.kafka.common.network.SaslChannelBuilder.configure(SaslChannelBuilder.java:170) ~[kafka-clients-4.1.0.jar:?]
	... 22 more
2026-01-28 13:39:23 INFO  [JettyShutdownThread] Server:652 - Stopped oejs.Server@3e1624c7{STOPPING}[12.0.22,sto=30000]
2026-01-28 13:39:23 INFO  [JettyShutdownThread] Server:141 - Shutdown oejs.Server@3e1624c7{STOPPING}[12.0.22,sto=30000]
2026-01-28 13:39:23 INFO  [kafka-shutdown-hook] AppInfoParser:89 - App info kafka.server for 1 unregistered
2026-01-28 13:39:23 INFO  [JettyShutdownThread] AbstractConnector:381 - Stopped ServerConnector@cabac10{SSL, (ssl, http/1.1)}{0.0.0.0:8443}
2026-01-28 13:39:23 INFO  [JettyShutdownThread] AbstractConnector:381 - Stopped ServerConnector@25c53f74{HTTP/1.1, (http/1.1)}{localhost:8080}

Strimzi Operator logs:

2026-01-28 13:44:14 WARN  KafkaAgentClient:139 - Reconciliation #815(timer) Kafka((kafka/kafka): Failed to get broker state
java.lang.RuntimeException: Failed to send HTTP request to Kafka Agent
	at io.strimzi.operator.cluster.operator.resource.KafkaAgentClient.doGet(KafkaAgentClient.java:116)
	at io.strimzi.operator.cluster.operator.resource.KafkaAgentClient.getBrokerState(KafkaAgentClient.java:133)
	at io.strimzi.operator.cluster.operator.resource.KafkaRoller.restartIfNecessary(KafkaRoller.java:406)
	at io.strimzi.operator.cluster.operator.resource.KafkaRoller.lambda$schedule$7(KafkaRoller.java:336)
	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
	at java.base/java.util.concurrent.FutureTask.run(Unknown Source)
	at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
	at java.base/java.lang.Thread.run(Unknown Source)
Caused by: java.net.ConnectException
	at java.net.http/jdk.internal.net.http.HttpClientImpl.send(Unknown Source)
	at java.net.http/jdk.internal.net.http.HttpClientFacade.send(Unknown Source)
	at io.strimzi.operator.cluster.operator.resource.KafkaAgentClient.doGet(KafkaAgentClient.java:110)
	... 9 more
Caused by: java.net.ConnectException
	at java.net.http/jdk.internal.net.http.common.Utils.toConnectException(Unknown Source)
	at java.net.http/jdk.internal.net.http.PlainHttpConnection.connectAsync(Unknown Source)
	at java.net.http/jdk.internal.net.http.PlainHttpConnection.checkRetryConnect(Unknown Source)
	at java.net.http/jdk.internal.net.http.PlainHttpConnection.lambda$connectAsync$1(Unknown Source)
	at java.base/java.util.concurrent.CompletableFuture.uniHandle(Unknown Source)
	at java.base/java.util.concurrent.CompletableFuture$UniHandle.tryFire(Unknown Source)
	at java.base/java.util.concurrent.CompletableFuture.postComplete(Unknown Source)
	at java.base/java.util.concurrent.CompletableFuture$AsyncSupply.run(Unknown Source)
	... 3 more
Caused by: java.nio.channels.ClosedChannelException
	at java.base/sun.nio.ch.SocketChannelImpl.ensureOpen(Unknown Source)
	at java.base/sun.nio.ch.SocketChannelImpl.beginConnect(Unknown Source)
	at java.base/sun.nio.ch.SocketChannelImpl.connect(Unknown Source)
	at java.net.http/jdk.internal.net.http.PlainHttpConnection.lambda$connectAsync$0(Unknown Source)
	at java.base/java.security.AccessController.doPrivileged(Unknown Source)
	... 10 more
2026-01-28 13:44:14 WARN  KafkaRoller:412 - Reconciliation #815(timer) Kafka(kafka/kafka): Pod kafka-kafka-1 is not ready. We will check if KafkaRoller can do anything about it.

One thing that I noticed it is that in version 0.48.0 and below there were some init scripts (like kafka_tls_prepare_certificates.sh, tls_utils.sh) that were adding certificates to the truststore:

STRIMZI_BROKER_ID=4
Preparing truststore for replication listener
Adding /opt/kafka/cluster-ca-certs/ca.crt to truststore /tmp/kafka/cluster.truststore.p12 with alias ca
Certificate was added to keystore
Preparing truststore for replication listener is complete
Looking for the CA matching the server certificate
CA matching the server certificate found: /opt/kafka/cluster-ca-certs/ca.crt
Preparing keystore for replication and clienttls listener
Preparing keystore for replication and clienttls listener is complete
Preparing truststore for client authentication
Adding /opt/kafka/client-ca-certs/ca.crt to truststore /tmp/kafka/clients.truststore.p12 with alias ca
Certificate was added to keystore
Preparing truststore for client authentication is complete
Starting Kafka with configuration:
##############################
##############################
# This file is automatically generated by the Strimzi Cluster Operator
# Any changes to this file will be ignored and overwritten!
##############################
##############################

##########
# Node ID
##########
node.id=4

And also i could see the client-ca-certs, cluster-ca-certs and broker-certs in /opt/kafka.

In version 0.49.0 I see that these scripts are not executed anymore and I can't see these folders on the disk anymore.

STRIMZI_BROKER_ID=3
Starting Kafka with configuration:
##############################
##############################
# This file is automatically generated by the Strimzi Cluster Operator
# Any changes to this file will be ignored and overwritten!
##############################
##############################
##########
# Node ID
##########
node.id=3

But I'm not sure if this can cause my error or not.
Do you have any idea how to resolve this problem, please?

scholzj · 2026-01-28T19:46:46Z

scholzj
Jan 28, 2026
Maintainer

I'm not entirely sure what exactly is the problem in your case.

The operator log suggests some issue connecting to the broker - so you would need to explain what the state of the brokers is, share their full logs, etc.
The broker log is not complete from the start. So it is hard to understand what the problem there is. You would need to share the full log.

The TLS preparation scripts are not run by design as the PKCS12 stores that were being prepared by them are not used anymore. We now use the PEM files directly.

2 replies

user222635 Jan 29, 2026
Author

Thank you for the quick response!

When I'm trying to deploy the new version, the Kafka controller is working as expected, but the pod of the broker is failing entering in a state of CrashLoopBackOff because of the error above.
What I'm basically trying to do is to change the listener authentication configuration from the v1beta2 version to v1, and the problem is coming from this configuration:
v1beta2:

...
kafka:
    listeners:
      - configuration:
          useServiceDnsDomain: true
        name: tls
        port: 9093
        tls: true
        type: internal
      - authentication:
          enableOauthBearer: true
          jwksEndpointUri: https://url/protocol/openid-connect/certs
          jwksExpirySeconds: 360
          jwksMinRefreshPauseSeconds: 1
          jwksRefreshSeconds: 300
          maxSecondsWithoutReauthentication: 3600
          type: oauth
          userNameClaim: preferred_username
          validIssuerUri: https://url
        configuration:
          useServiceDnsDomain: true
        name: iam
        port: 9098
        tls: false
        type: internal

v1:

...
  kafka:
    config:
      principal.builder.class: io.strimzi.kafka.oauth.server.OAuthKafkaPrincipalBuilder
    listeners:
      - configuration:
          useServiceDnsDomain: true
        name: tls
        port: 9093
        tls: true
        type: internal
      - authentication:
          listenerConfig:
            oauthbearer.sasl.jaas.config: |
              org.apache.kafka.common.security.oauthbearer.OAuthBearerLoginModule
              required
              oauth.valid.issuer.uri="https://url/auth/realms/fts"
              oauth.jwks.endpoint.uri="https://url/protocol/openid-connect/certs"
              oauth.username.claim="preferred_username"
              oauth.ssl.truststore.location="/mnt/oauth-certs/ca.crt"
              oauth.ssl.truststore.type="PEM";
            oauthbearer.sasl.server.callback.handler.class: io.strimzi.kafka.oauth.server.JaasServerOauthValidatorCallbackHandler
            sasl.enabled.mechanisms: OAUTHBEARER
          sasl: true
          type: custom
        configuration:
          useServiceDnsDomain: true
        name: iam
        port: 9098
        tls: false
        type: internal

I will share here the following:

Kafka resource configuration
Kafka nodepool resource configuration
Full logs from the broker
ConfigMap generated by Kafka NodePool
Kafka controller nodepool resource configuration
ConfigMap generated by Kafka controller NodePool

Kafka resource configuration:

apiVersion: kafka.strimzi.io/v1
kind: Kafka
metadata:
  name: kafka
  namespace: kafka
spec:
  entityOperator:
    template:
      topicOperatorContainer:
        env:
          - name: STRIMZI_USE_FINALIZERS
            value: 'false'
    topicOperator:
      jvmOptions:
        '-XX':
          MaxRAMPercentage: '60'
      livenessProbe:
        failureThreshold: 5
        initialDelaySeconds: 30
        timeoutSeconds: 10
      logging:
        type: external
        valueFrom:
          configMapKeyRef:
            key: log4j2.properties
            name: kafka-log4j-config
      readinessProbe:
        failureThreshold: 5
        initialDelaySeconds: 30
        timeoutSeconds: 10
      resources:
        limits:
          cpu: 1000m
          memory: 384Mi
        requests:
          cpu: 100m
          memory: 256Mi
      startupProbe:
        timeoutSeconds: 10
    userOperator:
      livenessProbe:
        failureThreshold: 5
        initialDelaySeconds: 30
        timeoutSeconds: 10
      logging:
        type: external
        valueFrom:
          configMapKeyRef:
            key: log4j2.properties
            name: kafka-log4j-config
      readinessProbe:
        failureThreshold: 5
        initialDelaySeconds: 30
        timeoutSeconds: 10
  kafka:
    config:
      auto.create.topics.enable: 'false'
      compression.type: snappy
      default.replication.factor: 1
      log.segment.bytes: 104857600
      min.insync.replicas: 1
      num.io.threads: 25
      num.network.threads: 10
      num.partitions: 1
      offsets.topic.replication.factor: 1
      principal.builder.class: io.strimzi.kafka.oauth.server.OAuthKafkaPrincipalBuilder
      transaction.state.log.min.isr: 1
      transaction.state.log.replication.factor: 1
    image: quay.io/strimzi/kafka:0.49.0-kafka-4.1.0
    jvmOptions:
      '-XX':
        MaxRAMPercentage: '60'
    listeners:
      - configuration:
          useServiceDnsDomain: true
        name: tls
        port: 9093
        tls: true
        type: internal
      - authentication:
          listenerConfig:
            oauthbearer.sasl.jaas.config: |
              org.apache.kafka.common.security.oauthbearer.OAuthBearerLoginModule
              required
              oauth.valid.issuer.uri="https://url/auth/realms/fts"
              oauth.jwks.endpoint.uri="https://url/protocol/openid-connect/certs"
              oauth.username.claim="preferred_username"
              oauth.ssl.truststore.location="/mnt/oauth-certs/ca.crt"
              oauth.ssl.truststore.type="PEM";
            oauthbearer.sasl.server.callback.handler.class: io.strimzi.kafka.oauth.server.JaasServerOauthValidatorCallbackHandler
            sasl.enabled.mechanisms: OAUTHBEARER
          sasl: true
          type: custom
        configuration:
          useServiceDnsDomain: true
        name: iam
        port: 9098
        tls: false
        type: internal
    livenessProbe:
      initialDelaySeconds: 60
      timeoutSeconds: 5
    logging:
      type: external
      valueFrom:
        configMapKeyRef:
          key: log4j2.properties
          name: kafka-log4j-config
    metricsConfig:
      type: jmxPrometheusExporter
      valueFrom:
        configMapKeyRef:
          key: kafka-metrics-config.yml
          name: kafka-metrics
    readinessProbe:
      initialDelaySeconds: 60
      timeoutSeconds: 5
    template:
      kafkaContainer:
        volumeMounts:
          - mountPath: /mnt/oauth-certs
            name: certificates
      pod:
        volumes:
          - name: certificates
            secret:
              secretName: kafka-cluster-ca-cert
    version: 4.1.0
  kafkaExporter:
    groupRegex: .*
    image: quay.io/strimzi/kafka:0.49.0-kafka-4.1.0
    topicRegex: .*
status:
  autoRebalance:
    lastTransitionTime: '2026-01-26T12:08:47.807736762Z'
    state: Idle
  clusterId: 3bV6k2TNQjqbODi-A9mt_w
  conditions:
    - lastTransitionTime: '2026-01-28T12:39:10.474023516Z'
      message: Pod is unschedulable or is not starting
      reason: FatalProblem
      status: 'True'
      type: NotReady
  kafkaMetadataVersion: 4.1-IV1
  kafkaNodePools:
    - name: controller-kafka
    - name: kafka
  kafkaVersion: 4.1.0
  observedGeneration: 14
  operatorLastSuccessfulVersion: 0.49.0

Kafka nodepool resource configuration:

apiVersion: kafka.strimzi.io/v1
kind: KafkaNodePool
metadata:
  name: kafka
  namespace: kafka
spec:
  replicas: 1
  roles:
    - broker
  storage:
    type: jbod
    volumes:
      - deleteClaim: false
        id: 0
        size: 30Gi
        type: persistent-claim
status:
  conditions: []
  labelSelector: >-
    strimzi.io/cluster=kafka,strimzi.io/name=kafka-kafka,strimzi.io/kind=Kafka,strimzi.io/pool-name=kafka
  nodeIds:
    - 1
  observedGeneration: 1
  replicas: 1
  roles:
    - broker

Full logs from the broker:

removed directory '/tmp/hsperfdata_kafka'
removed directory '/tmp/kafka'
removed '/tmp/kafka-agent.properties'
removed '/tmp/strimzi.properties'
STRIMZI_BROKER_ID=1
Starting Kafka with configuration:
##############################
##############################
# This file is automatically generated by the Strimzi Cluster Operator
# Any changes to this file will be ignored and overwritten!
##############################
##############################
##########
# Node ID
##########
node.id=1
##########
# KRaft configuration
##########
process.roles=broker
controller.listener.names=CONTROLPLANE-9090
controller.quorum.voters=0@kafka-controller-kafka-0.kafka-kafka-brokers.kafka.svc.cluster.local:9090
##########
# KRaft metadata log dir configuration
##########
metadata.log.dir=/var/lib/kafka/data-0/kafka-log1
##########
# Kafka message logs configuration
##########
log.dirs=/var/lib/kafka/data-0/kafka-log1
##########
# Control Plane listener
##########
listener.name.controlplane-9090.ssl.keystore.certificate.chain=${strimzisecrets:kafka/kafka-kafka-1:kafka-kafka-1.crt}
listener.name.controlplane-9090.ssl.keystore.key=${strimzisecrets:kafka/kafka-kafka-1:kafka-kafka-1.key}
listener.name.controlplane-9090.ssl.keystore.type=PEM
listener.name.controlplane-9090.ssl.truststore.certificates=${strimzisecrets:kafka/kafka-cluster-ca-cert:*.crt}
listener.name.controlplane-9090.ssl.truststore.type=PEM
listener.name.controlplane-9090.ssl.client.auth=required
##########
# Replication listener
##########
listener.name.replication-9091.ssl.keystore.certificate.chain=${strimzisecrets:kafka/kafka-kafka-1:kafka-kafka-1.crt}
listener.name.replication-9091.ssl.keystore.key=${strimzisecrets:kafka/kafka-kafka-1:kafka-kafka-1.key}
listener.name.replication-9091.ssl.keystore.type=PEM
listener.name.replication-9091.ssl.truststore.certificates=${strimzisecrets:kafka/kafka-cluster-ca-cert:*.crt}
listener.name.replication-9091.ssl.truststore.type=PEM
listener.name.replication-9091.ssl.client.auth=required
##########
# Listener configuration: TLS-9093
##########
listener.name.tls-9093.ssl.keystore.certificate.chain=${strimzisecrets:kafka/kafka-kafka-1:kafka-kafka-1.crt}
listener.name.tls-9093.ssl.keystore.key=${strimzisecrets:kafka/kafka-kafka-1:kafka-kafka-1.key}
listener.name.tls-9093.ssl.keystore.type=PEM
##########
# Listener configuration: IAM-9098
##########
listener.name.iam-9098.oauthbearer.sasl.jaas.config=[hidden]
listener.name.iam-9098.oauthbearer.sasl.server.callback.handler.class=io.strimzi.kafka.oauth.server.JaasServerOauthValidatorCallbackHandler
listener.name.iam-9098.sasl.enabled.mechanisms=OAUTHBEARER
##########
# Common listener configuration
##########
listener.security.protocol.map=CONTROLPLANE-9090:SSL,REPLICATION-9091:SSL,TLS-9093:SSL,IAM-9098:SASL_PLAINTEXT
listeners=REPLICATION-9091://0.0.0.0:9091,TLS-9093://0.0.0.0:9093,IAM-9098://0.0.0.0:9098
inter.broker.listener.name=REPLICATION-9091
advertised.listeners=REPLICATION-9091://kafka-kafka-1.kafka-kafka-brokers.kafka.svc:9091,TLS-9093://kafka-kafka-1.kafka-kafka-brokers.kafka.svc.cluster.local:9093,IAM-9098://kafka-kafka-1.kafka-kafka-brokers.kafka.svc.cluster.local:9098
sasl.enabled.mechanisms=
ssl.endpoint.identification.algorithm=HTTPS
##########
# Config providers
##########
# Configuration providers configured by the user and by Strimzi
config.providers=strimzienv,strimzisecrets,strimzifile,strimzidir
config.providers.strimzienv.class=org.apache.kafka.common.config.provider.EnvVarConfigProvider
config.providers.strimzienv.param.allowlist.pattern=.*
config.providers.strimzisecrets.class=io.strimzi.kafka.KubernetesSecretConfigProvider
config.providers.strimzifile.class=org.apache.kafka.common.config.provider.FileConfigProvider
config.providers.strimzifile.param.allowed.paths=/opt/kafka
config.providers.strimzidir.class=org.apache.kafka.common.config.provider.DirectoryConfigProvider
config.providers.strimzidir.param.allowed.paths=/opt/kafka
##########
# metric.reporters configuration
##########
# metric.reporters configured by Strimzi
metric.reporters=org.apache.kafka.common.metrics.JmxReporter
##########
# User provided configuration
##########
min.insync.replicas=1
auto.create.topics.enable=false
compression.type=snappy
default.replication.factor=1
log.segment.bytes=104857600
num.io.threads=25
num.network.threads=10
num.partitions=1
offsets.topic.replication.factor=1
principal.builder.class=io.strimzi.kafka.oauth.server.OAuthKafkaPrincipalBuilder
transaction.state.log.min.isr=1
transaction.state.log.replication.factor=1
Making sure the Kraft storage is formatted with cluster ID E_5LTVgNRHStIKirYSmHWw and metadata version 4.1
2026-01-29 09:23:08.683 | main | INFO | io.prometheus.jmx.JavaAgent | Starting ...
2026-01-29 09:23:08.956 | main | INFO | io.prometheus.jmx.JavaAgent | HTTP enabled [true]
2026-01-29 09:23:08.957 | main | INFO | io.prometheus.jmx.JavaAgent | HTTP host:port [0.0.0.0:9404]
2026-01-29 09:23:08.957 | main | INFO | io.prometheus.jmx.JavaAgent | Starting HTTPServer ...
2026-01-29 09:23:09.001 | main | INFO | io.prometheus.jmx.JavaAgent | HTTPServer started
2026-01-29 09:23:09.002 | main | INFO | io.prometheus.jmx.JavaAgent | OpenTelemetry enabled [false]
2026-01-29 09:23:09.002 | main | INFO | io.prometheus.jmx.JavaAgent | Running ...
2026-01-29 09:23:09 INFO  [main] Log4jControllerRegistration$:33 - Registered `kafka:type=kafka.Log4jController` MBean
2026-01-29 09:23:09 INFO  [main] AbstractKubernetesConfigProvider:68 - Configuring Kubernetes Secret config provider with configuration {}
2026-01-29 09:23:10 INFO  [main] AbstractKubernetesConfigProvider:123 - Retrieving configuration from Secret kafka-cluster-ca-cert in namespace kafka
2026-01-29 09:23:10 INFO  [main] AbstractKubernetesConfigProvider:123 - Retrieving configuration from Secret kafka-kafka-1 in namespace kafka
2026-01-29 09:23:10 INFO  [main] AbstractKubernetesConfigProvider:62 - Closing Kubernetes Secret config provider
2026-01-29 09:23:10 INFO  [main] AbstractConfig:380 - KafkaConfig values:
	add.partitions.to.txn.retry.backoff.max.ms = 100
	add.partitions.to.txn.retry.backoff.ms = 20
	advertised.listeners = REPLICATION-9091://kafka-kafka-1.kafka-kafka-brokers.kafka.svc:9091,TLS-9093://kafka-kafka-1.kafka-kafka-brokers.kafka.svc.cluster.local:9093,IAM-9098://kafka-kafka-1.kafka-kafka-brokers.kafka.svc.cluster.local:9098
	alter.config.policy.class.name = null
	alter.log.dirs.replication.quota.window.num = 11
	alter.log.dirs.replication.quota.window.size.seconds = 1
	authorizer.class.name =
	auto.create.topics.enable = false
	auto.leader.rebalance.enable = true
	background.threads = 10
	broker.heartbeat.interval.ms = 2000
	broker.id = 1
	broker.rack = null
	broker.session.timeout.ms = 9000
	client.quota.callback.class = null
	compression.gzip.level = -1
	compression.lz4.level = 9
	compression.type = snappy
	compression.zstd.level = 3
	connection.failed.authentication.delay.ms = 100
	connections.max.idle.ms = 600000
	connections.max.reauth.ms = 0
	controlled.shutdown.enable = true
	controller.listener.names = CONTROLPLANE-9090
	controller.performance.always.log.threshold.ms = 2000
	controller.performance.sample.period.ms = 60000
	controller.quorum.append.linger.ms = 25
	controller.quorum.bootstrap.servers = []
	controller.quorum.election.backoff.max.ms = 1000
	controller.quorum.election.timeout.ms = 1000
	controller.quorum.fetch.timeout.ms = 2000
	controller.quorum.request.timeout.ms = 2000
	controller.quorum.retry.backoff.ms = 20
	controller.quorum.voters = [0@kafka-controller-kafka-0.kafka-kafka-brokers.kafka.svc.cluster.local:9090]
	controller.quota.window.num = 11
	controller.quota.window.size.seconds = 1
	controller.socket.timeout.ms = 30000
	create.topic.policy.class.name = null
	default.replication.factor = 1
	delegation.token.expiry.check.interval.ms = 3600000
	delegation.token.expiry.time.ms = 86400000
	delegation.token.max.lifetime.ms = 604800000
	delegation.token.secret.key = null
	delete.records.purgatory.purge.interval.requests = 1
	delete.topic.enable = true
	early.start.listeners = null
	fetch.max.bytes = 57671680
	fetch.purgatory.purge.interval.requests = 1000
	group.consumer.assignors = [uniform, range]
	group.consumer.heartbeat.interval.ms = 5000
	group.consumer.max.heartbeat.interval.ms = 15000
	group.consumer.max.session.timeout.ms = 60000
	group.consumer.max.size = 2147483647
	group.consumer.migration.policy = bidirectional
	group.consumer.min.heartbeat.interval.ms = 5000
	group.consumer.min.session.timeout.ms = 45000
	group.consumer.regex.refresh.interval.ms = 600000
	group.consumer.session.timeout.ms = 45000
	group.coordinator.append.linger.ms = 5
	group.coordinator.rebalance.protocols = [classic, consumer, streams]
	group.coordinator.threads = 4
	group.initial.rebalance.delay.ms = 3000
	group.max.session.timeout.ms = 1800000
	group.max.size = 2147483647
	group.min.session.timeout.ms = 6000
	group.share.assignors = [simple]
	group.share.delivery.count.limit = 5
	group.share.enable = false
	group.share.heartbeat.interval.ms = 5000
	group.share.max.heartbeat.interval.ms = 15000
	group.share.max.record.lock.duration.ms = 60000
	group.share.max.session.timeout.ms = 60000
	group.share.max.share.sessions = 2000
	group.share.max.size = 200
	group.share.min.heartbeat.interval.ms = 5000
	group.share.min.record.lock.duration.ms = 15000
	group.share.min.session.timeout.ms = 45000
	group.share.partition.max.record.locks = 2000
	group.share.persister.class.name = org.apache.kafka.server.share.persister.DefaultStatePersister
	group.share.record.lock.duration.ms = 30000
	group.share.session.timeout.ms = 45000
	group.streams.heartbeat.interval.ms = 5000
	group.streams.max.heartbeat.interval.ms = 15000
	group.streams.max.session.timeout.ms = 60000
	group.streams.max.size = 2147483647
	group.streams.max.standby.replicas = 2
	group.streams.min.heartbeat.interval.ms = 5000
	group.streams.min.session.timeout.ms = 45000
	group.streams.num.standby.replicas = 0
	group.streams.session.timeout.ms = 45000
	initial.broker.registration.timeout.ms = 60000
	inter.broker.listener.name = REPLICATION-9091
	internal.metadata.delete.delay.millis = 60000
	internal.metadata.log.segment.bytes = null
	internal.metadata.max.batch.size.in.bytes = 8388608
	internal.metadata.max.fetch.size.in.bytes = 8388608
	kafka.metrics.polling.interval.secs = 10
	kafka.metrics.reporters = []
	leader.imbalance.check.interval.seconds = 300
	listener.security.protocol.map = CONTROLPLANE-9090:SSL,REPLICATION-9091:SSL,TLS-9093:SSL,IAM-9098:SASL_PLAINTEXT
	listeners = REPLICATION-9091://0.0.0.0:9091,TLS-9093://0.0.0.0:9093,IAM-9098://0.0.0.0:9098
	log.cleaner.backoff.ms = 15000
	log.cleaner.dedupe.buffer.size = 134217728
	log.cleaner.delete.retention.ms = 86400000
	log.cleaner.enable = true
	log.cleaner.io.buffer.load.factor = 0.9
	log.cleaner.io.buffer.size = 524288
	log.cleaner.io.max.bytes.per.second = 1.7976931348623157E308
	log.cleaner.max.compaction.lag.ms = 9223372036854775807
	log.cleaner.min.cleanable.ratio = 0.5
	log.cleaner.min.compaction.lag.ms = 0
	log.cleaner.threads = 1
	log.cleanup.policy = [delete]
	log.dir = /tmp/kafka-logs
	log.dir.failure.timeout.ms = 30000
	log.dirs = /var/lib/kafka/data-0/kafka-log1
	log.flush.interval.messages = 9223372036854775807
	log.flush.interval.ms = null
	log.flush.offset.checkpoint.interval.ms = 60000
	log.flush.scheduler.interval.ms = 9223372036854775807
	log.flush.start.offset.checkpoint.interval.ms = 60000
	log.index.interval.bytes = 4096
	log.index.size.max.bytes = 10485760
	log.initial.task.delay.ms = 30000
	log.local.retention.bytes = -2
	log.local.retention.ms = -2
	log.message.timestamp.after.max.ms = 3600000
	log.message.timestamp.before.max.ms = 9223372036854775807
	log.message.timestamp.type = CreateTime
	log.preallocate = false
	log.retention.bytes = -1
	log.retention.check.interval.ms = 300000
	log.retention.hours = 168
	log.retention.minutes = null
	log.retention.ms = null
	log.roll.hours = 168
	log.roll.jitter.hours = 0
	log.roll.jitter.ms = null
	log.roll.ms = null
	log.segment.bytes = 104857600
	log.segment.delete.delay.ms = 60000
	max.connection.creation.rate = 2147483647
	max.connections = 2147483647
	max.connections.per.ip = 2147483647
	max.connections.per.ip.overrides =
	max.incremental.fetch.session.cache.slots = 1000
	max.request.partition.size.limit = 2000
	message.max.bytes = 1048588
	metadata.log.dir = /var/lib/kafka/data-0/kafka-log1
	metadata.log.max.record.bytes.between.snapshots = 20971520
	metadata.log.max.snapshot.interval.ms = 3600000
	metadata.log.segment.bytes = 1073741824
	metadata.log.segment.ms = 604800000
	metadata.max.idle.interval.ms = 500
	metadata.max.retention.bytes = 104857600
	metadata.max.retention.ms = 604800000
	metric.reporters = [org.apache.kafka.common.metrics.JmxReporter]
	metrics.num.samples = 2
	metrics.recording.level = INFO
	metrics.sample.window.ms = 30000
	min.insync.replicas = 1
	node.id = 1
	num.io.threads = 25
	num.network.threads = 10
	num.partitions = 1
	num.recovery.threads.per.data.dir = 2
	num.replica.alter.log.dirs.threads = null
	num.replica.fetchers = 1
	offset.metadata.max.bytes = 4096
	offsets.commit.timeout.ms = 5000
	offsets.load.buffer.size = 5242880
	offsets.retention.check.interval.ms = 600000
	offsets.retention.minutes = 10080
	offsets.topic.compression.codec = 0
	offsets.topic.num.partitions = 50
	offsets.topic.replication.factor = 1
	offsets.topic.segment.bytes = 104857600
	principal.builder.class = class io.strimzi.kafka.oauth.server.OAuthKafkaPrincipalBuilder
	process.roles = [broker]
	producer.id.expiration.check.interval.ms = 600000
	producer.id.expiration.ms = 86400000
	producer.purgatory.purge.interval.requests = 1000
	queued.max.request.bytes = -1
	queued.max.requests = 500
	quota.window.num = 11
	quota.window.size.seconds = 1
	remote.fetch.max.wait.ms = 500
	remote.list.offsets.request.timeout.ms = 30000
	remote.log.index.file.cache.total.size.bytes = 1073741824
	remote.log.manager.copier.thread.pool.size = 10
	remote.log.manager.copy.max.bytes.per.second = 9223372036854775807
	remote.log.manager.copy.quota.window.num = 11
	remote.log.manager.copy.quota.window.size.seconds = 1
	remote.log.manager.expiration.thread.pool.size = 10
	remote.log.manager.fetch.max.bytes.per.second = 9223372036854775807
	remote.log.manager.fetch.quota.window.num = 11
	remote.log.manager.fetch.quota.window.size.seconds = 1
	remote.log.manager.task.interval.ms = 30000
	remote.log.manager.task.retry.backoff.max.ms = 30000
	remote.log.manager.task.retry.backoff.ms = 500
	remote.log.manager.task.retry.jitter = 0.2
	remote.log.manager.thread.pool.size = 2
	remote.log.metadata.custom.metadata.max.bytes = 128
	remote.log.metadata.manager.class.name = org.apache.kafka.server.log.remote.metadata.storage.TopicBasedRemoteLogMetadataManager
	remote.log.metadata.manager.class.path = null
	remote.log.metadata.manager.impl.prefix = rlmm.config.
	remote.log.metadata.manager.listener.name = null
	remote.log.reader.max.pending.tasks = 100
	remote.log.reader.threads = 10
	remote.log.storage.manager.class.name = null
	remote.log.storage.manager.class.path = null
	remote.log.storage.manager.impl.prefix = rsm.config.
	remote.log.storage.system.enable = false
	replica.fetch.backoff.ms = 1000
	replica.fetch.max.bytes = 1048576
	replica.fetch.min.bytes = 1
	replica.fetch.response.max.bytes = 10485760
	replica.fetch.wait.max.ms = 500
	replica.high.watermark.checkpoint.interval.ms = 5000
	replica.lag.time.max.ms = 30000
	replica.selector.class = null
	replica.socket.receive.buffer.bytes = 65536
	replica.socket.timeout.ms = 30000
	replication.quota.window.num = 11
	replication.quota.window.size.seconds = 1
	request.timeout.ms = 30000
	sasl.client.callback.handler.class = null
	sasl.enabled.mechanisms = []
	sasl.jaas.config = null
	sasl.kerberos.kinit.cmd = /usr/bin/kinit
	sasl.kerberos.min.time.before.relogin = 60000
	sasl.kerberos.principal.to.local.rules = [DEFAULT]
	sasl.kerberos.service.name = null
	sasl.kerberos.ticket.renew.jitter = 0.05
	sasl.kerberos.ticket.renew.window.factor = 0.8
	sasl.login.callback.handler.class = null
	sasl.login.class = null
	sasl.login.connect.timeout.ms = null
	sasl.login.read.timeout.ms = null
	sasl.login.refresh.buffer.seconds = 300
	sasl.login.refresh.min.period.seconds = 60
	sasl.login.refresh.window.factor = 0.8
	sasl.login.refresh.window.jitter = 0.05
	sasl.login.retry.backoff.max.ms = 10000
	sasl.login.retry.backoff.ms = 100
	sasl.mechanism.controller.protocol = GSSAPI
	sasl.mechanism.inter.broker.protocol = GSSAPI
	sasl.oauthbearer.assertion.algorithm = RS256
	sasl.oauthbearer.assertion.claim.aud = null
	sasl.oauthbearer.assertion.claim.exp.seconds = 300
	sasl.oauthbearer.assertion.claim.iss = null
	sasl.oauthbearer.assertion.claim.jti.include = false
	sasl.oauthbearer.assertion.claim.nbf.seconds = 60
	sasl.oauthbearer.assertion.claim.sub = null
	sasl.oauthbearer.assertion.file = null
	sasl.oauthbearer.assertion.private.key.file = null
	sasl.oauthbearer.assertion.private.key.passphrase = null
	sasl.oauthbearer.assertion.template.file = null
	sasl.oauthbearer.client.credentials.client.id = null
	sasl.oauthbearer.client.credentials.client.secret = null
	sasl.oauthbearer.clock.skew.seconds = 30
	sasl.oauthbearer.expected.audience = null
	sasl.oauthbearer.expected.issuer = null
	sasl.oauthbearer.jwks.endpoint.refresh.ms = 3600000
	sasl.oauthbearer.jwks.endpoint.retry.backoff.max.ms = 10000
	sasl.oauthbearer.jwks.endpoint.retry.backoff.ms = 100
	sasl.oauthbearer.jwks.endpoint.url = null
	sasl.oauthbearer.jwt.retriever.class = class org.apache.kafka.common.security.oauthbearer.DefaultJwtRetriever
	sasl.oauthbearer.jwt.validator.class = class org.apache.kafka.common.security.oauthbearer.DefaultJwtValidator
	sasl.oauthbearer.scope = null
	sasl.oauthbearer.scope.claim.name = scope
	sasl.oauthbearer.sub.claim.name = sub
	sasl.oauthbearer.token.endpoint.url = null
	sasl.server.callback.handler.class = null
	sasl.server.max.receive.size = 524288
	security.inter.broker.protocol = PLAINTEXT
	security.providers = null
	server.max.startup.time.ms = 9223372036854775807
	share.coordinator.append.linger.ms = 5
	share.coordinator.cold.partition.snapshot.interval.ms = 300000
	share.coordinator.load.buffer.size = 5242880
	share.coordinator.snapshot.update.records.per.snapshot = 500
	share.coordinator.state.topic.compression.codec = 0
	share.coordinator.state.topic.min.isr = 2
	share.coordinator.state.topic.num.partitions = 50
	share.coordinator.state.topic.prune.interval.ms = 300000
	share.coordinator.state.topic.replication.factor = 3
	share.coordinator.state.topic.segment.bytes = 104857600
	share.coordinator.threads = 1
	share.coordinator.write.timeout.ms = 5000
	share.fetch.purgatory.purge.interval.requests = 1000
	socket.connection.setup.timeout.max.ms = 30000
	socket.connection.setup.timeout.ms = 10000
	socket.listen.backlog.size = 50
	socket.receive.buffer.bytes = 102400
	socket.request.max.bytes = 104857600
	socket.send.buffer.bytes = 102400
	ssl.allow.dn.changes = false
	ssl.allow.san.changes = false
	ssl.cipher.suites = []
	ssl.client.auth = none
	ssl.enabled.protocols = [TLSv1.2, TLSv1.3]
	ssl.endpoint.identification.algorithm = HTTPS
	ssl.engine.factory.class = null
	ssl.key.password = null
	ssl.keymanager.algorithm = SunX509
	ssl.keystore.certificate.chain = null
	ssl.keystore.key = null
	ssl.keystore.location = null
	ssl.keystore.password = null
	ssl.keystore.type = JKS
	ssl.principal.mapping.rules = DEFAULT
	ssl.protocol = TLSv1.3
	ssl.provider = null
	ssl.secure.random.implementation = null
	ssl.trustmanager.algorithm = PKIX
	ssl.truststore.certificates = null
	ssl.truststore.location = null
	ssl.truststore.password = null
	ssl.truststore.type = JKS
	telemetry.max.bytes = 1048576
	transaction.abort.timed.out.transaction.cleanup.interval.ms = 10000
	transaction.max.timeout.ms = 900000
	transaction.partition.verification.enable = true
	transaction.remove.expired.transaction.cleanup.interval.ms = 3600000
	transaction.state.log.load.buffer.size = 5242880
	transaction.state.log.min.isr = 1
	transaction.state.log.num.partitions = 50
	transaction.state.log.replication.factor = 1
	transaction.state.log.segment.bytes = 104857600
	transaction.two.phase.commit.enable = false
	transactional.id.expiration.ms = 604800000
	unclean.leader.election.enable = false
	unclean.leader.election.interval.ms = 300000
	unstable.api.versions.enable = false
	unstable.feature.versions.enable = false
All of the log directories are already formatted.
KRaft storage formatting is done
Removing quorum-state file
Preparing Kafka Agent configuration
+ exec /usr/bin/tini -w -e 143 -- /opt/kafka/bin/kafka-server-start.sh /tmp/strimzi.properties
2026-01-29 09:23:12.902 | main | INFO | io.prometheus.jmx.JavaAgent | Starting ...
2026-01-29 09:23:13.186 | main | INFO | io.prometheus.jmx.JavaAgent | HTTP enabled [true]
2026-01-29 09:23:13.187 | main | INFO | io.prometheus.jmx.JavaAgent | HTTP host:port [0.0.0.0:9404]
2026-01-29 09:23:13.187 | main | INFO | io.prometheus.jmx.JavaAgent | Starting HTTPServer ...
2026-01-29 09:23:13.243 | main | INFO | io.prometheus.jmx.JavaAgent | HTTPServer started
2026-01-29 09:23:13.244 | main | INFO | io.prometheus.jmx.JavaAgent | OpenTelemetry enabled [false]
2026-01-29 09:23:13.244 | main | INFO | io.prometheus.jmx.JavaAgent | Running ...
2026-01-29 09:23:13 INFO  [main] KafkaAgent:345 - Starting KafkaAgent with sslTrustStoreSecretName=kafka-cluster-ca-cert sslKeyStoreSecretName=kafka-kafka-1 namespace=kafka
2026-01-29 09:23:14 INFO  [main] Server:555 - jetty-12.0.22; built: 2025-06-02T15:25:31.946Z; git: 335c9ab44a5591f0ea941bf350e139b8c4f5537c; jvm 17.0.10+7
2026-01-29 09:23:15 INFO  [main] ContextHandler:764 - Started oejsh.ContextHandler@69da0b12{/v1/broker-state,/v1/broker-state,b=null,a=AVAILABLE,h=iska.KafkaAgent$@2764c546{STARTED}}
2026-01-29 09:23:15 INFO  [main] ContextHandler:764 - Started oejsh.ContextHandler@59496961{/v1/ready,/v1/ready,b=null,a=AVAILABLE,h=iska.KafkaAgent$@408b87aa{STARTED}}
2026-01-29 09:23:15 INFO  [main] SslContextFactory:337 - x509=X509@7d0cc890(kafka-kafka-1,h=[kafka-kafka-bootstrap.kafka, kafka-kafka-bootstrap.kafka.svc.cluster.local, kafka-kafka-bootstrap.kafka.svc, kafka-kafka-1.kafka-kafka-brokers.kafka.svc, kafka-kafka-bootstrap, kafka-kafka-brokers.kafka.svc, kafka-kafka-brokers.kafka.svc.cluster.local, kafka-kafka-brokers.kafka, kafka-kafka-1.kafka-kafka-brokers.kafka.svc.cluster.local, kafka-kafka-brokers, kafka-kafka],a=[],w=[]) for Server@49293b43[provider=null,keyStore=null,trustStore=null]
2026-01-29 09:23:15 INFO  [main] AbstractConnector:326 - Started ServerConnector@3d98d138{SSL, (ssl, http/1.1)}{0.0.0.0:8443}
2026-01-29 09:23:15 INFO  [main] AbstractConnector:326 - Started ServerConnector@60fb2bf6{HTTP/1.1, (http/1.1)}{localhost:8080}
2026-01-29 09:23:15 INFO  [main] Server:612 - Started oejs.Server@4441d567{STARTING}[12.0.22,sto=30000] @2516ms
2026-01-29 09:23:15 INFO  [main] KafkaAgent:129 - Starting metrics registry
2026-01-29 09:23:15 INFO  [main] KafkaAgent:165 - Found class org.apache.kafka.server.metrics.KafkaYammerMetrics for Kafka 3.3 and newer.
2026-01-29 09:23:15 INFO  [main] Log4jControllerRegistration$:33 - Registered `kafka:type=kafka.Log4jController` MBean
2026-01-29 09:23:15 INFO  [main] AbstractKubernetesConfigProvider:68 - Configuring Kubernetes Secret config provider with configuration {}
2026-01-29 09:23:15 INFO  [main] AbstractKubernetesConfigProvider:123 - Retrieving configuration from Secret kafka-cluster-ca-cert in namespace kafka
2026-01-29 09:23:15 INFO  [main] AbstractKubernetesConfigProvider:123 - Retrieving configuration from Secret kafka-kafka-1 in namespace kafka
2026-01-29 09:23:15 INFO  [main] AbstractKubernetesConfigProvider:62 - Closing Kubernetes Secret config provider
2026-01-29 09:23:15 INFO  [main] AbstractKubernetesConfigProvider:68 - Configuring Kubernetes Secret config provider with configuration {}
2026-01-29 09:23:15 INFO  [main] AbstractKubernetesConfigProvider:123 - Retrieving configuration from Secret kafka-cluster-ca-cert in namespace kafka
2026-01-29 09:23:15 INFO  [main] AbstractKubernetesConfigProvider:123 - Retrieving configuration from Secret kafka-kafka-1 in namespace kafka
2026-01-29 09:23:15 INFO  [main] AbstractKubernetesConfigProvider:62 - Closing Kubernetes Secret config provider
2026-01-29 09:23:15 INFO  [main] AbstractKubernetesConfigProvider:68 - Configuring Kubernetes Secret config provider with configuration {}
2026-01-29 09:23:15 INFO  [main] AbstractKubernetesConfigProvider:123 - Retrieving configuration from Secret kafka-cluster-ca-cert in namespace kafka
2026-01-29 09:23:15 INFO  [main] AbstractKubernetesConfigProvider:123 - Retrieving configuration from Secret kafka-kafka-1 in namespace kafka
2026-01-29 09:23:15 INFO  [main] AbstractKubernetesConfigProvider:62 - Closing Kubernetes Secret config provider
2026-01-29 09:23:15 INFO  [main] LoggingSignalHandler:72 - Registered signal handlers for TERM, INT, HUP
2026-01-29 09:23:15 INFO  [main] BrokerServer:69 - [BrokerServer id=1] Transition from SHUTDOWN to STARTING
2026-01-29 09:23:15 INFO  [main] SharedServer:69 - [SharedServer id=1] Starting SharedServer
2026-01-29 09:23:15 INFO  [main] AbstractKubernetesConfigProvider:68 - Configuring Kubernetes Secret config provider with configuration {}
2026-01-29 09:23:15 INFO  [main] AbstractKubernetesConfigProvider:123 - Retrieving configuration from Secret kafka-cluster-ca-cert in namespace kafka
2026-01-29 09:23:15 INFO  [main] AbstractKubernetesConfigProvider:123 - Retrieving configuration from Secret kafka-kafka-1 in namespace kafka
2026-01-29 09:23:15 INFO  [main] AbstractKubernetesConfigProvider:62 - Closing Kubernetes Secret config provider
2026-01-29 09:23:16 INFO  [main] LogLoader:477 - [LogLoader partition=__cluster_metadata-0, dir=/var/lib/kafka/data-0/kafka-log1] Recovering unflushed segment 0. 0 recovered for __cluster_metadata-0.
2026-01-29 09:23:16 INFO  [main] UnifiedLog:2440 - [LogLoader partition=__cluster_metadata-0, dir=/var/lib/kafka/data-0/kafka-log1] Loading producer state till offset 0
2026-01-29 09:23:16 INFO  [main] UnifiedLog:2462 - [LogLoader partition=__cluster_metadata-0, dir=/var/lib/kafka/data-0/kafka-log1] Reloading from producer snapshot and rebuilding producer state from offset 0
2026-01-29 09:23:16 INFO  [main] SnapshotFile:48 - Deleted producer state snapshot /var/lib/kafka/data-0/kafka-log1/__cluster_metadata-0/00000000000000001464.snapshot
2026-01-29 09:23:16 INFO  [main] SnapshotFile:48 - Deleted producer state snapshot /var/lib/kafka/data-0/kafka-log1/__cluster_metadata-0/00000000000000002114.snapshot
2026-01-29 09:23:16 INFO  [main] UnifiedLog:2498 - [LogLoader partition=__cluster_metadata-0, dir=/var/lib/kafka/data-0/kafka-log1] Producer state recovery took 2ms for snapshot load and 0ms for segment recovery from offset 0
2026-01-29 09:23:16 INFO  [main] ProducerStateManager:441 - [ProducerStateManager partition=__cluster_metadata-0] Wrote producer snapshot at offset 2114 with 0 producer ids in 8 ms.
2026-01-29 09:23:16 INFO  [main] UnifiedLog:2440 - [LogLoader partition=__cluster_metadata-0, dir=/var/lib/kafka/data-0/kafka-log1] Loading producer state till offset 2114
2026-01-29 09:23:16 INFO  [main] UnifiedLog:2462 - [LogLoader partition=__cluster_metadata-0, dir=/var/lib/kafka/data-0/kafka-log1] Reloading from producer snapshot and rebuilding producer state from offset 2114
2026-01-29 09:23:16 INFO  [main] ProducerStateManager:302 - [ProducerStateManager partition=__cluster_metadata-0] Loading producer state from snapshot file 'SnapshotFile(offset=2114, file=/var/lib/kafka/data-0/kafka-log1/__cluster_metadata-0/00000000000000002114.snapshot)'
2026-01-29 09:23:16 INFO  [main] UnifiedLog:2498 - [LogLoader partition=__cluster_metadata-0, dir=/var/lib/kafka/data-0/kafka-log1] Producer state recovery took 7ms for snapshot load and 0ms for segment recovery from offset 2114
2026-01-29 09:23:16 INFO  [main] KafkaMetadataLog$:580 - Initialized snapshots with IDs SortedSet() from /var/lib/kafka/data-0/kafka-log1/__cluster_metadata-0
2026-01-29 09:23:16 INFO  [raft-expiration-reaper] TimingWheelExpirationService$ExpiredOperationReaper:133 - [raft-expiration-reaper]: Starting
2026-01-29 09:23:16 INFO  [main] KafkaRaftClient:503 - [RaftManager id=1] Reading KRaft snapshot and log as part of the initialization
2026-01-29 09:23:16 INFO  [main] KafkaRaftClient:505 - [RaftManager id=1] Starting voters are VoterSet(voters={0=VoterNode(voterKey=ReplicaKey(id=0, directoryId=<undefined>), listeners=Endpoints(endpoints={ListenerName(CONTROLPLANE-9090)=kafka-controller-kafka-0.kafka-kafka-brokers.kafka.svc.cluster.local/10.46.221.175:9090}), supportedKRaftVersion=SupportedVersionRange[min_version:0, max_version:0])})
2026-01-29 09:23:16 INFO  [main] KafkaRaftClient:531 - [RaftManager id=1] Starting request manager with static voters: [kafka-controller-kafka-0.kafka-kafka-brokers.kafka.svc.cluster.local:9090 (id: 0 rack: null isFenced: false)]
2026-01-29 09:23:16 WARN  [main] QuorumState:158 - [RaftManager id=1] Epoch from quorum store file (/var/lib/kafka/data-0/kafka-log1/__cluster_metadata-0/quorum-state) is 0, which is smaller than last written epoch 1 in the log
2026-01-29 09:23:16 INFO  [main] QuorumState:732 - [RaftManager id=1] Attempting durable transition to UnattachedState(epoch=1, leaderId=OptionalInt.empty, votedKey=Optional.empty, voters=[0], electionTimeoutMs=1004, highWatermark=Optional.empty) from null
2026-01-29 09:23:16 INFO  [main] QuorumState:749 - [RaftManager id=1] Completed transition to UnattachedState(epoch=1, leaderId=OptionalInt.empty, votedKey=Optional.empty, voters=[0], electionTimeoutMs=1004, highWatermark=Optional.empty) from null
2026-01-29 09:23:16 INFO  [kafka-1-raft-outbound-request-thread] KafkaNetworkChannel$SendThread:133 - [kafka-1-raft-outbound-request-thread]: Starting
2026-01-29 09:23:16 INFO  [kafka-1-raft-io-thread] KafkaRaftClientDriver:133 - [kafka-1-raft-io-thread]: Starting
2026-01-29 09:23:16 INFO  [kafka-1-metadata-loader-event-handler] MetadataLoader:233 - [MetadataLoader id=1] initializeNewPublishers: the loader is still catching up because we still don't know the high water mark yet.
2026-01-29 09:23:16 INFO  [main] BrokerServer:69 - [BrokerServer id=1] Starting broker
2026-01-29 09:23:16 INFO  [main] AbstractKubernetesConfigProvider:68 - Configuring Kubernetes Secret config provider with configuration {}
2026-01-29 09:23:16 INFO  [main] AbstractKubernetesConfigProvider:123 - Retrieving configuration from Secret kafka-cluster-ca-cert in namespace kafka
2026-01-29 09:23:16 INFO  [main] AbstractKubernetesConfigProvider:123 - Retrieving configuration from Secret kafka-kafka-1 in namespace kafka
2026-01-29 09:23:16 INFO  [kafka-1-metadata-loader-event-handler] MetadataLoader:233 - [MetadataLoader id=1] initializeNewPublishers: the loader is still catching up because we still don't know the high water mark yet.
2026-01-29 09:23:16 INFO  [main] AbstractKubernetesConfigProvider:62 - Closing Kubernetes Secret config provider
2026-01-29 09:23:16 INFO  [broker-1-ThrottledChannelReaper-Fetch] ClientQuotaManager$ThrottledChannelReaper:133 - [broker-1-ThrottledChannelReaper-Fetch]: Starting
2026-01-29 09:23:16 INFO  [broker-1-ThrottledChannelReaper-Produce] ClientQuotaManager$ThrottledChannelReaper:133 - [broker-1-ThrottledChannelReaper-Produce]: Starting
2026-01-29 09:23:16 INFO  [broker-1-ThrottledChannelReaper-Request] ClientQuotaManager$ThrottledChannelReaper:133 - [broker-1-ThrottledChannelReaper-Request]: Starting
2026-01-29 09:23:16 INFO  [broker-1-ThrottledChannelReaper-ControllerMutation] ClientQuotaManager$ThrottledChannelReaper:133 - [broker-1-ThrottledChannelReaper-ControllerMutation]: Starting
2026-01-29 09:23:16 INFO  [kafka-1-metadata-loader-event-handler] MetadataLoader:233 - [MetadataLoader id=1] initializeNewPublishers: the loader is still catching up because we still don't know the high water mark yet.
2026-01-29 09:23:16 INFO  [main] BrokerServer:57 - [BrokerServer id=1] Waiting for controller quorum voters future
2026-01-29 09:23:16 INFO  [main] BrokerServer:60 - [BrokerServer id=1] Finished waiting for controller quorum voters future
2026-01-29 09:23:16 INFO  [kafka-1-metadata-loader-event-handler] MetadataLoader:233 - [MetadataLoader id=1] initializeNewPublishers: the loader is still catching up because we still don't know the high water mark yet.
2026-01-29 09:23:16 INFO  [kafka-1-metadata-loader-event-handler] MetadataLoader:233 - [MetadataLoader id=1] initializeNewPublishers: the loader is still catching up because we still don't know the high water mark yet.
2026-01-29 09:23:17 INFO  [kafka-1-raft-io-thread] KafkaRaftClient:3430 - [RaftManager id=1] Registered the listener org.apache.kafka.image.loader.MetadataLoader@555458969
2026-01-29 09:23:17 INFO  [kafka-1-metadata-loader-event-handler] MetadataLoader:233 - [MetadataLoader id=1] initializeNewPublishers: the loader is still catching up because we still don't know the high water mark yet.
2026-01-29 09:23:17 INFO  [broker-1-to-controller-forwarding-channel-manager] NodeToControllerRequestThread:133 - [broker-1-to-controller-forwarding-channel-manager]: Starting
2026-01-29 09:23:17 INFO  [client-metrics-reaper] SystemTimerReaper$Reaper:133 - [client-metrics-reaper]: Starting
2026-01-29 09:23:17 INFO  [kafka-1-metadata-loader-event-handler] MetadataLoader:233 - [MetadataLoader id=1] initializeNewPublishers: the loader is still catching up because we still don't know the high water mark yet.
2026-01-29 09:23:17 INFO  [kafka-1-metadata-loader-event-handler] MetadataLoader:233 - [MetadataLoader id=1] initializeNewPublishers: the loader is still catching up because we still don't know the high water mark yet.
2026-01-29 09:23:17 INFO  [kafka-1-metadata-loader-event-handler] MetadataLoader:233 - [MetadataLoader id=1] initializeNewPublishers: the loader is still catching up because we still don't know the high water mark yet.
2026-01-29 09:23:17 INFO  [kafka-1-metadata-loader-event-handler] MetadataLoader:233 - [MetadataLoader id=1] initializeNewPublishers: the loader is still catching up because we still don't know the high water mark yet.
2026-01-29 09:23:17 INFO  [kafka-1-raft-io-thread] QuorumState:732 - [RaftManager id=1] Attempting durable transition to FollowerState(fetchTimeoutMs=2000, epoch=1, leader=0, leaderEndpoints=Endpoints(endpoints={ListenerName(CONTROLPLANE-9090)=kafka-controller-kafka-0.kafka-kafka-brokers.kafka.svc/<unresolved>:9090}), votedKey=Optional.empty, voters=[0], highWatermark=Optional.empty, fetchingSnapshot=Optional.empty) from UnattachedState(epoch=1, leaderId=OptionalInt.empty, votedKey=Optional.empty, voters=[0], electionTimeoutMs=1004, highWatermark=Optional.empty)
2026-01-29 09:23:17 INFO  [kafka-1-raft-io-thread] QuorumState:749 - [RaftManager id=1] Completed transition to FollowerState(fetchTimeoutMs=2000, epoch=1, leader=0, leaderEndpoints=Endpoints(endpoints={ListenerName(CONTROLPLANE-9090)=kafka-controller-kafka-0.kafka-kafka-brokers.kafka.svc/<unresolved>:9090}), votedKey=Optional.empty, voters=[0], highWatermark=Optional.empty, fetchingSnapshot=Optional.empty) from UnattachedState(epoch=1, leaderId=OptionalInt.empty, votedKey=Optional.empty, voters=[0], electionTimeoutMs=1004, highWatermark=Optional.empty)
2026-01-29 09:23:17 INFO  [kafka-1-metadata-loader-event-handler] MetadataLoader:233 - [MetadataLoader id=1] initializeNewPublishers: the loader is still catching up because we still don't know the high water mark yet.
2026-01-29 09:23:17 INFO  [broker-1-to-controller-forwarding-channel-manager] NodeToControllerRequestThread:69 - [broker-1-to-controller-forwarding-channel-manager]: Recorded new KRaft controller, from now on will use node kafka-controller-kafka-0.kafka-kafka-brokers.kafka.svc.cluster.local:9090 (id: 0 rack: null isFenced: false)
2026-01-29 09:23:17 INFO  [kafka-1-raft-io-thread] FollowerState:278 - [RaftManager id=1] High watermark set to Optional[LogOffsetMetadata(offset=2752, metadata=Optional.empty)] for the first time for epoch 1
2026-01-29 09:23:17 INFO  [kafka-1-raft-io-thread] KafkaRaftClient:416 - [RaftManager id=1] Setting the next offset of org.apache.kafka.image.loader.MetadataLoader@555458969 to 0 since there are no snapshots
2026-01-29 09:23:17 INFO  [kafka-1-metadata-loader-event-handler] MetadataLoader:238 - [MetadataLoader id=1] maybePublishMetadata(LOG_DELTA): The loader is still catching up because we have loaded up to offset 0, but the high water mark is 2752
2026-01-29 09:23:17 INFO  [SECURITY] [main] ConnectionQuotas:69 - Updated connection-accept-rate max connection creation rate to 2147483647
    2026-01-29 09:23:17 INFO  [kafka-1-metadata-loader-event-handler] MetadataLoader:238 - [MetadataLoader id=1] maybePublishMetadata(LOG_DELTA): The loader is still catching up because we have loaded up to offset 2751, but the high water mark is 2753
2026-01-29 09:23:17 INFO  [kafka-1-metadata-loader-event-handler] MetadataLoader:238 - [MetadataLoader id=1] initializeNewPublishers: The loader is still catching up because we have loaded up to offset 2751, but the high water mark is 2753
2026-01-29 09:23:17 INFO  [kafka-1-metadata-loader-event-handler] MetadataLoader:247 - [MetadataLoader id=1] maybePublishMetadata(LOG_DELTA): The loader finished catching up to the current high water mark of 2753
2026-01-29 09:23:17 INFO  [kafka-1-metadata-loader-event-handler] MetadataLoader:306 - [MetadataLoader id=1] InitializeNewPublishers: initializing SnapshotGenerator with a snapshot at offset 2752
2026-01-29 09:23:18 INFO  [SECURITY] [main] SocketServer:69 - [SocketServer listenerType=BROKER, nodeId=1] Created data-plane acceptor and processors for endpoint : ListenerName(REPLICATION-9091)
    2026-01-29 09:23:18 INFO  [SECURITY] [main] ConnectionQuotas:69 - Updated connection-accept-rate max connection creation rate to 2147483647
    2026-01-29 09:23:18 INFO  [SECURITY] [main] SocketServer:69 - [SocketServer listenerType=BROKER, nodeId=1] Created data-plane acceptor and processors for endpoint : ListenerName(TLS-9093)
    2026-01-29 09:23:18 INFO  [SECURITY] [main] ConnectionQuotas:69 - Updated connection-accept-rate max connection creation rate to 2147483647
    2026-01-29 09:23:19 INFO  [main] JWTSignatureValidator:410 - JWKS keys change detected. Keys updated.
2026-01-29 09:23:19 ERROR [SECURITY] [main] OAuthBearerLoginModule:319 - No principal name in JWT claim: sub
    java.io.IOException: No principal name in JWT claim: sub
	at org.apache.kafka.common.security.oauthbearer.internals.unsecured.OAuthBearerUnsecuredLoginCallbackHandler.handle(OAuthBearerUnsecuredLoginCallbackHandler.java:163) ~[kafka-clients-4.1.0.jar:?]
	at org.apache.kafka.common.security.oauthbearer.OAuthBearerLoginModule.identifyToken(OAuthBearerLoginModule.java:317) ~[kafka-clients-4.1.0.jar:?]
	at org.apache.kafka.common.security.oauthbearer.OAuthBearerLoginModule.login(OAuthBearerLoginModule.java:302) ~[kafka-clients-4.1.0.jar:?]
	at java.base/javax.security.auth.login.LoginContext.invoke(Unknown Source) ~[?:?]
	at java.base/javax.security.auth.login.LoginContext$4.run(Unknown Source) ~[?:?]
	at java.base/javax.security.auth.login.LoginContext$4.run(Unknown Source) ~[?:?]
	at java.base/java.security.AccessController.doPrivileged(Unknown Source) ~[?:?]
	at java.base/javax.security.auth.login.LoginContext.invokePriv(Unknown Source) ~[?:?]
	at java.base/javax.security.auth.login.LoginContext.login(Unknown Source) ~[?:?]
	at org.apache.kafka.common.security.authenticator.AbstractLogin.login(AbstractLogin.java:61) ~[kafka-clients-4.1.0.jar:?]
	at org.apache.kafka.common.security.authenticator.LoginManager.<init>(LoginManager.java:69) ~[kafka-clients-4.1.0.jar:?]
	at org.apache.kafka.common.security.authenticator.LoginManager.acquireLoginManager(LoginManager.java:116) ~[kafka-clients-4.1.0.jar:?]
	at org.apache.kafka.common.network.SaslChannelBuilder.configure(SaslChannelBuilder.java:170) ~[kafka-clients-4.1.0.jar:?]
	at org.apache.kafka.common.network.ChannelBuilders.create(ChannelBuilders.java:188) ~[kafka-clients-4.1.0.jar:?]
	at org.apache.kafka.common.network.ChannelBuilders.serverChannelBuilder(ChannelBuilders.java:105) ~[kafka-clients-4.1.0.jar:?]
	at kafka.network.Processor.<init>(SocketServer.scala:876) ~[kafka_2.13-4.1.0.jar:?]
	at kafka.network.Acceptor.newProcessor(SocketServer.scala:784) ~[kafka_2.13-4.1.0.jar:?]
	at kafka.network.Acceptor.$anonfun$addProcessors$1(SocketServer.scala:750) ~[kafka_2.13-4.1.0.jar:?]
	at scala.collection.immutable.Range.foreach$mVc$sp(Range.scala:192) ~[scala-library-2.13.16.jar:?]
	at kafka.network.Acceptor.addProcessors(SocketServer.scala:749) ~[kafka_2.13-4.1.0.jar:?]
	at kafka.network.DataPlaneAcceptor.configure(SocketServer.scala:467) ~[kafka_2.13-4.1.0.jar:?]
	at kafka.network.SocketServer.createDataPlaneAcceptorAndProcessors(SocketServer.scala:222) ~[kafka_2.13-4.1.0.jar:?]
	at kafka.network.SocketServer.$anonfun$new$16(SocketServer.scala:148) ~[kafka_2.13-4.1.0.jar:?]
	at kafka.network.SocketServer.$anonfun$new$16$adapted(SocketServer.scala:148) ~[kafka_2.13-4.1.0.jar:?]
	at scala.collection.IterableOnceOps.foreach(IterableOnce.scala:619) ~[scala-library-2.13.16.jar:?]
	at scala.collection.IterableOnceOps.foreach$(IterableOnce.scala:617) ~[scala-library-2.13.16.jar:?]
	at scala.collection.AbstractIterable.foreach(Iterable.scala:935) ~[scala-library-2.13.16.jar:?]
	at kafka.network.SocketServer.<init>(SocketServer.scala:148) ~[kafka_2.13-4.1.0.jar:?]
	at kafka.server.BrokerServer.startup(BrokerServer.scala:281) ~[kafka_2.13-4.1.0.jar:?]
	at kafka.server.KafkaRaftServer.$anonfun$startup$2(KafkaRaftServer.scala:95) ~[kafka_2.13-4.1.0.jar:?]
	at kafka.server.KafkaRaftServer.$anonfun$startup$2$adapted(KafkaRaftServer.scala:95) ~[kafka_2.13-4.1.0.jar:?]
	at scala.Option.foreach(Option.scala:437) [scala-library-2.13.16.jar:?]
	at kafka.server.KafkaRaftServer.startup(KafkaRaftServer.scala:95) [kafka_2.13-4.1.0.jar:?]
	at kafka.Kafka$.main(Kafka.scala:97) [kafka_2.13-4.1.0.jar:?]
	at kafka.Kafka.main(Kafka.scala) [kafka_2.13-4.1.0.jar:?]
Caused by: org.apache.kafka.common.security.oauthbearer.internals.unsecured.OAuthBearerConfigException: No principal name in JWT claim: sub
	at org.apache.kafka.common.security.oauthbearer.internals.unsecured.OAuthBearerUnsecuredLoginCallbackHandler.handleTokenCallback(OAuthBearerUnsecuredLoginCallbackHandler.java:217) ~[kafka-clients-4.1.0.jar:?]
	at org.apache.kafka.common.security.oauthbearer.internals.unsecured.OAuthBearerUnsecuredLoginCallbackHandler.handle(OAuthBearerUnsecuredLoginCallbackHandler.java:161) ~[kafka-clients-4.1.0.jar:?]
	... 34 more
Caused by: org.apache.kafka.common.security.oauthbearer.internals.unsecured.OAuthBearerIllegalTokenException: No principal name in JWT claim: sub
	at org.apache.kafka.common.security.oauthbearer.internals.unsecured.OAuthBearerUnsecuredJws.<init>(OAuthBearerUnsecuredJws.java:108) ~[kafka-clients-4.1.0.jar:?]
	at org.apache.kafka.common.security.oauthbearer.internals.unsecured.OAuthBearerUnsecuredLoginCallbackHandler.handleTokenCallback(OAuthBearerUnsecuredLoginCallbackHandler.java:209) ~[kafka-clients-4.1.0.jar:?]
	at org.apache.kafka.common.security.oauthbearer.internals.unsecured.OAuthBearerUnsecuredLoginCallbackHandler.handle(OAuthBearerUnsecuredLoginCallbackHandler.java:161) ~[kafka-clients-4.1.0.jar:?]
	... 34 more
2026-01-29 09:23:19 INFO  [main] BrokerServer:69 - [BrokerServer id=1] Transition from STARTING to STARTED
2026-01-29 09:23:19 ERROR [main] BrokerServer:85 - [BrokerServer id=1] Fatal error during broker startup. Prepare to shutdown
org.apache.kafka.common.KafkaException: javax.security.auth.login.LoginException: An internal error occurred while retrieving token from callback handler
	at org.apache.kafka.common.network.SaslChannelBuilder.configure(SaslChannelBuilder.java:184) ~[kafka-clients-4.1.0.jar:?]
	at org.apache.kafka.common.network.ChannelBuilders.create(ChannelBuilders.java:188) ~[kafka-clients-4.1.0.jar:?]
	at org.apache.kafka.common.network.ChannelBuilders.serverChannelBuilder(ChannelBuilders.java:105) ~[kafka-clients-4.1.0.jar:?]
	at kafka.network.Processor.<init>(SocketServer.scala:876) ~[kafka_2.13-4.1.0.jar:?]
	at kafka.network.Acceptor.newProcessor(SocketServer.scala:784) ~[kafka_2.13-4.1.0.jar:?]
	at kafka.network.Acceptor.$anonfun$addProcessors$1(SocketServer.scala:750) ~[kafka_2.13-4.1.0.jar:?]
	at scala.collection.immutable.Range.foreach$mVc$sp(Range.scala:192) ~[scala-library-2.13.16.jar:?]
	at kafka.network.Acceptor.addProcessors(SocketServer.scala:749) ~[kafka_2.13-4.1.0.jar:?]
	at kafka.network.DataPlaneAcceptor.configure(SocketServer.scala:467) ~[kafka_2.13-4.1.0.jar:?]
	at kafka.network.SocketServer.createDataPlaneAcceptorAndProcessors(SocketServer.scala:222) ~[kafka_2.13-4.1.0.jar:?]
	at kafka.network.SocketServer.$anonfun$new$16(SocketServer.scala:148) ~[kafka_2.13-4.1.0.jar:?]
	at kafka.network.SocketServer.$anonfun$new$16$adapted(SocketServer.scala:148) ~[kafka_2.13-4.1.0.jar:?]
	at scala.collection.IterableOnceOps.foreach(IterableOnce.scala:619) ~[scala-library-2.13.16.jar:?]
	at scala.collection.IterableOnceOps.foreach$(IterableOnce.scala:617) ~[scala-library-2.13.16.jar:?]
	at scala.collection.AbstractIterable.foreach(Iterable.scala:935) ~[scala-library-2.13.16.jar:?]
	at kafka.network.SocketServer.<init>(SocketServer.scala:148) ~[kafka_2.13-4.1.0.jar:?]
	at kafka.server.BrokerServer.startup(BrokerServer.scala:281) ~[kafka_2.13-4.1.0.jar:?]
	at kafka.server.KafkaRaftServer.$anonfun$startup$2(KafkaRaftServer.scala:95) ~[kafka_2.13-4.1.0.jar:?]
	at kafka.server.KafkaRaftServer.$anonfun$startup$2$adapted(KafkaRaftServer.scala:95) ~[kafka_2.13-4.1.0.jar:?]
	at scala.Option.foreach(Option.scala:437) [scala-library-2.13.16.jar:?]
	at kafka.server.KafkaRaftServer.startup(KafkaRaftServer.scala:95) [kafka_2.13-4.1.0.jar:?]
	at kafka.Kafka$.main(Kafka.scala:97) [kafka_2.13-4.1.0.jar:?]
	at kafka.Kafka.main(Kafka.scala) [kafka_2.13-4.1.0.jar:?]
Caused by: javax.security.auth.login.LoginException: An internal error occurred while retrieving token from callback handler
	at org.apache.kafka.common.security.oauthbearer.OAuthBearerLoginModule.identifyToken(OAuthBearerLoginModule.java:320) ~[kafka-clients-4.1.0.jar:?]
	at org.apache.kafka.common.security.oauthbearer.OAuthBearerLoginModule.login(OAuthBearerLoginModule.java:302) ~[kafka-clients-4.1.0.jar:?]
	at java.base/javax.security.auth.login.LoginContext.invoke(Unknown Source) ~[?:?]
	at java.base/javax.security.auth.login.LoginContext$4.run(Unknown Source) ~[?:?]
	at java.base/javax.security.auth.login.LoginContext$4.run(Unknown Source) ~[?:?]
	at java.base/java.security.AccessController.doPrivileged(Unknown Source) ~[?:?]
	at java.base/javax.security.auth.login.LoginContext.invokePriv(Unknown Source) ~[?:?]
	at java.base/javax.security.auth.login.LoginContext.login(Unknown Source) ~[?:?]
	at org.apache.kafka.common.security.authenticator.AbstractLogin.login(AbstractLogin.java:61) ~[kafka-clients-4.1.0.jar:?]
	at org.apache.kafka.common.security.authenticator.LoginManager.<init>(LoginManager.java:69) ~[kafka-clients-4.1.0.jar:?]
	at org.apache.kafka.common.security.authenticator.LoginManager.acquireLoginManager(LoginManager.java:116) ~[kafka-clients-4.1.0.jar:?]
	at org.apache.kafka.common.network.SaslChannelBuilder.configure(SaslChannelBuilder.java:170) ~[kafka-clients-4.1.0.jar:?]
	... 22 more
2026-01-29 09:23:19 INFO  [main] BrokerServer:69 - [BrokerServer id=1] Transition from STARTED to SHUTTING_DOWN
2026-01-29 09:23:19 INFO  [main] BrokerServer:69 - [BrokerServer id=1] shutting down
2026-01-29 09:23:19 INFO  [broker-1-lifecycle-manager-event-handler] BrokerLifecycleManager:69 - [BrokerLifecycleManager id=1] Skipping controlled shutdown because we are in state NOT_RUNNING.
2026-01-29 09:23:19 INFO  [broker-1-lifecycle-manager-event-handler] KafkaEventQueue:498 - [BrokerLifecycleManager id=1] beginShutdown: shutting down event queue.
2026-01-29 09:23:19 INFO  [broker-1-lifecycle-manager-event-handler] BrokerLifecycleManager:69 - [BrokerLifecycleManager id=1] Transitioning from NOT_RUNNING to SHUTTING_DOWN.
2026-01-29 09:23:19 INFO  [main] NodeToControllerRequestThread:91 - [broker-1-to-controller-forwarding-channel-manager]: Shutting down
2026-01-29 09:23:19 INFO  [broker-1-to-controller-forwarding-channel-manager] NodeToControllerRequestThread:148 - [broker-1-to-controller-forwarding-channel-manager]: Stopped
2026-01-29 09:23:19 INFO  [main] NodeToControllerRequestThread:110 - [broker-1-to-controller-forwarding-channel-manager]: Shutdown completed
2026-01-29 09:23:19 INFO  [main] NodeToControllerChannelManagerImpl:69 - Node to controller channel manager for forwarding shutdown
2026-01-29 09:23:19 INFO  [main] LogManager:69 - Shutting down.
2026-01-29 09:23:19 INFO  [main] LogManager:69 - Shutdown complete.
2026-01-29 09:23:19 INFO  [main] ClientQuotaManager$ThrottledChannelReaper:91 - [broker-1-ThrottledChannelReaper-Fetch]: Shutting down
2026-01-29 09:23:19 INFO  [broker-1-ThrottledChannelReaper-Fetch] ClientQuotaManager$ThrottledChannelReaper:148 - [broker-1-ThrottledChannelReaper-Fetch]: Stopped
2026-01-29 09:23:19 INFO  [main] ClientQuotaManager$ThrottledChannelReaper:110 - [broker-1-ThrottledChannelReaper-Fetch]: Shutdown completed
2026-01-29 09:23:19 INFO  [main] ClientQuotaManager$ThrottledChannelReaper:91 - [broker-1-ThrottledChannelReaper-Produce]: Shutting down
2026-01-29 09:23:19 INFO  [broker-1-ThrottledChannelReaper-Produce] ClientQuotaManager$ThrottledChannelReaper:148 - [broker-1-ThrottledChannelReaper-Produce]: Stopped
2026-01-29 09:23:19 INFO  [main] ClientQuotaManager$ThrottledChannelReaper:110 - [broker-1-ThrottledChannelReaper-Produce]: Shutdown completed
2026-01-29 09:23:19 INFO  [main] ClientQuotaManager$ThrottledChannelReaper:91 - [broker-1-ThrottledChannelReaper-Request]: Shutting down
2026-01-29 09:23:19 INFO  [broker-1-ThrottledChannelReaper-Request] ClientQuotaManager$ThrottledChannelReaper:148 - [broker-1-ThrottledChannelReaper-Request]: Stopped
2026-01-29 09:23:19 INFO  [main] ClientQuotaManager$ThrottledChannelReaper:110 - [broker-1-ThrottledChannelReaper-Request]: Shutdown completed
2026-01-29 09:23:19 INFO  [main] ClientQuotaManager$ThrottledChannelReaper:91 - [broker-1-ThrottledChannelReaper-ControllerMutation]: Shutting down
2026-01-29 09:23:19 INFO  [broker-1-ThrottledChannelReaper-ControllerMutation] ClientQuotaManager$ThrottledChannelReaper:148 - [broker-1-ThrottledChannelReaper-ControllerMutation]: Stopped
2026-01-29 09:23:19 INFO  [main] ClientQuotaManager$ThrottledChannelReaper:110 - [broker-1-ThrottledChannelReaper-ControllerMutation]: Shutdown completed
2026-01-29 09:23:19 INFO  [main] BrokerTopicStats:260 - Broker and topic stats closed
2026-01-29 09:23:19 INFO  [main] KafkaEventQueue:520 - [BrokerLifecycleManager id=1] closed event queue.
2026-01-29 09:23:19 INFO  [main] SystemTimerReaper$Reaper:91 - [client-metrics-reaper]: Shutting down
2026-01-29 09:23:19 INFO  [main] SystemTimerReaper$Reaper:110 - [client-metrics-reaper]: Shutdown completed
2026-01-29 09:23:19 INFO  [client-metrics-reaper] SystemTimerReaper$Reaper:148 - [client-metrics-reaper]: Stopped
2026-01-29 09:23:19 INFO  [main] SharedServer:69 - [SharedServer id=1] Stopping SharedServer
2026-01-29 09:23:19 INFO  [main] KafkaEventQueue:498 - [MetadataLoader id=1] beginShutdown: shutting down event queue.
2026-01-29 09:23:19 INFO  [kafka-1-metadata-loader-event-handler] KafkaEventQueue:498 - [SnapshotGenerator id=1] close: shutting down event queue.
2026-01-29 09:23:19 INFO  [kafka-1-metadata-loader-event-handler] KafkaEventQueue:520 - [SnapshotGenerator id=1] closed event queue.
2026-01-29 09:23:19 INFO  [main] KafkaEventQueue:520 - [MetadataLoader id=1] closed event queue.
2026-01-29 09:23:19 INFO  [main] KafkaEventQueue:520 - [SnapshotGenerator id=1] closed event queue.
2026-01-29 09:23:19 INFO  [main] TimingWheelExpirationService$ExpiredOperationReaper:91 - [raft-expiration-reaper]: Shutting down
2026-01-29 09:23:19 INFO  [main] TimingWheelExpirationService$ExpiredOperationReaper:110 - [raft-expiration-reaper]: Shutdown completed
2026-01-29 09:23:19 INFO  [raft-expiration-reaper] TimingWheelExpirationService$ExpiredOperationReaper:148 - [raft-expiration-reaper]: Stopped
2026-01-29 09:23:19 INFO  [main] KafkaRaftClientDriver:91 - [kafka-1-raft-io-thread]: Shutting down
2026-01-29 09:23:19 INFO  [main] KafkaRaftClient:3594 - [RaftManager id=1] Beginning graceful shutdown
2026-01-29 09:23:19 INFO  [kafka-1-raft-io-thread] KafkaRaftClient:3786 - [RaftManager id=1] Graceful shutdown completed
2026-01-29 09:23:19 INFO  [kafka-1-raft-io-thread] KafkaRaftClientDriver:148 - [kafka-1-raft-io-thread]: Stopped
2026-01-29 09:23:19 INFO  [main] KafkaRaftClientDriver:77 - [RaftManager id=1] Completed graceful shutdown of RaftClient
2026-01-29 09:23:19 INFO  [main] KafkaRaftClientDriver:110 - [kafka-1-raft-io-thread]: Shutdown completed
2026-01-29 09:23:19 INFO  [main] KafkaNetworkChannel$SendThread:91 - [kafka-1-raft-outbound-request-thread]: Shutting down
2026-01-29 09:23:19 INFO  [kafka-1-raft-outbound-request-thread] KafkaNetworkChannel$SendThread:148 - [kafka-1-raft-outbound-request-thread]: Stopped
2026-01-29 09:23:19 INFO  [main] KafkaNetworkChannel$SendThread:110 - [kafka-1-raft-outbound-request-thread]: Shutdown completed
2026-01-29 09:23:19 INFO  [main] ProducerStateManager:441 - [ProducerStateManager partition=__cluster_metadata-0] Wrote producer snapshot at offset 2755 with 0 producer ids in 4 ms.
2026-01-29 09:23:19 INFO  [main] Metrics:684 - Metrics scheduler closed
2026-01-29 09:23:19 INFO  [main] Metrics:694 - Metrics reporters closed
2026-01-29 09:23:19 INFO  [main] AppInfoParser:89 - App info kafka.server for 1 unregistered
2026-01-29 09:23:19 INFO  [main] BrokerServer:69 - [BrokerServer id=1] shut down completed
2026-01-29 09:23:19 INFO  [main] BrokerServer:69 - [BrokerServer id=1] Transition from SHUTTING_DOWN to SHUTDOWN
2026-01-29 09:23:19 ERROR [main] Kafka$:28 - Exiting Kafka due to fatal exception during startup.
org.apache.kafka.common.KafkaException: javax.security.auth.login.LoginException: An internal error occurred while retrieving token from callback handler
	at org.apache.kafka.common.network.SaslChannelBuilder.configure(SaslChannelBuilder.java:184) ~[kafka-clients-4.1.0.jar:?]
	at org.apache.kafka.common.network.ChannelBuilders.create(ChannelBuilders.java:188) ~[kafka-clients-4.1.0.jar:?]
	at org.apache.kafka.common.network.ChannelBuilders.serverChannelBuilder(ChannelBuilders.java:105) ~[kafka-clients-4.1.0.jar:?]
	at kafka.network.Processor.<init>(SocketServer.scala:876) ~[kafka_2.13-4.1.0.jar:?]
	at kafka.network.Acceptor.newProcessor(SocketServer.scala:784) ~[kafka_2.13-4.1.0.jar:?]
	at kafka.network.Acceptor.$anonfun$addProcessors$1(SocketServer.scala:750) ~[kafka_2.13-4.1.0.jar:?]
	at scala.collection.immutable.Range.foreach$mVc$sp(Range.scala:192) ~[scala-library-2.13.16.jar:?]
	at kafka.network.Acceptor.addProcessors(SocketServer.scala:749) ~[kafka_2.13-4.1.0.jar:?]
	at kafka.network.DataPlaneAcceptor.configure(SocketServer.scala:467) ~[kafka_2.13-4.1.0.jar:?]
	at kafka.network.SocketServer.createDataPlaneAcceptorAndProcessors(SocketServer.scala:222) ~[kafka_2.13-4.1.0.jar:?]
	at kafka.network.SocketServer.$anonfun$new$16(SocketServer.scala:148) ~[kafka_2.13-4.1.0.jar:?]
	at kafka.network.SocketServer.$anonfun$new$16$adapted(SocketServer.scala:148) ~[kafka_2.13-4.1.0.jar:?]
	at scala.collection.IterableOnceOps.foreach(IterableOnce.scala:619) ~[scala-library-2.13.16.jar:?]
	at scala.collection.IterableOnceOps.foreach$(IterableOnce.scala:617) ~[scala-library-2.13.16.jar:?]
	at scala.collection.AbstractIterable.foreach(Iterable.scala:935) ~[scala-library-2.13.16.jar:?]
	at kafka.network.SocketServer.<init>(SocketServer.scala:148) ~[kafka_2.13-4.1.0.jar:?]
	at kafka.server.BrokerServer.startup(BrokerServer.scala:281) ~[kafka_2.13-4.1.0.jar:?]
	at kafka.server.KafkaRaftServer.$anonfun$startup$2(KafkaRaftServer.scala:95) ~[kafka_2.13-4.1.0.jar:?]
	at kafka.server.KafkaRaftServer.$anonfun$startup$2$adapted(KafkaRaftServer.scala:95) ~[kafka_2.13-4.1.0.jar:?]
	at scala.Option.foreach(Option.scala:437) ~[scala-library-2.13.16.jar:?]
	at kafka.server.KafkaRaftServer.startup(KafkaRaftServer.scala:95) ~[kafka_2.13-4.1.0.jar:?]
	at kafka.Kafka$.main(Kafka.scala:97) [kafka_2.13-4.1.0.jar:?]
	at kafka.Kafka.main(Kafka.scala) [kafka_2.13-4.1.0.jar:?]
Caused by: javax.security.auth.login.LoginException: An internal error occurred while retrieving token from callback handler
	at org.apache.kafka.common.security.oauthbearer.OAuthBearerLoginModule.identifyToken(OAuthBearerLoginModule.java:320) ~[kafka-clients-4.1.0.jar:?]
	at org.apache.kafka.common.security.oauthbearer.OAuthBearerLoginModule.login(OAuthBearerLoginModule.java:302) ~[kafka-clients-4.1.0.jar:?]
	at java.base/javax.security.auth.login.LoginContext.invoke(Unknown Source) ~[?:?]
	at java.base/javax.security.auth.login.LoginContext$4.run(Unknown Source) ~[?:?]
	at java.base/javax.security.auth.login.LoginContext$4.run(Unknown Source) ~[?:?]
	at java.base/java.security.AccessController.doPrivileged(Unknown Source) ~[?:?]
	at java.base/javax.security.auth.login.LoginContext.invokePriv(Unknown Source) ~[?:?]
	at java.base/javax.security.auth.login.LoginContext.login(Unknown Source) ~[?:?]
	at org.apache.kafka.common.security.authenticator.AbstractLogin.login(AbstractLogin.java:61) ~[kafka-clients-4.1.0.jar:?]
	at org.apache.kafka.common.security.authenticator.LoginManager.<init>(LoginManager.java:69) ~[kafka-clients-4.1.0.jar:?]
	at org.apache.kafka.common.security.authenticator.LoginManager.acquireLoginManager(LoginManager.java:116) ~[kafka-clients-4.1.0.jar:?]
	at org.apache.kafka.common.network.SaslChannelBuilder.configure(SaslChannelBuilder.java:170) ~[kafka-clients-4.1.0.jar:?]
	... 22 more
2026-01-29 09:23:19 INFO  [JettyShutdownThread] Server:652 - Stopped oejs.Server@4441d567{STOPPING}[12.0.22,sto=30000]
2026-01-29 09:23:19 INFO  [kafka-shutdown-hook] AppInfoParser:89 - App info kafka.server for 1 unregistered
2026-01-29 09:23:19 INFO  [JettyShutdownThread] Server:141 - Shutdown oejs.Server@4441d567{STOPPING}[12.0.22,sto=30000]
2026-01-29 09:23:19 INFO  [JettyShutdownThread] AbstractConnector:381 - Stopped ServerConnector@3d98d138{SSL, (ssl, http/1.1)}{0.0.0.0:8443}
2026-01-29 09:23:19 INFO  [JettyShutdownThread] AbstractConnector:381 - Stopped ServerConnector@60fb2bf6{HTTP/1.1, (http/1.1)}{localhost:8080}

ConfigMap generated by Kafka NodePool:

apiVersion: v1
data:
  cluster.id: 3bV6k2TNQjqbODi-A9mt_w
  log4j2.properties: >
    name=CustomKafkaConfig

    appender.console.type=Console

    appender.console.name=STDOUT

    appender.console.layout.type=PatternLayout

    appender.console.layout.pattern=%d{yyyy-MM-dd HH:mm:ss} %-5p [%t] %c{1}:%L -
    %m%n


    appender.securityAppender.type=Console

    appender.securityAppender.name=SECURITY

    appender.securityAppender.layout.type=PatternLayout

    appender.securityAppender.layout.pattern=%d{yyyy-MM-dd HH:mm:ss} %-5p
    [SECURITY] [%t] %c{1}:%L - %m%n    


    rootLogger.level=INFO

    rootLogger.appenderRefs=console

    rootLogger.appenderRef.console.ref=STDOUT

    rootLogger.additivity=false

    logger.kafka.name=kafka

    logger.kafka.level=INFO

    logger.kafka.appenderRefs=console

    logger.kafka.appenderRef.console.ref=STDOUT

    logger.kafka.additivity=false

    logger.orgapachekafka.name=org.apache.kafka

    logger.orgapachekafka.level=INFO

    logger.orgapachekafka.appenderRefs=console

    logger.orgapachekafka.appenderRef.console.ref=STDOUT

    logger.orgapachekafka.additivity=false

    logger.requestlogger.name=kafka.request.logger

    logger.requestlogger.level=WARN

    logger.requestlogger.appenderRefs=console

    logger.requestlogger.appenderRef.console.ref=STDOUT

    logger.requestlogger.additivity=false

    logger.requestchannel.name=kafka.network.RequestChannel$

    logger.requestchannel.level=WARN

    logger.requestchannel.appenderRefs=console

    logger.requestchannel.appenderRef.console.ref=STDOUT

    logger.requestchannel.additivity=false

    logger.controller.name=org.apache.kafka.controller

    logger.controller.level=INFO

    logger.controller.appenderRefs=console

    logger.controller.appenderRef.console.ref=STDOUT

    logger.controller.additivity=false

    logger.logcleaner.name=kafka.log.LogCleaner

    logger.logcleaner.level=INFO

    logger.logcleaner.appenderRefs=console

    logger.logcleaner.appenderRef.console.ref=STDOUT

    logger.logcleaner.additivity=false

    logger.statechange.name=state.change.logger

    logger.statechange.level=INFO

    logger.statechange.appenderRefs=console

    logger.statechange.appenderRef.console.ref=STDOUT

    logger.statechange.additivity=false

    logger.authorizer.name=kafka.authorizer.logger

    logger.authorizer.level=INFO

    logger.authorizer.appenderRefs=console

    logger.authorizer.appenderRef.console.ref=STDOUT

    logger.authorizer.additivity=false


    logger.security.name=org.apache.kafka.common.security

    logger.security.level=INFO

    logger.security.appenderRefs=securityAppender

    logger.security.appenderRef.securityAppender.ref=SECURITY

    logger.security.additivity=false


    logger.networkjava.name=org.apache.kafka.common.network

    logger.networkjava.level=INFO

    logger.networkjava.appenderRefs=securityAppender

    logger.networkjava.appenderRef.securityAppender.ref=SECURITY

    logger.networkjava.additivity=false


    logger.networkscala.name=kafka.network

    logger.networkscala.level=INFO

    logger.networkscala.appenderRefs=securityAppender

    logger.networkscala.appenderRef.securityAppender.ref=SECURITY

    logger.networkscala.additivity=false


    logger.auth.name=org.apache.kafka.server.authorizer

    logger.auth.level=INFO

    logger.auth.appenderRefs=securityAppender

    logger.auth.appenderRef.securityAppender.ref=SECURITY

    logger.auth.additivity=false


    logger.topiceventhandler.name=io.strimzi.operator.topic

    logger.topiceventhandler.level=WARN

    logger.topiceventhandler.appenderRefs=console

    logger.topiceventhandler.appenderRef.console.ref=STDOUT

    logger.topiceventhandler.additivity=false


    monitorInterval=30
  metadata.version: '4.1'
  metrics-config.json: >-
    {"lowercaseOutputName":true,"rules":[{"pattern":"kafka.server<type=(.+),
    name=(.+), clientId=(.+), topic=(.+),
    partition=(.*)><>Value","name":"kafka_server_$1_$2","type":"GAUGE","labels":{"clientId":"$3","topic":"$4","partition":"$5"}},{"pattern":"kafka.server<type=(.+),
    name=(.+), clientId=(.+), brokerHost=(.+),
    brokerPort=(.+)><>Value","name":"kafka_server_$1_$2","type":"GAUGE","labels":{"clientId":"$3","broker":"$4:$5"}},{"pattern":"kafka.server<type=(.+),
    cipher=(.+), protocol=(.+), listener=(.+),
    networkProcessor=(.+)><>connections","name":"kafka_server_$1_connections_tls_info","type":"GAUGE","labels":{"cipher":"$2","protocol":"$3","listener":"$4","networkProcessor":"$5"}},{"pattern":"kafka.server<type=(.+),
    clientSoftwareName=(.+), clientSoftwareVersion=(.+), listener=(.+),
    networkProcessor=(.+)><>connections","name":"kafka_server_$1_connections_software","type":"GAUGE","labels":{"clientSoftwareName":"$2","clientSoftwareVersion":"$3","listener":"$4","networkProcessor":"$5"}},{"pattern":"kafka.server<type=(.+),
    listener=(.+),
    networkProcessor=(.+)><>(.+):","name":"kafka_server_$1_$4","type":"GAUGE","labels":{"listener":"$2","networkProcessor":"$3"}},{"pattern":"kafka.server<type=(.+),
    listener=(.+),
    networkProcessor=(.+)><>(.+)","name":"kafka_server_$1_$4","type":"GAUGE","labels":{"listener":"$2","networkProcessor":"$3"}},{"pattern":"kafka.(\\w+)<type=(.+),
    name=(.+)Percent\\w*><>MeanRate","name":"kafka_$1_$2_$3_percent","type":"GAUGE"},{"pattern":"kafka.(\\w+)<type=(.+),
    name=(.+)Percent\\w*><>Value","name":"kafka_$1_$2_$3_percent","type":"GAUGE"},{"pattern":"kafka.(\\w+)<type=(.+),
    name=(.+)Percent\\w*,
    (.+)=(.+)><>Value","name":"kafka_$1_$2_$3_percent","type":"GAUGE","labels":{"$4":"$5"}},{"pattern":"kafka.(\\w+)<type=(.+),
    name=(.+)PerSec\\w*, (.+)=(.+),
    (.+)=(.+)><>Count","name":"kafka_$1_$2_$3_total","type":"COUNTER","labels":{"$4":"$5","$6":"$7"}},{"pattern":"kafka.(\\w+)<type=(.+),
    name=(.+)PerSec\\w*,
    (.+)=(.+)><>Count","name":"kafka_$1_$2_$3_total","type":"COUNTER","labels":{"$4":"$5"}},{"pattern":"kafka.(\\w+)<type=(.+),
    name=(.+)PerSec\\w*><>Count","name":"kafka_$1_$2_$3_total","type":"COUNTER"},{"pattern":"kafka.(\\w+)<type=(.+),
    name=(.+), (.+)=(.+),
    (.+)=(.+)><>Value","name":"kafka_$1_$2_$3","type":"GAUGE","labels":{"$4":"$5","$6":"$7"}},{"pattern":"kafka.(\\w+)<type=(.+),
    name=(.+),
    (.+)=(.+)><>Value","name":"kafka_$1_$2_$3","type":"GAUGE","labels":{"$4":"$5"}},{"pattern":"kafka.(\\w+)<type=(.+),
    name=(.+)><>Value","name":"kafka_$1_$2_$3","type":"GAUGE"},{"pattern":"kafka.(\\w+)<type=(.+),
    name=(.+), (.+)=(.+),
    (.+)=(.+)><>Count","name":"kafka_$1_$2_$3_count","type":"COUNTER","labels":{"$4":"$5","$6":"$7"}},{"pattern":"kafka.(\\w+)<type=(.+),
    name=(.+), (.+)=(.*),
    (.+)=(.+)><>(\\d+)thPercentile","name":"kafka_$1_$2_$3","type":"GAUGE","labels":{"$4":"$5","$6":"$7","quantile":"0.$8"}},{"pattern":"kafka.(\\w+)<type=(.+),
    name=(.+),
    (.+)=(.+)><>Count","name":"kafka_$1_$2_$3_count","type":"COUNTER","labels":{"$4":"$5"}},{"pattern":"kafka.(\\w+)<type=(.+),
    name=(.+),
    (.+)=(.*)><>(\\d+)thPercentile","name":"kafka_$1_$2_$3","type":"GAUGE","labels":{"$4":"$5","quantile":"0.$6"}},{"pattern":"kafka.(\\w+)<type=(.+),
    name=(.+)><>Count","name":"kafka_$1_$2_$3_count","type":"COUNTER"},{"pattern":"kafka.(\\w+)<type=(.+),
    name=(.+)><>(\\d+)thPercentile","name":"kafka_$1_$2_$3","type":"GAUGE","labels":{"quantile":"0.$4"}}]}
  server.config: >-
    ##############################

    ##############################

    # This file is automatically generated by the Strimzi Cluster Operator

    # Any changes to this file will be ignored and overwritten!

    ##############################

    ##############################


    ##########

    # Node ID

    ##########

    node.id=1


    ##########

    # KRaft configuration

    ##########

    process.roles=broker

    controller.listener.names=CONTROLPLANE-9090

    controller.quorum.voters=0@kafka-controller-kafka-0.kafka-kafka-brokers.kafka.svc.cluster.local:9090


    ##########

    # KRaft metadata log dir configuration

    ##########

    metadata.log.dir=/var/lib/kafka/data-0/kafka-log1


    ##########

    # Kafka message logs configuration

    ##########

    log.dirs=/var/lib/kafka/data-0/kafka-log1


    ##########

    # Control Plane listener

    ##########

    listener.name.controlplane-9090.ssl.keystore.certificate.chain=${strimzisecrets:kafka/kafka-kafka-1:kafka-kafka-1.crt}

    listener.name.controlplane-9090.ssl.keystore.key=${strimzisecrets:kafka/kafka-kafka-1:kafka-kafka-1.key}

    listener.name.controlplane-9090.ssl.keystore.type=PEM

    listener.name.controlplane-9090.ssl.truststore.certificates=${strimzisecrets:kafka/kafka-cluster-ca-cert:*.crt}

    listener.name.controlplane-9090.ssl.truststore.type=PEM

    listener.name.controlplane-9090.ssl.client.auth=required


    ##########

    # Replication listener

    ##########

    listener.name.replication-9091.ssl.keystore.certificate.chain=${strimzisecrets:kafka/kafka-kafka-1:kafka-kafka-1.crt}

    listener.name.replication-9091.ssl.keystore.key=${strimzisecrets:kafka/kafka-kafka-1:kafka-kafka-1.key}

    listener.name.replication-9091.ssl.keystore.type=PEM

    listener.name.replication-9091.ssl.truststore.certificates=${strimzisecrets:kafka/kafka-cluster-ca-cert:*.crt}

    listener.name.replication-9091.ssl.truststore.type=PEM

    listener.name.replication-9091.ssl.client.auth=required


    ##########

    # Listener configuration: TLS-9093

    ##########

    listener.name.tls-9093.ssl.keystore.certificate.chain=${strimzisecrets:kafka/kafka-kafka-1:kafka-kafka-1.crt}

    listener.name.tls-9093.ssl.keystore.key=${strimzisecrets:kafka/kafka-kafka-1:kafka-kafka-1.key}

    listener.name.tls-9093.ssl.keystore.type=PEM



    ##########

    # Listener configuration: IAM-9098

    ##########

    listener.name.iam-9098.oauthbearer.sasl.jaas.config=org.apache.kafka.common.security.oauthbearer.OAuthBearerLoginModule
    required
    oauth.valid.issuer.uri="https://url"
    oauth.jwks.endpoint.uri="https://url/protocol/openid-connect/certs"
    oauth.username.claim="preferred_username"
    oauth.ssl.truststore.location="/mnt/oauth-certs/ca.crt"
    oauth.ssl.truststore.type="PEM";

    listener.name.iam-9098.oauthbearer.sasl.server.callback.handler.class=io.strimzi.kafka.oauth.server.JaasServerOauthValidatorCallbackHandler

    listener.name.iam-9098.sasl.enabled.mechanisms=OAUTHBEARER


    ##########

    # Common listener configuration

    ##########

    listener.security.protocol.map=CONTROLPLANE-9090:SSL,REPLICATION-9091:SSL,TLS-9093:SSL,IAM-9098:SASL_PLAINTEXT

    listeners=REPLICATION-9091://0.0.0.0:9091,TLS-9093://0.0.0.0:9093,IAM-9098://0.0.0.0:9098

    inter.broker.listener.name=REPLICATION-9091

    advertised.listeners=REPLICATION-9091://kafka-kafka-1.kafka-kafka-brokers.kafka.svc:9091,TLS-9093://kafka-kafka-1.kafka-kafka-brokers.kafka.svc.cluster.local:9093,IAM-9098://kafka-kafka-1.kafka-kafka-brokers.kafka.svc.cluster.local:9098

    sasl.enabled.mechanisms=

    ssl.endpoint.identification.algorithm=HTTPS


    ##########

    # Config providers

    ##########

    # Configuration providers configured by the user and by Strimzi

    config.providers=strimzienv,strimzisecrets,strimzifile,strimzidir

    config.providers.strimzienv.class=org.apache.kafka.common.config.provider.EnvVarConfigProvider

    config.providers.strimzienv.param.allowlist.pattern=.*

    config.providers.strimzisecrets.class=io.strimzi.kafka.KubernetesSecretConfigProvider

    config.providers.strimzifile.class=org.apache.kafka.common.config.provider.FileConfigProvider

    config.providers.strimzifile.param.allowed.paths=/opt/kafka

    config.providers.strimzidir.class=org.apache.kafka.common.config.provider.DirectoryConfigProvider

    config.providers.strimzidir.param.allowed.paths=/opt/kafka


    ##########

    # metric.reporters configuration

    ##########

    # metric.reporters configured by Strimzi

    metric.reporters=org.apache.kafka.common.metrics.JmxReporter


    ##########

    # User provided configuration

    ##########

    min.insync.replicas=1

    auto.create.topics.enable=false

    compression.type=snappy

    default.replication.factor=1

    log.segment.bytes=104857600

    num.io.threads=25

    num.network.threads=10

    num.partitions=1

    offsets.topic.replication.factor=1

    principal.builder.class=io.strimzi.kafka.oauth.server.OAuthKafkaPrincipalBuilder

    transaction.state.log.min.isr=1

    transaction.state.log.replication.factor=1
kind: ConfigMap
metadata:
  labels:
    app.kubernetes.io/instance: kafka
    app.kubernetes.io/managed-by: strimzi-cluster-operator
    app.kubernetes.io/name: kafka
    app.kubernetes.io/part-of: strimzi-kafka
    strimzi.io/cluster: kafka
    strimzi.io/component-type: kafka
    strimzi.io/kind: Kafka
    strimzi.io/name: kafka-kafka
    strimzi.io/pod-name: kafka-kafka-1
    strimzi.io/pool-name: kafka
  name: kafka-kafka-1
  namespace: kafka
  ownerReferences:
    - apiVersion: kafka.strimzi.io/v1beta2
      blockOwnerDeletion: true
      controller: false
      kind: KafkaNodePool
      name: kafka
      uid: 68643f6b-b47f-4803-86f7-226acf7c83dc

Kafka controller nodepool resource configuration:

apiVersion: kafka.strimzi.io/v1
kind: KafkaNodePool
metadata:
  name: controller-kafka
  namespace: kafka
spec:
  replicas: 1
  roles:
    - controller
  storage:
    type: jbod
    volumes:
      - deleteClaim: false
        id: 1
        kraftMetadata: shared
        size: 1Gi
        type: persistent-claim
status:
  clusterId: 3bV6k2TNQjqbODi-A9mt_w
  conditions: []
  labelSelector: >-
    strimzi.io/cluster=kafka,strimzi.io/name=kafka-kafka,strimzi.io/kind=Kafka,strimzi.io/pool-name=controller-kafka
  nodeIds:
    - 0
  observedGeneration: 1
  replicas: 1
  roles:
    - controller

ConfigMap generated by Kafka controller NodePool:

apiVersion: v1
data:
  cluster.id: 3bV6k2TNQjqbODi-A9mt_w
  log4j2.properties: >
    name=CustomKafkaConfig

    appender.console.type=Console

    appender.console.name=STDOUT

    appender.console.layout.type=PatternLayout

    appender.console.layout.pattern=%d{yyyy-MM-dd HH:mm:ss} %-5p [%t] %c{1}:%L -
    %m%n


    appender.securityAppender.type=Console

    appender.securityAppender.name=SECURITY

    appender.securityAppender.layout.type=PatternLayout

    appender.securityAppender.layout.pattern=%d{yyyy-MM-dd HH:mm:ss} %-5p
    [SECURITY] [%t] %c{1}:%L - %m%n    


    rootLogger.level=INFO

    rootLogger.appenderRefs=console

    rootLogger.appenderRef.console.ref=STDOUT

    rootLogger.additivity=false

    logger.kafka.name=kafka

    logger.kafka.level=INFO

    logger.kafka.appenderRefs=console

    logger.kafka.appenderRef.console.ref=STDOUT

    logger.kafka.additivity=false

    logger.orgapachekafka.name=org.apache.kafka

    logger.orgapachekafka.level=INFO

    logger.orgapachekafka.appenderRefs=console

    logger.orgapachekafka.appenderRef.console.ref=STDOUT

    logger.orgapachekafka.additivity=false

    logger.requestlogger.name=kafka.request.logger

    logger.requestlogger.level=WARN

    logger.requestlogger.appenderRefs=console

    logger.requestlogger.appenderRef.console.ref=STDOUT

    logger.requestlogger.additivity=false

    logger.requestchannel.name=kafka.network.RequestChannel$

    logger.requestchannel.level=WARN

    logger.requestchannel.appenderRefs=console

    logger.requestchannel.appenderRef.console.ref=STDOUT

    logger.requestchannel.additivity=false

    logger.controller.name=org.apache.kafka.controller

    logger.controller.level=INFO

    logger.controller.appenderRefs=console

    logger.controller.appenderRef.console.ref=STDOUT

    logger.controller.additivity=false

    logger.logcleaner.name=kafka.log.LogCleaner

    logger.logcleaner.level=INFO

    logger.logcleaner.appenderRefs=console

    logger.logcleaner.appenderRef.console.ref=STDOUT

    logger.logcleaner.additivity=false

    logger.statechange.name=state.change.logger

    logger.statechange.level=INFO

    logger.statechange.appenderRefs=console

    logger.statechange.appenderRef.console.ref=STDOUT

    logger.statechange.additivity=false

    logger.authorizer.name=kafka.authorizer.logger

    logger.authorizer.level=INFO

    logger.authorizer.appenderRefs=console

    logger.authorizer.appenderRef.console.ref=STDOUT

    logger.authorizer.additivity=false


    logger.security.name=org.apache.kafka.common.security

    logger.security.level=INFO

    logger.security.appenderRefs=securityAppender

    logger.security.appenderRef.securityAppender.ref=SECURITY

    logger.security.additivity=false


    logger.networkjava.name=org.apache.kafka.common.network

    logger.networkjava.level=INFO

    logger.networkjava.appenderRefs=securityAppender

    logger.networkjava.appenderRef.securityAppender.ref=SECURITY

    logger.networkjava.additivity=false


    logger.networkscala.name=kafka.network

    logger.networkscala.level=INFO

    logger.networkscala.appenderRefs=securityAppender

    logger.networkscala.appenderRef.securityAppender.ref=SECURITY

    logger.networkscala.additivity=false


    logger.auth.name=org.apache.kafka.server.authorizer

    logger.auth.level=INFO

    logger.auth.appenderRefs=securityAppender

    logger.auth.appenderRef.securityAppender.ref=SECURITY

    logger.auth.additivity=false


    logger.topiceventhandler.name=io.strimzi.operator.topic

    logger.topiceventhandler.level=WARN

    logger.topiceventhandler.appenderRefs=console

    logger.topiceventhandler.appenderRef.console.ref=STDOUT

    logger.topiceventhandler.additivity=false


    monitorInterval=30
  metadata.version: '4.1'
  metrics-config.json: >-
    {"lowercaseOutputName":true,"rules":[{"pattern":"kafka.server<type=(.+),
    name=(.+), clientId=(.+), topic=(.+),
    partition=(.*)><>Value","name":"kafka_server_$1_$2","type":"GAUGE","labels":{"clientId":"$3","topic":"$4","partition":"$5"}},{"pattern":"kafka.server<type=(.+),
    name=(.+), clientId=(.+), brokerHost=(.+),
    brokerPort=(.+)><>Value","name":"kafka_server_$1_$2","type":"GAUGE","labels":{"clientId":"$3","broker":"$4:$5"}},{"pattern":"kafka.server<type=(.+),
    cipher=(.+), protocol=(.+), listener=(.+),
    networkProcessor=(.+)><>connections","name":"kafka_server_$1_connections_tls_info","type":"GAUGE","labels":{"cipher":"$2","protocol":"$3","listener":"$4","networkProcessor":"$5"}},{"pattern":"kafka.server<type=(.+),
    clientSoftwareName=(.+), clientSoftwareVersion=(.+), listener=(.+),
    networkProcessor=(.+)><>connections","name":"kafka_server_$1_connections_software","type":"GAUGE","labels":{"clientSoftwareName":"$2","clientSoftwareVersion":"$3","listener":"$4","networkProcessor":"$5"}},{"pattern":"kafka.server<type=(.+),
    listener=(.+),
    networkProcessor=(.+)><>(.+):","name":"kafka_server_$1_$4","type":"GAUGE","labels":{"listener":"$2","networkProcessor":"$3"}},{"pattern":"kafka.server<type=(.+),
    listener=(.+),
    networkProcessor=(.+)><>(.+)","name":"kafka_server_$1_$4","type":"GAUGE","labels":{"listener":"$2","networkProcessor":"$3"}},{"pattern":"kafka.(\\w+)<type=(.+),
    name=(.+)Percent\\w*><>MeanRate","name":"kafka_$1_$2_$3_percent","type":"GAUGE"},{"pattern":"kafka.(\\w+)<type=(.+),
    name=(.+)Percent\\w*><>Value","name":"kafka_$1_$2_$3_percent","type":"GAUGE"},{"pattern":"kafka.(\\w+)<type=(.+),
    name=(.+)Percent\\w*,
    (.+)=(.+)><>Value","name":"kafka_$1_$2_$3_percent","type":"GAUGE","labels":{"$4":"$5"}},{"pattern":"kafka.(\\w+)<type=(.+),
    name=(.+)PerSec\\w*, (.+)=(.+),
    (.+)=(.+)><>Count","name":"kafka_$1_$2_$3_total","type":"COUNTER","labels":{"$4":"$5","$6":"$7"}},{"pattern":"kafka.(\\w+)<type=(.+),
    name=(.+)PerSec\\w*,
    (.+)=(.+)><>Count","name":"kafka_$1_$2_$3_total","type":"COUNTER","labels":{"$4":"$5"}},{"pattern":"kafka.(\\w+)<type=(.+),
    name=(.+)PerSec\\w*><>Count","name":"kafka_$1_$2_$3_total","type":"COUNTER"},{"pattern":"kafka.(\\w+)<type=(.+),
    name=(.+), (.+)=(.+),
    (.+)=(.+)><>Value","name":"kafka_$1_$2_$3","type":"GAUGE","labels":{"$4":"$5","$6":"$7"}},{"pattern":"kafka.(\\w+)<type=(.+),
    name=(.+),
    (.+)=(.+)><>Value","name":"kafka_$1_$2_$3","type":"GAUGE","labels":{"$4":"$5"}},{"pattern":"kafka.(\\w+)<type=(.+),
    name=(.+)><>Value","name":"kafka_$1_$2_$3","type":"GAUGE"},{"pattern":"kafka.(\\w+)<type=(.+),
    name=(.+), (.+)=(.+),
    (.+)=(.+)><>Count","name":"kafka_$1_$2_$3_count","type":"COUNTER","labels":{"$4":"$5","$6":"$7"}},{"pattern":"kafka.(\\w+)<type=(.+),
    name=(.+), (.+)=(.*),
    (.+)=(.+)><>(\\d+)thPercentile","name":"kafka_$1_$2_$3","type":"GAUGE","labels":{"$4":"$5","$6":"$7","quantile":"0.$8"}},{"pattern":"kafka.(\\w+)<type=(.+),
    name=(.+),
    (.+)=(.+)><>Count","name":"kafka_$1_$2_$3_count","type":"COUNTER","labels":{"$4":"$5"}},{"pattern":"kafka.(\\w+)<type=(.+),
    name=(.+),
    (.+)=(.*)><>(\\d+)thPercentile","name":"kafka_$1_$2_$3","type":"GAUGE","labels":{"$4":"$5","quantile":"0.$6"}},{"pattern":"kafka.(\\w+)<type=(.+),
    name=(.+)><>Count","name":"kafka_$1_$2_$3_count","type":"COUNTER"},{"pattern":"kafka.(\\w+)<type=(.+),
    name=(.+)><>(\\d+)thPercentile","name":"kafka_$1_$2_$3","type":"GAUGE","labels":{"quantile":"0.$4"}}]}
  server.config: >-
    ##############################

    ##############################

    # This file is automatically generated by the Strimzi Cluster Operator

    # Any changes to this file will be ignored and overwritten!

    ##############################

    ##############################


    ##########

    # Node ID

    ##########

    node.id=0


    ##########

    # KRaft configuration

    ##########

    process.roles=controller

    controller.listener.names=CONTROLPLANE-9090

    controller.quorum.voters=0@kafka-controller-kafka-0.kafka-kafka-brokers.kafka.svc.cluster.local:9090


    ##########

    # KRaft metadata log dir configuration

    ##########

    metadata.log.dir=/var/lib/kafka/data-1/kafka-log0


    ##########

    # Kafka message logs configuration

    ##########

    log.dirs=/var/lib/kafka/data-1/kafka-log0


    ##########

    # Control Plane listener

    ##########

    listener.name.controlplane-9090.ssl.keystore.certificate.chain=${strimzisecrets:kafka/kafka-controller-kafka-0:kafka-controller-kafka-0.crt}

    listener.name.controlplane-9090.ssl.keystore.key=${strimzisecrets:kafka/kafka-controller-kafka-0:kafka-controller-kafka-0.key}

    listener.name.controlplane-9090.ssl.keystore.type=PEM

    listener.name.controlplane-9090.ssl.truststore.certificates=${strimzisecrets:kafka/kafka-cluster-ca-cert:*.crt}

    listener.name.controlplane-9090.ssl.truststore.type=PEM

    listener.name.controlplane-9090.ssl.client.auth=required


    ##########

    # Common listener configuration

    ##########

    listener.security.protocol.map=CONTROLPLANE-9090:SSL

    listeners=CONTROLPLANE-9090://0.0.0.0:9090

    advertised.listeners=CONTROLPLANE-9090://kafka-controller-kafka-0.kafka-kafka-brokers.kafka.svc:9090

    sasl.enabled.mechanisms=

    ssl.endpoint.identification.algorithm=HTTPS


    ##########

    # Config providers

    ##########

    # Configuration providers configured by the user and by Strimzi

    config.providers=strimzienv,strimzisecrets

    config.providers.strimzienv.class=org.apache.kafka.common.config.provider.EnvVarConfigProvider

    config.providers.strimzienv.param.allowlist.pattern=.*

    config.providers.strimzisecrets.class=io.strimzi.kafka.KubernetesSecretConfigProvider


    ##########

    # metric.reporters configuration

    ##########

    # metric.reporters configured by Strimzi

    metric.reporters=org.apache.kafka.common.metrics.JmxReporter


    ##########

    # User provided configuration

    ##########

    min.insync.replicas=1

    auto.create.topics.enable=false

    compression.type=snappy

    default.replication.factor=1

    log.segment.bytes=104857600

    num.io.threads=25

    num.network.threads=10

    num.partitions=1

    offsets.topic.replication.factor=1

    principal.builder.class=io.strimzi.kafka.oauth.server.OAuthKafkaPrincipalBuilder

    transaction.state.log.min.isr=1

    transaction.state.log.replication.factor=1
kind: ConfigMap
metadata:
  labels:
    app.kubernetes.io/instance: kafka
    app.kubernetes.io/managed-by: strimzi-cluster-operator
    app.kubernetes.io/name: kafka
    app.kubernetes.io/part-of: strimzi-kafka
    strimzi.io/cluster: kafka
    strimzi.io/component-type: kafka
    strimzi.io/kind: Kafka
    strimzi.io/name: kafka-kafka
    strimzi.io/pod-name: kafka-controller-kafka-0
    strimzi.io/pool-name: controller-kafka
  name: kafka-controller-kafka-0
  namespace: kafka
  ownerReferences:
    - apiVersion: kafka.strimzi.io/v1beta2
      blockOwnerDeletion: true
      controller: false
      kind: KafkaNodePool
      name: controller-kafka

scholzj Jan 29, 2026
Maintainer

@mstruk any idea what yould this mean Marko?

mstruk · 2026-01-29T14:20:59Z

mstruk
Jan 29, 2026
Collaborator

It looks like the listener configuration needs the following config parameter:

unsecuredLoginStringClaim_sub="unused"

It's something the operator used to automatically add, but now has to be added manually.

1 reply

user222635 Jan 29, 2026
Author

This config parameter worked and it fixed the problem.

I really appreciate your help, thank you very much both of you!

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Strimzi

Upgrade v1beta2 to v1 - Kafka Listener Authentication #12361

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{editor}}'s edit

{{editor}}'s edit

Uh oh!

Replies: 2 comments 3 replies

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{editor}}'s edit

{{editor}}'s edit

Uh oh!

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{editor}}'s edit

{{editor}}'s edit

Uh oh!

Select a reply

Uh oh!

Strimzi

Upgrade v1beta2 to v1 - Kafka Listener Authentication #12361

Uh oh!

Uh oh!

user222635 Jan 28, 2026

Replies: 2 comments · 3 replies

Uh oh!

scholzj Jan 28, 2026 Maintainer

Uh oh!

Uh oh!

user222635 Jan 29, 2026 Author

Uh oh!

scholzj Jan 29, 2026 Maintainer

Uh oh!

mstruk Jan 29, 2026 Collaborator

Uh oh!

Uh oh!

user222635 Jan 29, 2026 Author

user222635
Jan 28, 2026

Replies: 2 comments 3 replies

scholzj
Jan 28, 2026
Maintainer

user222635 Jan 29, 2026
Author

scholzj Jan 29, 2026
Maintainer

mstruk
Jan 29, 2026
Collaborator

user222635 Jan 29, 2026
Author