Broker fails to start MDS server with LDAP in CP 8.0.0 RBAC #491
Description
Start Confluent Platform version 8.0.0 with RBAC enabled using the configuration below:
authorizer.class.name=io.confluent.kafka.security.authorizer.ConfluentServerAuthorizer confluent.authorizer.access.rule.providers=CONFLUENT super.users=User:admin
The broker always fails to start the MDS server when using the LDAP identity provider.
Trace the error in the metadata-service.log file:
[2025-09-10 14:37:19,454] INFO [io.confluent.security.store.kafka.coordinator.MetadataNodeManager clientId=_confluent-metadata-coordinator-0, groupId=_confluent-metadata-coordinator-group](Re-)joining group (io.confluent.security.store.kafka.coordinator.MetadataServiceCoordinator) [2025-09-10 14:37:19,455] INFO [io.confluent.security.store.kafka.coordinator.MetadataNodeManager clientId=_confluent-metadata-coordinator-0, groupId=_confluent-metadata-coordinator-group]Successfully joined group with generation Generation{generationId=144523, memberId='_confluent-metadata-coordinator-0-6a3d7390-ad8a-4423-9eff-7290d4b8f6e2', protocol='v0'} (io.confluent.security.store.kafka.coordinator.MetadataServiceCoordinator) [2025-09-10 14:37:19,455] INFO [io.confluent.security.store.kafka.coordinator.MetadataNodeManager clientId=_confluent-metadata-coordinator-0, groupId=_confluent-metadata-coordinator-group]Request joining group due to: rebalance failed due to 'Unexpected error from SyncGroup: The server experienced an unexpected error when processing the request.' (KafkaException) (io.confluent.security.store.kafka.coordinator.MetadataServiceCoordinator) [2025-09-10 14:37:19,455] ERROR [io.confluent.security.store.kafka.coordinator.MetadataNodeManager clientId=_confluent-metadata-coordinator-0, groupId=_confluent-metadata-coordinator-group]Metadata service coordinator encountered exception during poll (io.confluent.security.store.kafka.coordinator.MetadataNodeManager) org.apache.kafka.common.KafkaException: Unexpected error from SyncGroup: The server experienced an unexpected error when processing the request. at org.apache.kafka.clients.consumer.internals.AbstractCoordinator$SyncGroupResponseHandler.handle(AbstractCoordinator.java:893) ~[clients_clients-export-project.jar:?] at org.apache.kafka.clients.consumer.internals.AbstractCoordinator$SyncGroupResponseHandler.handle(AbstractCoordinator.java:812) ~[clients_clients-export-project.jar:?] at org.apache.kafka.clients.consumer.internals.AbstractCoordinator$CoordinatorResponseHandler.onSuccess(AbstractCoordinator.java:1311) ~[clients_clients-export-project.jar:?] at org.apache.kafka.clients.consumer.internals.AbstractCoordinator$CoordinatorResponseHandler.onSuccess(AbstractCoordinator.java:1286) ~[clients_clients-export-project.jar:?] at org.apache.kafka.clients.consumer.internals.RequestFuture$1.onSuccess(RequestFuture.java:206) ~[clients_clients-export-project.jar:?] at org.apache.kafka.clients.consumer.internals.RequestFuture.fireSuccess(RequestFuture.java:169) ~[clients_clients-export-project.jar:?] at org.apache.kafka.clients.consumer.internals.RequestFuture.complete(RequestFuture.java:129) ~[clients_clients-export-project.jar:?] at org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient$RequestFutureCompletionHandler.fireCompletion(ConsumerNetworkClient.java:617) ~[clients_clients-export-project.jar:?] at org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.firePendingCompletedRequests(ConsumerNetworkClient.java:429) ~[clients_clients-export-project.jar:?] at org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:314) ~[clients_clients-export-project.jar:?] at org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:232) ~[clients_clients-export-project.jar:?] at org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:216) ~[clients_clients-export-project.jar:?] at org.apache.kafka.clients.consumer.internals.AbstractCoordinator.joinGroupIfNeeded(AbstractCoordinator.java:481) ~[clients_clients-export-project.jar:?] at org.apache.kafka.clients.consumer.internals.AbstractCoordinator.ensureActiveGroup(AbstractCoordinator.java:415) ~[clients_clients-export-project.jar:?] at org.apache.kafka.clients.consumer.internals.AbstractCoordinator.ensureActiveGroup(AbstractCoordinator.java:395) ~[clients_clients-export-project.jar:?] at io.confluent.security.store.kafka.coordinator.MetadataServiceCoordinator.poll(MetadataServiceCoordinator.java:157) ~[ce-auth-providers_ce-auth-providers-project.jar:?] at io.confluent.security.store.kafka.coordinator.MetadataNodeManager$MetadataNodeManagerThread.run(MetadataNodeManager.java:361) [ce-auth-providers_ce-auth-providers-project.jar:?]
Trace the error in the server.log file:
[2025-09-10 14:35:29,509] INFO [Producer clientId=_confluent-metadata-auth-producer-0] Cluster ID: 7nSt8kn1R2ClWOQRxqZqXg (org.apache.kafka.clients.Metadata) [2025-09-10 14:35:29,525] INFO [GroupCoordinator id=0 topic=__consumer_offsets partition=44] Assignment received from leader _confluent-metadata-coordinator-0-6a3d7390-ad8a-4423-9eff-7290d4b8f6e2 for group _confluent-metadata-coordinator-group for generation 1. The group has 1 members, 0 of which are static. (org.apache.kafka.coordinator.group.GroupMetadataManager) [2025-09-10 14:35:29,526] INFO [GroupCoordinator id=0 topic=__consumer_offsets partition=44] Preparing to rebalance group _confluent-metadata-coordinator-group in state PreparingRebalance with old generation 1 (reason: Error UNKNOWN_SERVER_ERROR when storing group assignmentduring SyncGroup (member: _confluent-metadata-coordinator-0-6a3d7390-ad8a-4423-9eff-7290d4b8f6e2).). (org.apache.kafka.coordinator.group.GroupMetadataManager) [2025-09-10 14:35:29,534] INFO [GroupCoordinator id=0 topic=__consumer_offsets partition=44] Stabilized group _confluent-metadata-coordinator-group generation 2 with 1 members. (org.apache.kafka.coordinator.group.GroupMetadataManager)