Skip to content

Migrate to Kraft: Broker fails to start with wrong authorizer.class.name setting #1805

Open
@42tg

Description

@42tg

Describe the issue
Today I executed an Migration from Zookeeper to Kraft with the cp-ansible script as described here.

Until the Cluster runs in Dual Write mode everything went well. But while the executing the Kraft Mode Step

ansible-playbook -i <inventory-file> confluent.platform.ZKtoKraftMigration.yml \
  --tags migrate_to_kraft

The restarted Broker shuts down and does not came back online and produced the following Error in the server.log:

[2024-10-21 13:45:29,211] ERROR Exiting Kafka due to fatal exception during startup. (kafka.Kafka$)
java.lang.NullPointerException
        at kafka.zk.KafkaZkClient$.apply(KafkaZkClient.scala:2300)
        at kafka.security.authorizer.AclAuthorizer.configure(AclAuthorizer.scala:211)
        at kafka.server.BrokerServer.$anonfun$startup$13(BrokerServer.scala:390)
        at kafka.server.BrokerServer.$anonfun$startup$13$adapted(BrokerServer.scala:390)
        at scala.Option.foreach(Option.scala:437)
        at kafka.server.BrokerServer.startup(BrokerServer.scala:390)
        at kafka.server.KafkaRaftServer.$anonfun$startup$2(KafkaRaftServer.scala:99)
        at kafka.server.KafkaRaftServer.$anonfun$startup$2$adapted(KafkaRaftServer.scala:99)
        at scala.Option.foreach(Option.scala:437)
        at kafka.server.KafkaRaftServer.startup(KafkaRaftServer.scala:99)
        at kafka.Kafka$.main(Kafka.scala:112)
        at kafka.Kafka.main(Kafka.scala)

The Manual Migration Guide pointed us to the right Direction:

If you are using ACLs, change the authorizer class. For more information, see ACL concepts.

wich describes the problem in more Detail and also the Solution for it:

So we have to change the authorizer.class.name accordingly from kafka.security.authorizer.AclAuthorizer to org.apache.kafka.metadata.authorizer.StandardAuthorizer.

However, changing those properties is not applied by the migration script itself, which causes us some headaches since the timing is crucial here. If the property is not set in the same run where the zookeeper and inter.broker.protocol.version are removed, the broker will not be able to recover properly, and the health_check.yml task from kafka_broker will not complete, as the Get Topics with UnderReplicatedPartitions step will fail.

To Reproduce
Steps to reproduce the behaviour:

  • Setup an Zookeeper Kafka Cluster without RBAC as default but with ACL's in use
  • Follow the Migration Guide

Expected behaviour
While running the Migration, changes for the Broker Properties should be applied also to the Broker Configurations as an normal Deployment do.

Inventory File
n/a

Logs
n/a

Environment (please complete the following information):

  • OS: Rocky Linux 8.10
  • CP-Ansible Branch: 7.7.1-post
  • Ansible Version: ansible core 2.16.10

Additional context
n/a

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions