KafkaMirror2 between two clusters is throwing error

[phanindra-rd]

i have setup two clusters with mirrormaker2 to replicate topic from source to destination. using external traffic with authentication scram-sha-512

- name: external port: 9094 type: ingress tls: true authentication: type: scram-sha-512

I receiving error logs in mirror pods as below:

2025-12-16 16:18:25 INFO [main] AbstractLogin:62 - Successfully logged in. 2025-12-16 16:18:25 INFO [main[] AbstractConfig:380 - These configurations '[producer.sasl.jaas.config, group.id, rest.advertised.port, plugin.path, consumer.sasl.mechanism, admin.security.protocol, config.providers.strimzifile.class, offset.lag.max, config.providers.strimzienv.param.allowlist.pattern, config.providers.strimzisecrets.class, status.storage.replication.factor, config.providers.strimzidir.class, consumer.ssl.truststore.certificates, offset.storage.topic, admin.sasl.mechanism, consumer.security.protocol, value.converter, admin.ssl.truststore.type, key.converter, consumer.client.rack, producer.ssl.truststore.certificates, consumer.sasl.jaas.config, config.storage.topic, producer.security.protocol, rest.advertised.host.name, status.storage.topic, consumer.ssl.truststore.type, header.converter, producer.sasl.mechanism, config.storage.replication.factor, producer.ssl.truststore.type, offset.storage.replication.factor, config.providers.strimzidir.param.allowed.paths, config.providers.strimzienv.class, admin.sasl.jaas.config, admin.ssl.truststore.certificates]' were supplied but are not used yet. 2025-12-16 16:18:25 INFO [main] AppInfoParser:125 - Kafka version: 4.0.0 2025-12-16 16:18:25 INFO [main] AppInfoParser:126 - Kafka commitId: 985bc99521dd22bb 2025-12-16 16:18:25 INFO [main] AppInfoParser:127 - Kafka startTimeMs: 1765901905276 2025-12-16 16:18:25 INFO [kafka-admin-client-thread | adminclient-1] Selector:621 - [AdminClient clientId=adminclient-1] Failed authentication with kafka-market-scheduling.dr.gcp.devops.com/10.39.184.31 (channelId=-1) (SSL handshake failed) 2025-12-16 16:18:25 INFO [kafka-admin-client-thread | adminclient-1] NetworkClient:1072 - [AdminClient clientId=adminclient-1] Node -1 disconnected. 2025-12-16 16:18:25 ERROR [kafka-admin-client-thread | adminclient-1] NetworkClient:889 - [AdminClient clientId=adminclient-1] Connection to node -1 (kafka-market-scheduling.dr.gcp.devops.com/10.39.184.31:443) failed authentication due to: SSL handshake failed 2025-12-16 16:18:25 WARN [kafka-admin-client-thread | adminclient-1] AdminMetadataManager:281 - [AdminClient clientId=adminclient-1] Metadata update failed due to authentication error org.apache.kafka.common.errors.SslAuthenticationException: SSL handshake failed Caused by: javax.net.ssl.SSLException: Received fatal alert: internal_error at java.base/sun.security.ssl.Alert.createSSLException(Alert.java:133) ~[?:?] at java.base/sun.security.ssl.Alert.createSSLException(Alert.java:117) ~[?:?] at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:370) ~[?:?] at java.base/sun.security.ssl.Alert$AlertConsumer.consume(Alert.java:293) ~[?:?] at java.base/sun.security.ssl.TransportContext.dispatch(TransportContext.java:209) ~[?:?] at java.base/sun.security.ssl.SSLTransport.decode(SSLTransport.java:172) ~[?:?] at java.base/sun.security.ssl.SSLEngineImpl.decode(SSLEngineImpl.java:736) ~[?:?] at java.base/sun.security.ssl.SSLEngineImpl.readRecord(SSLEngineImpl.java:691) ~[?:?] at java.base/sun.security.ssl.SSLEngineImpl.unwrap(SSLEngineImpl.java:506) ~[?:?] at java.base/sun.security.ssl.SSLEngineImpl.unwrap(SSLEngineImpl.java:482) ~[?:?] at java.base/javax.net.ssl.SSLEngine.unwrap(SSLEngine.java:679) ~[?:?] at org.apache.kafka.common.network.SslTransportLayer.handshakeUnwrap(SslTransportLayer.java:528) ~[kafka-clients-4.0.0.jar:?] at org.apache.kafka.common.network.SslTransportLayer.doHandshake(SslTransportLayer.java:382) ~[kafka-clients-4.0.0.jar:?] at org.apache.kafka.common.network.SslTransportLayer.handshake(SslTransportLayer.java:302) ~[kafka-clients-4.0.0.jar:?] at org.apache.kafka.common.network.KafkaChannel.prepare(KafkaChannel.java:178) ~[kafka-clients-4.0.0.jar:?] at org.apache.kafka.common.network.Selector.pollSelectionKeys(Selector.java:548) ~[kafka-clients-4.0.0.jar:?] at org.apache.kafka.common.network.Selector.poll(Selector.java:486) ~[kafka-clients-4.0.0.jar:?] at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:645) [kafka-clients-4.0.0.jar:?] at org.apache.kafka.clients.admin.KafkaAdminClient$AdminClientRunnable.processRequests(KafkaAdminClient.java:1560) [kafka-clients-4.0.0.jar:?] at org.apache.kafka.clients.admin.KafkaAdminClient$AdminClientRunnable.run(KafkaAdminClient.java:1491) [kafka-clients-4.0.0.jar:?] at java.base/java.lang.Thread.run(Thread.java:840) [?:?] 2025-12-16 16:18:25 INFO [main] Metrics:684 - Metrics scheduler closed 2025-12-16 16:18:25 INFO [main] Metrics:688 - Closing reporter org.apache.kafka.common.metrics.JmxReporter 2025-12-16 16:18:25 INFO [main] Metrics:694 - Metrics reporters closed 2025-12-16 16:18:25 INFO [kafka-admin-client-thread | adminclient-1] AdminMetadataManager:332 - [AdminClient clientId=adminclient-1] Rebootstrapping with Cluster(id = null, nodes = [kafka-market-scheduling.dr.gcp.devops.com:443 (id: -1 rack: null isFenced: false)], partitions = ], controller = null) 2025-12-16 16:18:25 INFO [kafka-admin-client-thread | adminclient-1] AdminMetadataManager:332 - [AdminClient clientId=adminclient-1] Rebootstrapping with Cluster(id = null, nodes = [kafka-market-scheduling.dr.gcp.devops.com:443 (id: -1 rack: null isFenced: false)], partitions = ], controller = null) 2025-12-16 16:18:25 INFO [kafka-admin-client-thread | adminclient-1] AppInfoParser:89 - App info kafka.admin.client for adminclient-1 unregistered 2025-12-16 16:18:25 INFO [kafka-admin-client-thread | adminclient-1] AdminMetadataManager:298 - [AdminClient clientId=adminclient-1] Metadata update failed org.apache.kafka.common.errors.TimeoutException: The AdminClient thread has exited. Call: fetchMetadata 2025-12-16 16:18:25 INFO [kafka-admin-client-thread | adminclient-1] AdminMetadataManager:298 - [AdminClient clientId=adminclient-1] Metadata update failed org.apache.kafka.common.errors.TimeoutException: The AdminClient thread has exited. Call: fetchMetadata 2025-12-16 16:18:25 INFO [kafka-admin-client-thread | adminclient-1] KafkaAdminClient:1506 - [AdminClient clientId=adminclient-1] Timed out 2 remaining operation(s) during close. 2025-12-16 16:18:25 INFO [kafka-admin-client-thread | adminclient-1] Metrics:684 - Metrics scheduler closed 2025-12-16 16:18:25 INFO [kafka-admin-client-thread | adminclient-1] Metrics:688 - Closing reporter org.apache.kafka.common.metrics.JmxReporter 2025-12-16 16:18:25 INFO [kafka-admin-client-thread | adminclient-1] Metrics:694 - Metrics reporters closed 2025-12-16 16:18:25 ERROR [main] AbstractConnectCli:102 - Stopping due to error org.apache.kafka.connect.errors.ConnectException: Failed to connect to and describe Kafka cluster. Check worker's broker connection and security properties. at org.apache.kafka.connect.runtime.WorkerConfig.lookupKafkaClusterId(WorkerConfig.java:296) ~[connect-runtime-4.0.0.jar:?] at org.apache.kafka.connect.runtime.WorkerConfig.lookupKafkaClusterId(WorkerConfig.java:276) ~[connect-runtime-4.0.0.jar:?] at org.apache.kafka.connect.runtime.WorkerConfig.kafkaClusterId(WorkerConfig.java:406) ~[connect-runtime-4.0.0.jar:?] at org.apache.kafka.connect.cli.AbstractConnectCli.startConnect(AbstractConnectCli.java:124) ~[connect-runtime-4.0.0.jar:?] at org.apache.kafka.connect.cli.AbstractConnectCli.run(AbstractConnectCli.java:95) [connect-runtime-4.0.0.jar:?] at org.apache.kafka.connect.cli.ConnectDistributed.main(ConnectDistributed.java:112) [connect-runtime-4.0.0.jar:?] Caused by: java.util.concurrent.ExecutionException: org.apache.kafka.common.errors.SslAuthenticationException: SSL handshake failed at java.base/java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:396) ~[?:?] at java.base/java.util.concurrent.CompletableFuture.get(CompletableFuture.java:2073) ~[?:?] at org.apache.kafka.common.internals.KafkaFutureImpl.get(KafkaFutureImpl.java:155) ~[kafka-clients-4.0.0.jar:?] at org.apache.kafka.connect.runtime.WorkerConfig.lookupKafkaClusterId(WorkerConfig.java:290) ~[connect-runtime-4.0.0.jar:?] ... 5 more Caused by: org.apache.kafka.common.errors.SslAuthenticationException: SSL handshake failed Caused by: javax.net.ssl.SSLException: Received fatal alert: internal_error at java.base/sun.security.ssl.Alert.createSSLException(Alert.java:133) ~[?:?] at java.base/sun.security.ssl.Alert.createSSLException(Alert.java:117) ~[?:?] at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:370) ~[?:?] at java.base/sun.security.ssl.Alert$AlertConsumer.consume(Alert.java:293) ~[?:?] at java.base/sun.security.ssl.TransportContext.dispatch(TransportContext.java:209) ~[?:?] at java.base/sun.security.ssl.SSLTransport.decode(SSLTransport.java:172) ~[?:?] at java.base/sun.security.ssl.SSLEngineImpl.decode(SSLEngineImpl.java:736) ~[?:?] at java.base/sun.security.ssl.SSLEngineImpl.readRecord(SSLEngineImpl.java:691) ~[?:?] at java.base/sun.security.ssl.SSLEngineImpl.unwrap(SSLEngineImpl.java:506) ~[?:?] at java.base/sun.security.ssl.SSLEngineImpl.unwrap(SSLEngineImpl.java:482) ~[?:?] at java.base/javax.net.ssl.SSLEngine.unwrap(SSLEngine.java:679) ~[?:?] at org.apache.kafka.common.network.SslTransportLayer.handshakeUnwrap(SslTransportLayer.java:528) ~[kafka-clients-4.0.0.jar:?] at org.apache.kafka.common.network.SslTransportLayer.doHandshake(SslTransportLayer.java:382) ~[kafka-clients-4.0.0.jar:?] at org.apache.kafka.common.network.SslTransportLayer.handshake(SslTransportLayer.java:302) ~[kafka-clients-4.0.0.jar:?] at org.apache.kafka.common.network.KafkaChannel.prepare(KafkaChannel.java:178) ~[kafka-clients-4.0.0.jar:?] at org.apache.kafka.common.network.Selector.pollSelectionKeys(Selector.java:548) ~[kafka-clients-4.0.0.jar:?] at org.apache.kafka.common.network.Selector.poll(Selector.java:486) ~[kafka-clients-4.0.0.jar:?] at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:645) ~[kafka-clients-4.0.0.jar:?] at org.apache.kafka.clients.admin.KafkaAdminClient$AdminClientRunnable.processRequests(KafkaAdminClient.java:1560) ~[kafka-clients-4.0.0.jar:?] at org.apache.kafka.clients.admin.KafkaAdminClient$AdminClientRunnable.run(KafkaAdminClient.java:1491) ~[kafka-clients-4.0.0.jar:?] at java.base/java.lang.Thread.run(Thread.java:840) ~[?:?] stream closed: EOF for gigant-devops/market-scheduling-mirrormaker-mirrormaker2-0 (market-scheduling-mirrormaker-mirrormaker2)

[scholzj]

• Please make sure to properly fotmat everything because it is otherwise completely unreadable
• Make it clear what your problem is / what your question is
• Open issues only for things that you really know are issues and not for questions or configuration issues (as you are asked when opening the issues!)