Fetch Kafka Connect, MM2, Bridge trusted certificates Secret only once for each reference #12241

venkatesh2090 · 2025-12-13T00:56:33Z

Type of change

Select the type of your PR

Enhancement / new feature

Description

Fixes #11972

Updated when Secrets for trusted certificates are fetched and used in KafkaConnect, KafkaMirrorMaker2, and KafkaBridge assembly operators. It was fetched twice - once when the combined CA cert secret is generated, and again when the hash is calculated with certs and auth.

Refactored the reconcile loop such that it is only fetched once per reference.

KafkaConnect
It is fetched in tlsTrustedCertsSecret and returned with the method, which is passed into the generateAuthHash method.

KafkaBridge
A similar pattern is used here - but since a new secret isn't generated, it is fetched directly in the loop and passed to the hash method.

KafkaMirrorMaker2
Since there are multiple certs for each mirror's target and source, all the certs are fetched and stored in a HashMap. It relies on the fact that each cluster in MM2 has a unique alias. It is then passed to both secret generation and auth hash methods when needed.

I have also removed 2 tests in ReconcilerUtilsTest which checked if the authTlsHash method fetched the secrets correctly. That method doesn't fetch trusted cert secrets anymore. It expects certs to be passed in as parameter.

Disclosure: The operator tests were generated using AI and modified. I have reviewed it thoroughly before committing.

Checklist

Please go through this checklist and make sure all applicable tasks have been done

Write tests
Make sure all tests pass
Update documentation
Check RBAC rights for Kubernetes / OpenShift roles
Try your changes from Pod inside your Kubernetes and OpenShift cluster, not just locally
Reference relevant issue(s) and close them after merging
Update CHANGELOG.md
Supply screenshots for visual changes, such as Grafana dashboards

Refactored tlsTrustedCertsSecret to return a list of secrets instead of the concatenated version. Signed-off-by: Venkatesh Kannan <[email protected]>

Also fixed auth hash in MM2. Signed-off-by: Venkatesh Kannan <[email protected]>

Signed-off-by: Venkatesh Kannan <[email protected]>

codecov · 2025-12-13T02:38:15Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 74.89%. Comparing base (cdaa665) to head (c615a6a).
⚠️ Report is 15 commits behind head on main.

Additional details and impacted files

@@             Coverage Diff              @@
##               main   #12241      +/-   ##
============================================
+ Coverage     74.76%   74.89%   +0.12%     
- Complexity     6626     6643      +17     
============================================
  Files           377      376       -1     
  Lines         25362    25361       -1     
  Branches       3402     3401       -1     
============================================
+ Hits          18962    18994      +32     
+ Misses         5011     4983      -28     
+ Partials       1389     1384       -5

Files with missing lines	Coverage Δ
...ter/operator/assembly/AbstractConnectOperator.java	`87.81% <100.00%> (+2.45%)`	⬆️
...operator/assembly/KafkaBridgeAssemblyOperator.java	`88.70% <100.00%> (+1.82%)`	⬆️
...perator/assembly/KafkaConnectAssemblyOperator.java	`88.28% <100.00%> (-0.07%)`	⬇️
...or/assembly/KafkaMirrorMaker2AssemblyOperator.java	`86.56% <100.00%> (+0.47%)`	⬆️
...tor/cluster/operator/assembly/ReconcilerUtils.java	`80.97% <100.00%> (+3.14%)`	⬆️

... and 6 files with indirect coverage changes

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Signed-off-by: Venkatesh Kannan <[email protected]>

scholzj · 2025-12-15T09:21:13Z

/gha run pipeline=upgrade,regression

github-actions · 2025-12-15T09:21:43Z

⏳ System test verification started: link

The following 10 job(s) will be executed:

regression-brokers-and-security-amd64 (oracle-vm-8cpu-32gb-x86-64)
regression-operators-amd64 (oracle-vm-8cpu-32gb-x86-64)
regression-operands-amd64 (oracle-vm-8cpu-32gb-x86-64)
regression-brokers-and-security-arm64 (oracle-vm-8cpu-32gb-arm64)
regression-operators-arm64 (oracle-vm-8cpu-32gb-arm64)
regression-operands-arm64 (oracle-vm-8cpu-32gb-arm64)
upgrade-azp_kraft_upgrade-amd64 (oracle-vm-4cpu-16gb-x86-64)
upgrade-azp_kafka_upgrade-amd64 (oracle-vm-4cpu-16gb-x86-64)
upgrade-azp_kraft_upgrade-arm64 (oracle-vm-4cpu-16gb-arm64)
upgrade-azp_kafka_upgrade-arm64 (oracle-vm-4cpu-16gb-arm64)

Tests will start after successful build completion.

tinaselenge

Thanks for the PR. I left some comments. I think we need to leave KafkaBridge related changes out of this PR as mentioned below.

tinaselenge · 2025-12-15T11:22:42Z

...tor/src/main/java/io/strimzi/operator/cluster/operator/assembly/AbstractConnectOperator.java

     * @return  Future which completes when the reconciliation is done
     */
-    protected Future<Void> tlsTrustedCertsSecret(Reconciliation reconciliation, String namespace, KafkaConnectCluster connect) {
+    protected Future<List<String>> tlsTrustedCertsSecret(Reconciliation reconciliation, String namespace, KafkaConnectCluster connect) {


java doc for @return needs updating

tinaselenge · 2025-12-15T11:45:52Z

...tor/src/main/java/io/strimzi/operator/cluster/operator/assembly/AbstractConnectOperator.java

-                            return Future.succeededFuture();
-                        }
-                    });
+                    .compose(certificates -> tlsTrustedCertsSecret(reconciliation, namespace, connect, certificates));


Do we really need this to be broken down to another method? The only changes here seem to be encoding a list of certificates instead of String and then mapping the result with the list instead of empty.

I had to split it so I can use it here https://github.com/strimzi/strimzi-kafka-operator/pull/12241/files#diff-c6d9b371d64b172f4e174d61f4a6b81ed129010c41a016f0bbf29dddc62df009R138. More details in the other comment

tinaselenge · 2025-12-15T11:57:13Z

...src/main/java/io/strimzi/operator/cluster/operator/assembly/KafkaBridgeAssemblyOperator.java

            })
            .compose(i -> isPodDisruptionBudgetGeneration ? podDisruptionBudgetOperator.reconcile(reconciliation, namespace, bridge.getComponentName(), bridge.generatePodDisruptionBudget()) : Future.succeededFuture())
-            .compose(i -> ReconcilerUtils.authTlsHash(secretOperations, namespace, auth, trustedCertificates))
+            .compose(i -> ReconcilerUtils.trustedCertificates(reconciliation, secretOperations, trustedCertificates))


KafkaBridge resource currently doesn't use this internal secret so I don't think we should create one yet . The primary reason for this secret is to configure truststore directly from a secret instead of using generated certificates based on volume mounted secrets. It should be done in a separate PR like in #11549 which is closed for now.

I'll undo KafkaBridge changes as mentioned in the parent comment.

Actually reverting KafkaBridge changes is not straightforward because I changed authTlsHash. I have changed the function parameters to require a list of certificates which I fetch here.

tinaselenge · 2025-12-15T12:02:00Z

...rc/main/java/io/strimzi/operator/cluster/operator/assembly/KafkaConnectAssemblyOperator.java

        KafkaConnectBuild build;
        KafkaConnectStatus kafkaConnectStatus = new KafkaConnectStatus();
+        List<String> tlsCertificates;
+        List<String> oauthCertificates;


what are they for?

Committed by mistake, will remove

tinaselenge · 2025-12-15T12:09:03Z

...in/java/io/strimzi/operator/cluster/operator/assembly/KafkaMirrorMaker2AssemblyOperator.java

                .compose(i -> serviceOperations.reconcile(reconciliation, namespace, mirrorMaker2Cluster.getServiceName(), mirrorMaker2Cluster.generateService()))
                .compose(i -> serviceOperations.reconcile(reconciliation, namespace, mirrorMaker2Cluster.getComponentName(), mirrorMaker2Cluster.generateHeadlessService()))
-                .compose(i -> tlsTrustedCertsSecret(reconciliation, namespace, mirrorMaker2Cluster))
+                .compose(i -> updateMM2ClusterCertificateMap(reconciliation, mirrorMaker2Cluster, clusterCerts))


I'm not sure if I understand these changes? Can you please provide some explanation?

The updateMM2ClusterCertificateMap updates the clusterCerts map that's in the scope of a single reconcilation. The key is the alias of the cluster as specified in the CR and the value is a list of certs required for that cluster.

I fetch all the clusters beforehand and use them in the next stage.

I use the target cluster's alias kafkaMirrorMaker2.getSpec().getTarget().getAlias() to get the target cluster's CA cert and use that to generate the connect's internal tls secret. This just replicates the behaviour that already existed.

The original tlsTrustedCertsSecret gets certificate using connect.getTls().getTrustedCertificates() and fetches it. I didn't want it to fetch again in this context because it is already fetched in the updateMM2ClusterCertificateMap method. So I split tlsTrustedCertsSecret to be able to pass in the list of certificates from the map. The connect.getTls() method resolves to spec.target.tls from KafkaMirrorMaker2Cluster.fromCrd and eventually KafkaMirrorMaker2Cluster.buildKafkaConnectSpec usingspec.getTarget().getTls(). That's why I'm using the target's alias for tlsTrustedCertsSecret. The alias remains unique at this stage because KafkaMirrorMaker2Cluster.validateAndUpdateToNewAPI has a check for it

Later on, the map is used in generateAuthHash to generate the hash as well.

tinaselenge · 2025-12-15T12:43:18Z

...er-operator/src/main/java/io/strimzi/operator/cluster/operator/assembly/ReconcilerUtils.java

     * @return  Certificates extracted from the Secrets
     */
-    public static Future<String> trustedCertificates(Reconciliation reconciliation, SecretOperator secretOperations, List<CertSecretSource> certificateSources)   {
+    public static Future<List<String>> trustedCertificates(Reconciliation reconciliation, SecretOperator secretOperations, List<CertSecretSource> certificateSources)   {


Does this really need to be changed to a List? Can the hash not be computed from the final String instead of a List?

the auth hash method sums the hashes of each cert, but the trustedCertificates method returns the string list concatenated with "\n" as the delimiter. I believe it isn't possible to split it after it is concatenated. It would also not produce the same hash as before I assume if I just use hashCode of the concatenated string.

But if it is ok to use the hashCode of the concatenated string, it makes the implementation much simpler. I don't mind changing it if that is better in your opinion.

github-actions · 2025-12-15T14:49:08Z

🎉 System test verification passed: link

Signed-off-by: Venkatesh Kannan <[email protected]>

venkatesh2090 added 5 commits December 12, 2025 20:23

Remove redundant calls to fetch tls secrets.

68b123c

Refactored tlsTrustedCertsSecret to return a list of secrets instead of the concatenated version. Signed-off-by: Venkatesh Kannan <[email protected]>

Add tests.

8c897da

Also fixed auth hash in MM2. Signed-off-by: Venkatesh Kannan <[email protected]>

refactor: move auth hash method back to where it was

de9159f

Signed-off-by: Venkatesh Kannan <[email protected]>

fix stylcheck error

5e865b3

Signed-off-by: Venkatesh Kannan <[email protected]>

Fix NullPointerExceptions

3dd459a

Signed-off-by: Venkatesh Kannan <[email protected]>

venkatesh2090 added 2 commits December 13, 2025 14:03

refactor indentation

9b22b9d

Signed-off-by: Venkatesh Kannan <[email protected]>

Add coverage for oauth tls secret

63d40ad

Signed-off-by: Venkatesh Kannan <[email protected]>

scholzj added this to the 0.50.0 milestone Dec 15, 2025

scholzj requested review from katheris, ppatierno and tinaselenge December 15, 2025 09:42

tinaselenge reviewed Dec 15, 2025

View reviewed changes

Update javadoc, and remove unused code.

c615a6a

Signed-off-by: Venkatesh Kannan <[email protected]>

venkatesh2090 force-pushed the fix-trusted-certs branch from f1b8201 to c615a6a Compare December 21, 2025 23:36

Fetch Kafka Connect, MM2, Bridge trusted certificates Secret only once for each reference #12241

Are you sure you want to change the base?

Fetch Kafka Connect, MM2, Bridge trusted certificates Secret only once for each reference #12241

Uh oh!

Conversation

venkatesh2090 commented Dec 13, 2025

Type of change

Description

Checklist

Uh oh!

codecov bot commented Dec 13, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

scholzj commented Dec 15, 2025

Uh oh!

github-actions bot commented Dec 15, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

tinaselenge left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

venkatesh2090 Dec 21, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

tinaselenge Dec 15, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Dec 15, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

codecov bot commented Dec 13, 2025 •

edited

Loading

github-actions bot commented Dec 15, 2025 •

edited

Loading

venkatesh2090 Dec 21, 2025 •

edited

Loading

tinaselenge Dec 15, 2025 •

edited

Loading