TLS certificate does not support SANs for mongos headless service used with mongodb+srv

[Nuran14]

Report

We are using a sharded PerconaServerMongoDB cluster and need to connect to mongos using a headless Service with a mongodb+srv connection string.

We created a headless Service for mongos with the following DNS pattern:

-mongos-headless..svc.

The SRV record is generated correctly and mongodb+srv resolves the mongos endpoints successfully.

However, the TLS certificate generated by the PSMDB Operator does not contain SAN entries for the mongos-headless DNS names.

For example, the Operator-generated certificate contains entries such as:

• mng6-mongos
• mng6-mongos.demo
• mng6-mongos.demo.svc.ex.xx.local
• *.mng6-mongos.demo.svc.ex.xx.local

but it does not contain:

• mng6-mongos-headless.demo.svc.ex.xx.local
• *.mng6-mongos-headless.demo.svc.ex.xx.local

As a result, mongodb+srv resolves the SRV record successfully, but the TLS handshake fails because the resolved mongos hostname is not covered by the certificate SANs.

We also checked the CRD and found that splitHorizons is available under spec.replsets[], but there is no equivalent option under spec.sharding.mongos to provide additional DNS names/SANs.

More about the problem

The SRV lookup succeeds and returns mongos endpoints such as:

10-20-26-53.mng6-mongos-headless.demo.svc.ex.xx.local
10-20-26-59.mng6-mongos-headless.demo.svc.ex.xx.local

When connecting with:

mongosh "mongodb+srv://:@mng6-mongos-headless.demo.svc.ex.xx.local/admin?authSource=admin"
--tls
--tlsCAFile /tmp/ca.crt
--tlsCertificateKeyFile /tmp/client.pem

the connection fails with:

MongoServerSelectionError: Hostname/IP does not match certificate's altnames:
Host: 10-20-26-53.mng6-mongos-headless.demo.svc.ex.xx.local is not in the cert's altnames

The client certificate and CA are valid. The failure is specifically caused by hostname verification.

Using tlsAllowInvalidHostnames can work as a workaround, but we would prefer not to disable hostname validation.

Expected behavior:

There should be a supported way to add additional SANs for mongos, similar to how splitHorizons can add DNS names for replica set members.

For this use case, we need to be able to include a SAN such as:

*.mng6-mongos-headless.demo.svc.ex.xx.local

Steps to reproduce

1. Create a sharded PerconaServerMongoDB cluster with mongos enabled.

2. Create a headless Service that selects the mongos pods:

   selector:
   app.kubernetes.io/instance:
   app.kubernetes.io/component: mongos

   with a named port:

   ports:

   • name: mongodb
     port: 27017
     targetPort: 27017

3. Verify that the mongodb SRV record resolves:

   nslookup -type=SRV _mongodb._tcp.-mongos-headless..svc.

4. Export the client certificate and private key from the Operator-generated <cluster>-ssl Secret:

   kubectl get secret -ssl -n
   -o jsonpath='{.data.tls.crt}'
   | base64 -d > client.crt

   kubectl get secret -ssl -n
   -o jsonpath='{.data.tls.key}'
   | base64 -d > client.key

   Combine the certificate and private key into a single PEM file:

   cat client.crt client.key > client.pem

5. Export the CA certificate from the Operator-generated <cluster>-ca-cert Secret:

   kubectl get secret -ca-cert -n
   -o jsonpath='{.data.ca.crt}'
   | base64 -d > ca.crt

6. If the mongosh test is executed from a MongoDB pod, copy the generated files into the pod:

   kubectl cp client.pem /:/tmp/client.pem

   kubectl cp ca.crt /:/tmp/ca.crt

7. Inspect the Operator-generated TLS certificate SANs:

   kubectl get secret -ssl -n
   -o jsonpath='{.data.tls.crt}'
   | base64 -d
   | openssl x509 -noout -ext subjectAltName

8. Notice that regular mongos DNS names are present, for example:

   -mongos..svc.
   *.-mongos..svc.

   but mongos-headless DNS names are not present, for example:

   -mongos-headless..svc.
   *.-mongos-headless..svc.

9. Connect using mongodb+srv with the CA certificate and the combined client certificate/private key:

   mongosh "mongodb+srv://:@-mongos-headless..svc./admin?authSource=admin"
   --tls
   --tlsCAFile /tmp/ca.crt
   --tlsCertificateKeyFile /tmp/client.pem

10. The SRV record resolves successfully, but the connection fails with:

    Hostname/IP does not match certificate's altnames

    The failing hostname is one of the mongos endpoints returned by the SRV lookup, for example:

    .-mongos-headless..svc.

Versions

Kubernetes: v1.32.2
Operator: Percona Operator for MongoDB 1.22.0
Database: Percona Server for MongoDB 8.0.19-7

Anything else?

No response

[egegunes]

@Nuran14 This will be fixed under https://perconadev.atlassian.net/browse/K8SPSMDB-1791 in v1.24.0.

[Nuran14]

@egegunes This looks related to the existing per-pod mongos SAN issue, but our case involves SRV targets exposed through a mongos headless Service. The required hostname pattern is *.<cluster>-mongos-headless.<namespace>.<cluster-domain>.

Would it also make sense to expose a configuration option for mongos, similar to replsets[].splitHorizons, so additional DNS names could be provided and automatically included in the Operator-generated certificate SANs?

[egegunes]

I added a comment in the ticket that we should take a look at the headless service while working on it.

For additional DNS names, I think it's better to implement https://perconadev.atlassian.net/browse/K8SPSMDB-1228