Need help with "x509: certificate signed by unknown authority"

[sense-amid-madness]

After the latest update of the Let's Encrypt certificates for my domain due to imminent expiration, I now have a problem with my kubernetes apiserver not being able to talk to dex anymore.

Everything was working fine before, now the apiserver reports

"invalid bearer token, oidc: verify token: failed to verify signature: fetching keys oidc: get keys failed Get https://dex-server:32000/dex/keys: x509: certificate signed by unknown authority"

I have no idea where I could start debugging this issue - can anyone shed some light on what could be the issue? Why is Let's Encrypt not recognized as a valid authority anymore?

[sagikazarmark]

Do you actually use dex-server as a hostname in your configuration? Or do you use a fqdn with letsencrypt properly setup?

I'm asking, becausedex-server sounds like an internal service and I guess you configured an Ingress that does all the letsencrypt magic for you.

Sadly, that won't work with the internal service. One thing you can do, is configuring the fqdn of your service in Kubernetes.

[heidemn]

One thing you can do, is configuring the fqdn of your service in Kubernetes

Interesting problem.... I was also recently asking myself how to achieve this.

Since these won't be implemented kubernetes/kubernetes#39792 , coredns/coredns#2214 , I guess you would have to adjust the CoreDNS config as explained here: https://coredns.io/2017/05/08/custom-dns-entries-for-kubernetes/

If your ingress is responsible for the Let's Encrypt cert, then the additional config entry would look like this:
rewrite name your-domain.com your-ingress-service.your-namespace.svc.cluster.local
(disclaimer: I haven't tried this out yet)

---

Unrelated to the above, googling "let's encrypt certificate signed by unknown authority" suggests that your certificate is used, but somehow the certificate chain is not correctly configured.

[sagikazarmark]

What I meant by that is configuring Kubernetes to use the FQDN mentioned in the post. How that is resolved is an entirely different question. From the little information posted in this question I have to assume that it's registered in some sort of DNS.