can't access cluster: expired TLS cert

[cmusser]

I am having the same problems as in #9457 and am having trouble getting back on track. Both talosctl and kubectl no longer work.

❯ talosctl dashboard
failed to get node "192.168.50.101" version: rpc error: code = Unavailable desc = last connection error: connection error: desc = "error reading server preface: remote error: tls: expired certificate"

~ 
❯ kubectl get pods  
E1227 22:12:45.039731   67957 memcache.go:265] "Unhandled Error" err="couldn't get current server API group list: the server has asked for the client to provide credentials"
E1227 22:12:45.046543   67957 memcache.go:265] "Unhandled Error" err="couldn't get current server API group list: the server has asked for the client to provide credentials"
E1227 22:12:45.053475   67957 memcache.go:265] "Unhandled Error" err="couldn't get current server API group list: the server has asked for the client to provide credentials"
E1227 22:12:45.060126   67957 memcache.go:265] "Unhandled Error" err="couldn't get current server API group list: the server has asked for the client to provide credentials"
E1227 22:12:45.065520   67957 memcache.go:265] "Unhandled Error" err="couldn't get current server API group list: the server has asked for the client to provide credentials"
error: You must be logged in to the server (the server has asked for the client to provide credentials)

My specific questions with the recovery procedure that one user posted are:

1. In the following two commands, where does controlplane.yaml file come from. Was that something that I needed to obtain beforehand or is it lying around in a place I don't know about:

yq -r .machine.ca.crt controlplane.yaml | base64 -d > ca.crt
yq -r .machine.ca.key controlplane.yaml | base64 -d > ca.key

2. Generating a new cert and key and putting into the talos config seems straightforward (those are the next few commands in that procedure). But in this command: talosctl kubeconfig -n ... -e ..., what the -n and -e argument values? I have 3 control plane nodes and one worker. Do I send to one endpoint (-e) and target all the nodes?

3. More generally, I don't understand conceptually what happened. A certificate expired, but where? What uses it, to whom is it presented and for what purpose? How do I prevent this from happening? The answers to those won't me get up and running (except for that last one), but it would sure help to understand all the moving parts.

[cmusser]

Should have mentioned: this is Talos 1.9.1

[cmusser]

Solved. It's the same solution as in #9457. I had a "controlplane.yaml" file by another name, specifically, individual files for each of my three worker nodes. All of them had the same ca cert and key.

Given all that:

1. I did notice that the admin cert from talosctl gen does have only a 24 hour lifespan. What are the implications of that. Will talosctl commands that hit the cluster stop working a day from now?

2. I ran talosctl kubeconfig and that seems to have made a cert (in .kube/config) that has a year lifetime. That looks good. I ran it for each of the control plane nodes and it prompted me auth "admin@k1-a54" already exists [(r)ename/(o)verwrite]: . I chose "overwrite". That seemed appropriate, but was it need on all three or would doing it on one suffice?

[smira]

If you have a valid talosconfig, you can generate one with Talos using talosctl config new where you can control the TTL and other properies.

[cmusser]

Yep, ok. I ended up allowing the 1 day cert to expire, but I was able to regenerate it with the procedure above. What you're saying is: if I have a valid talosconfig (valid in this case means the cert in it isn't expired), I could have more easily regenerated the cert (maybe the whole thing, CA, key, CSR, and cert) with talosctl config new That command defaults to a year expiry, according to its help. Anyway, I am back in action. Takeaway: make sure you regenerate kubeconfig and talosconfig at least every year. Thanks for the help, @smira .