backend: Fix missing k8s token when in "in-cluster" mode

[jamowei]

This PR fixes the issue #1826 that you have to provide an extra k8s token in the UI although headlamp is running inside a k8s cluster with in-cluster mode enabled.

This are the logs you get, which are also confusing then:

{"level":"info","source":"/headlamp/backend/cmd/headlamp.go","line":305,"time":"2024-03-18T22:51:07Z","message":"Creating Headlamp handler"}
{"level":"info","source":"/headlamp/backend/cmd/headlamp.go","line":306,"time":"2024-03-18T22:51:07Z","message":"Kubeconfig path: "}
{"level":"info","source":"/headlamp/backend/cmd/headlamp.go","line":307,"time":"2024-03-18T22:51:07Z","message":"Static plugin dir: /headlamp/static-plugins"}
{"level":"info","source":"/headlamp/backend/cmd/headlamp.go","line":308,"time":"2024-03-18T22:51:07Z","message":"Plugins dir: /headlamp/plugins"}
{"level":"info","source":"/headlamp/backend/cmd/headlamp.go","line":309,"time":"2024-03-18T22:51:07Z","message":"Dynamic clusters support: false"}
{"level":"info","source":"/headlamp/backend/cmd/headlamp.go","line":310,"time":"2024-03-18T22:51:07Z","message":"Helm support: false"}
{"level":"info","source":"/headlamp/backend/cmd/headlamp.go","line":311,"time":"2024-03-18T22:51:07Z","message":"Proxy URLs: []"}
{"level":"info","context":"main","clusterURL":"https://10.96.0.1:443","source":"/headlamp/backend/pkg/kubeconfig/kubeconfig.go","line":172,"time":"2024-03-18T22:51:07Z","message":"Proxy setup"}
*** Headlamp Server ***
  API Routers:
{"level":"error","source":"/headlamp/backend/cmd/headlamp.go","line":365,"error":"read /: is a directory","time":"2024-03-18T22:51:07Z","message":"loading kubeconfig"}
{"level":"error","source":"/headlamp/backend/cmd/headlamp.go","line":376,"error":"open /home/headlamp/.config/Headlamp/kubeconfigs/config: no such file or directory","time":"2024-03-18T22:51:07Z","message":"loading dynamic kubeconfig"}

The problem was that the kubeconfig.GetInClusterContext() function did not use the service-token which gets mounted to the pod when creating the cluster-credentials.

I tested the fix with minikube and now I don't have to pass an extra token when opening the UI 🙂

[illume]

Thanks for this fix.

It looks like there's a formatting issue. Maybe this will fix it: make backend-format

[jamowei]

Thanks for this fix.

It looks like there's a formatting issue. Maybe this will fix it: make backend-format

Ah ok, thanks for the hint. I fixed it 🙂

[joaquimrocha]

@jamowei , left a comment. Also, can you squash these give a title + description to the commit?
The title should be in imperative mood (e.g. 'Fix', not 'Fixed' or 'Fixes') and have a backend: prefix.

[jamowei]

@jamowei , left a comment. Also, can you squash these give a title + description to the commit? The title should be in imperative mood (e.g. 'Fix', not 'Fixed' or 'Fixes') and have a backend: prefix.

@joaquimrocha Ok now?

[joaquimrocha]

@jamowei The 1st commit is now in the right format, thanks! It still seems though like there are several commits, including a merge commit and they are not all related to the changes. Please squash these together, and maybe remove the changes to the initial logging, unless you want to propose those in a different commit. I asked some colleagues for help.

[joaquimrocha]

@illume I think you did some changes related to this in the past, in case you still remember and this is related.

[joaquimrocha]

@yolossn , can you help me understand if this is fine?

[yolossn]

This change will make the in-cluster deployments open without any kind of authentication. This is a major change in the existing functionality.

[joaquimrocha]

BTW, each commit (although I think this PR should just be one) needs to be signed off. That's done by running git commit -s or, if you have already your commit done, git commit -s --amend.

[illume]

I reran this job that failed... https://github.com/headlamp-k8s/headlamp/actions/runs/11580255626/job/32326141784?pr=2477

(Maybe it was an intermittent error.)

[illume]

hrmmm. The Build Container and test job still seems to be failing. I'll try it again later, and if it's still failing I'll look into it further.

[mdoerries]

@illume @joaquimrocha Is there any update? We also urgently need this function!

[joaquimrocha]

@mdoerries We are now prioritizing this.

[jamowei]

hrmmm. The Build Container and test job still seems to be failing. I'll try it again later, and if it's still failing I'll look into it further.

I think you have to fix the E2E tests with playwright. There it looks like a token is expected, when running in "in-cluster" mode... Which in my opinion is wrong 😉

[joaquimrocha]

@jamowei , I got @yolossn (one of our team's engineers) to take a look and he's raised the concern that getting the token into the config like that, when in-cluster, may lead to a security problem because you are going to configure the access for all users that way, i.e. if Headlamp successfully accesses the cluster, it will not prompt the user for a token, and this in a shared environment can be a security problem, at least for those who do not expect this behavior.

AFAIU, you can accomplish the same by using a kubeconfig with the cluster's data (including token) and using the -kubeconfig even if you are deploying Headlamp. WDYT?

[joaquimrocha]

@jamowei Given my previous comment, I don't think we can merge this PR. At least not as is, we'd require a flag like -in-cluster-shared-token=true + docs about it for this to feel safe enough.

[jamowei]

@jamowei , I got @yolossn (one of our team's engineers) to take a look and he's raised the concern that getting the token into the config like that, when in-cluster, may lead to a security problem because you are going to configure the access for all users that way, i.e. if Headlamp successfully accesses the cluster, it will not prompt the user for a token, and this in a shared environment can be a security problem, at least for those who do not expect this behavior.

AFAIU, you can accomplish the same by using a kubeconfig with the cluster's data (including token) and using the -kubeconfig even if you are deploying Headlamp. WDYT?

Ok, I understand this point und I will try the -kubeconfig parameter instead. But for easy use and easy setup an extra parameter would be great! Maybe I have time to extend my PR for that...

[jamowei]

@jamowei Given my previous comment, I don't think we can merge this PR. At least not as is, we'd require a flag like -in-cluster-shared-token=true + docs about it for this to feel safe enough.

I would name it -use-service-account in combination with -in-cluster parameter, which then will enables the auto-login via the mounted service account token 😉

What do you think @joaquimrocha @mdoerries ?

[joaquimrocha]

I would name it -use-service-account in combination with -in-cluster parameter, which then will enables the auto-login via the mounted service account token 😉

What do you think @joaquimrocha @mdoerries ?

I think we need an arg name that clearly states what it is and what context, because maybe some admins would think use-service-account is for prompting users on the client side to use an SA token. I am not 100% happy with my previous suggestion for a name, but something along those lines, where it's clearly what we are turning on/off would be better IMO.
More suggestions are welcome!

[k8s-triage-robot]

Unknown CLA label state. Rechecking for CLA labels.

Send feedback to sig-contributor-experience at kubernetes/community.

/check-cla
/easycla

[linux-foundation-easycla]

• ❌ - login: @jamowei / name: Jan Weidenhaupt . The commit (936935b) is not authorized under a signed CLA. Please click here to be authorized. For further assistance with EasyCLA, please submit a support request ticket.