K8s API cache, pagination and search

[illume]

This issue is for an LXF mentorship project, see https://mentorship.lfx.linuxfoundation.org/project/1b9a4a99-2d92-4c1a-ac34-7fe554bf6393 There you can learn more and apply if interested. Please do not start this issue, as it is reserved for that project.

---

Having a cache layer for the Kubernetes API will allow us to reduce load on the Kubernetes API, and also enable some faster usage of the data.

There are two main improvements a K8s API cache in the backend headlamp-server will allow:

1. Kubernetes API pagination could be done better to allow grabbing from the frontend only the page of data the client is viewing. Rather than having to download several pages first.
2. For search, it would be better to be able to search the cached data from the API so again the frontend does not need to download it all.

It's important to consider the access to the cache is only by the same contexts which access it. So that data from the cache does not go out to the wrong people.

This is about implementing the API cache in the backend, providing a better pagination API, and providing a search API.

Additional context

Virtualization issue with discussion of pagination issue: #1312

Kubernetes Dashboard PR and design: https://github.com/kubernetes/dashboard/pull/9521/files#diff-dc34dd6f7585add2ed0a897e077541423b103eb8ee64a331beb494fdb3e6f4ed

Video explaining the issue: https://www.youtube.com/watch?v=1n6tBTj2sFM

[niladrix719]

Interesting problem statement, and the video explanation was very intuitive as well. Will there be any pre-LFX discussion around it, perhaps during community calls or elsewhere? I'd love to be a part of it

[ChayanDass]

@illume I'd love to contribute for this project under lfx-mentorship

[meethereum]

Sounds like something I would love to work upon.

This is about implementing the API cache in the backend, providing a better pagination API, and providing a search API.

Just wanted to ask if it is only expected to implement the cache in backend or also integrate it with the frontend to make it end-to-end?

[akshayw1]

@illume Want to work this, Could you please assign this?

[rishabhr4]

hi @illume
would love to work upon this project.
are there any related issues which could be picked up related to this issue?

[illume]

Welcome! everyone.

hi @niladrix719

Interesting problem statement, and the video explanation was very intuitive as well. Will there be any pre-LFX discussion around it, perhaps during community calls or elsewhere? I'd love to be a part of it

I think here and maybe in the headlamp slack channel will be discussion about it before the selection is made. We'll be answering any questions here.

• We have a headlamp channel on the kubernetes slack: https://kubernetes.slack.com/messages/headlamp
• Not as popular as the slack channel is the github discussions: https://github.com/kubernetes-sigs/headlamp/discussions

Hello @ChayanDass

I'd love to contribute for this project under lfx-mentorship

Great! We look forward to your application.

Hi @akshayw1

Want to work this, Could you please assign this?

This will be part of an LXF mentorship program. The person selected from all the applicants will be assigned.

Hello @rishabhr4

would love to work upon this project.
are there any related issues which could be picked up related to this issue?

• In Github we have some "good first issue" labeled. Also "backend" issues are about backend.
• See the development guide to get started

[Jaishree2310]

Hi @illume,
I'm Jaishree Singh, a software engineer with experience in the MERN stack and active contributions to open-source projects.
I am willing to submit a proposal for this project.
Let me know how to proceed further.
Thanks!

[aryasoni98]

Hello @illume
I’d love to be a part of this project and contribute. Looking forward to working together!