Memory usage of Kwok clusters

[adibrastegarnia]

We need to use Kwok to simulate thousands of clusters, and I am seeing each cluster consuming around 300MB of memory. Is there a way to reduce it further?

[wzshiming]

Are you starting the cluster with an all-in-one image? It's basically the apiserver that's taking up all the memory, and I haven't found a way to reduce the apiserver's memory footprint yet.

[adibrastegarnia]

@wzshiming yes I use all in one image. I tried to disable a bunch of controllers that I don't need, it helped a bit but not much. I am not sure if even it is technically possible to reduce it further.

[wzshiming]

Yes, if you look at apiserver's pprof you can see that about 100M+ is taken up by OpenAPI v2/v3, which in many cases are not used at all. You may need to change the code to disable them.

[adibrastegarnia]

@wzshiming thanks do you have any pointer to that part?

[wzshiming]

https://github.com/kubernetes/kubernetes/blob/4bed36e03e7bd699b089d33da6f7d7c9ef9eb661/staging/src/k8s.io/apiserver/pkg/server/genericapiserver.go#L443-L453

I'm not sure which features are disabled when you disable them.

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue as fresh with /remove-lifecycle stale
• Close this issue with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

[k8s-triage-robot]

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue as fresh with /remove-lifecycle rotten
• Close this issue with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle rotten