Optimize init pool: Reduce VM/Pod API calls #577
Optimize init pool: Reduce VM/Pod API calls #577kubevirt-bot merged 6 commits intok8snetworkplumbingwg:mainfrom
Conversation
…y usage Introduce pagination with 500 namespaces per page when gathering managed namespaces during initialization. This prevents loading all namespaces into memory at once in clusters with thousands of namespaces. Signed-off-by: Ram Lavi <ralavi@redhat.com>
Change getManagedNamespaces to return map[string]struct{} instead of
[]string, making it more idiomatic Go.
This set will be used in future commits to easily check whether a ns is
managed or not.
Signed-off-by: Ram Lavi <ralavi@redhat.com>
Currently during VM pool map init, only managed namespaces are called, but for each we do an API call, which can be slow in clusters with large number of namespaces. Replace per-namespace VM listing with cluster-wide listing and in-memory filtering, while keeping the pagination to avoid over using the memory. This dramatically reduces API calls during initialization. The Trade-off is fetching VMs from unmanaged namespaces then filters them out, but the network overhead is negligible compared to the latency savings from fewer round trips. Signed-off-by: Ram Lavi <ralavi@redhat.com>
Increase the pagination limit from 100 to 500 when fetching VMs during initialization. This further reduces API calls by fetching larger batches. Signed-off-by: Ram Lavi <ralavi@redhat.com>
Currently during Pod pool map init, only managed namespaces are called, but for each we do an API call, which can be slow in clusters with large number of namespaces. Replace per-namespace Pod listing with cluster-wide listing and in-memory filtering, while keeping the pagination to avoid over using the memory. This dramatically reduces API calls during initialization. The Trade-off is fetching Pods from unmanaged namespaces then filters them out, but the network overhead is negligible compared to the latency savings from fewer round trips. Signed-off-by: Ram Lavi <ralavi@redhat.com>
Increase the pagination limit from 100 to 500 when fetching Pods during initialization. This further reduces API calls by fetching larger batches. Signed-off-by: Ram Lavi <ralavi@redhat.com>
|
/cherry-pick release-0.49 |
|
@RamLavi: once the present PR merges, I will cherry-pick it on top of DetailsIn response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. |
|
Checked on CNV2.eng cluster: This shows that initMap is now more optimal. |
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: qinqon The full list of commands accepted by this bot can be found here. The pull request process is described here DetailsNeeds approval from an approver in each of these files:
Approvers can indicate their approval by writing |
|
@RamLavi: new pull request created: #578 DetailsIn response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. |
3 participants
What this PR does / why we need it:
Currently during Pod/VM pool map init, only managed namespaces are called, but for each we do an API call, which can be slow in clusters with large number of namespaces.
Replace per-namespace VM/Pod listing with cluster-wide listing and in-memory filtering, while slightly raising the pagination to 500 per call, to avoid over using the memory.
This dramatically reduces API calls during initialization.
This PR also has a small side quest: adds pagination to getManagedNamespaces, in order to control the memory usage in clusters with many namespaces.
The Trade-off is fetching VMs/Pods from unmanaged namespaces then filters
them out, but the network overhead is negligible compared to the latency
savings from fewer API round trips.
Special notes for your reviewer:
Release note: