-
Notifications
You must be signed in to change notification settings - Fork 123
libvirt: add cpuset support for CPU pinning #2744
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
libvirt: add cpuset support for CPU pinning #2744
Conversation
|
The CI e2e test failure in e2e / podvm / Create pod vm image (ubuntu, s390x, generic) is unrelated to this PR. Root cause: DNS resolution failure (curl: (6) Could not resolve host) when fetching kata-containers commit SHA from GitHub API in podvm/Makefile.inc:63. Evidence:
|
|
cc @ajaypvictor |
|
Hi @tak-ka3 , I think it's missing to expose the new option to the user deploying caa. You will need to:
|
|
Thanks @wainersm for the review! You're right. I checked and found that manager.go already has LIBVIRT_CPUSET environment variable support (line 46), but I missed adding it to kustomization.yaml. I'll add the LIBVIRT_CPUSET entry to the peer-pods-cm configmap. |
Add --cpuset flag and LIBVIRT_CPUSET env var to pin vCPUs to specific physical CPUs. Useful for AI inferencing where cores from different sockets improve performance. Changes: - Add CPUSet field to Config and vmConfig structures - Add --cpuset CLI flag and LIBVIRT_CPUSET environment variable - Embed CPUSet attribute in Domain XML for all architectures - Expose LIBVIRT_CPUSET in kustomization.yaml for deployment - Add unit tests for CPUSet functionality Fixes: confidential-containers#2397 Signed-off-by: tak-ka3 <[email protected]>
ce6cbb6 to
a6ae9b3
Compare
| #- LIBVIRT_VOL_NAME="" # Uncomment and set if you want to use a specific volume name. Defaults to podvm-base.qcow2 | ||
| #- LIBVIRT_CPU="2" | ||
| #- LIBVIRT_MEMORY="8192" | ||
| #- LIBVIRT_CPUSET="" # Uncomment and set to pin vCPUs to physical CPUs (e.g., "0,2,4,6" or "0-3") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
can we have an option to provide this as an annotation as well? similar to how we could provide vcpus as io.katacontainers.config.hypervisor.default_vcpus?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm not sure how do-able this is which our current set-up - the annotation are provided via the kata runtime and having a new annotation there for a single remote cloud-provider doesn't feel like the right approach, unless it can be transparently passed through?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
thanks steve for the clarification, was wondering if there was a similar or equivalent annotation from kata for cpusets which we could leverage similar to the image annotation (I agree that it's not a cleaner way).
Summary
Add support for the
--cpusetflag to pin vCPUs to specific physical CPUs in the libvirt provider. This enables CPU pinning for AI inferencing use cases where ensuring cores belong to different sockets can improve performance.Fixes: #2397
Changes
CPUSetfield toConfigandvmConfigstructs--cpusetCLI flag andLIBVIRT_CPUSETenvironment variableUsage
CLI flag:
cloud-api-adaptor libvirt --cpuset "0,2,4,6"Environment variable:
Supported formats:
"0,2,4,6""0-7""0,2,4-7,10"Generated XML
Test Results
Unit Tests ✅
All unit tests pass in libvirt environment:
Integration Test ✅
Verified on Azure VM (Ubuntu 22.04, libvirt 8.0.0):
--vcpus 2,cpuset=0-1Test plan
🤖 Generated with Claude Code