Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

libvirt: Add podvm instance cpu/mem size support for libvirt #1650

Open
stevenhorsman opened this issue Jan 4, 2024 · 3 comments · May be fixed by #2116
Open

libvirt: Add podvm instance cpu/mem size support for libvirt #1650

stevenhorsman opened this issue Jan 4, 2024 · 3 comments · May be fixed by #2116
Labels
provider/libvirt

Comments

@stevenhorsman
Copy link
Member

stevenhorsman commented Jan 4, 2024 •
edited
Loading

We have the mechanism to pass through cpu and memory annotations from kata-containers runtime via the pod spec, to select the instance size for some of the cloud provider (see

cloud-api-adaptor/src/cloud-api-adaptor/pkg/adaptor/cloud/cloud.go

Lines 192 to 203 in a5e37f4

// Get Pod VM instance type from annotations
instanceType := util.GetInstanceTypeFromAnnotation(req.Annotations)
// Get Pod VM cpu and memory from annotations
vcpus, memory := util.GetCPUAndMemoryFromAnnotation(req.Annotations)
// Pod VM spec
vmSpec := provider.InstanceTypeSpec{
InstanceType: instanceType,
VCPUs: vcpus,
Memory: memory,
}
)

Note: Azure, AWS & IBMCloud, which also can have the instance profile type specified bia the machine type annotation, or kustomize, which isn't applicable for libvirt.

Currently libvirt doesn't have this option and I think that the vm size created is hard-coded in

cloud-api-adaptor/src/cloud-providers/libvirt/libvirt.go

Lines 532 to 533 in a5e37f4

v.cpu = uint(2)
v.mem = uint(8)
as shown in:

# kcli info vm podvm-simple-test-530932ec
name: podvm-simple-test-530932ec
id: 8af3bdff-ce2e-423a-851c-19a8d9422cf1
status: up
autostart: False
plan:
cpus: 2
memory: 8192
net interface: eth0 mac: 52:54:00:88:83:c6 net: default type: routed
ip: 192.168.122.71
diskname: sda disksize: 6GB diskformat: sata type: qcow2 path: /var/lib/libvirt/images/podvm-simple-test-530932ec-root.qcow2
iso: /var/lib/libvirt/images/podvm-simple-test-530932ec-cloudinit.iso

This is a bit of an issue as a) it's not very flexible for workloads b) for dev scenarios it's a bit of a problem as two kcli cluster nodes take 4 vCPU and 6GB vRAM, so to run the most basic single peer pods test you need 10 vCPU and 20GB RAM. If we can add support for specifying instance size of the libvirt peer pod vms that will allow the flexiiblity that you get with the other cloud providers.

I don't think that libvirt has the concept of an instance profile/flavour like the other cloud providers, so we'd want to use the io.katacontainers.config.hypervisor.default_vcpus and io.katacontainers.config.hypervisor.default_memory annotations in the pod to drive the libvirt configuration, rather than using io.katacontainers.config.hypervisor.machine_type annotation like we do for some of the other platforms. I guess the default that we bake into the libvirt kustomize will need to be split into two fields as well for this?
@bpradipt
Copy link
Member

bpradipt commented Jan 8, 2024

Is there a VM profile or flavour concept in libvirt like the cloud providers?
If not we should probably look at supporting the vcpu and memory annotations for libvirt provider to specify a non-default pod VM size

@stevenhorsman
Copy link
Member Author

Is there a VM profile or flavour concept in libvirt like the cloud providers?
If not we should probably look at supporting the vcpu and memory annotations for libvirt provider to specify a non-default pod VM size

I'm not an expert, but I didn't see a profile/flavour concept when looking around, so I'll update the issue to be clearer about using annotations.

@bpradipt
Copy link
Member

bpradipt commented Jan 8, 2024

Is there a VM profile or flavour concept in libvirt like the cloud providers?
If not we should probably look at supporting the vcpu and memory annotations for libvirt provider to specify a non-default pod VM size

I'm not an expert, but I didn't see a profile/flavour concept when looking around, so I'll update the issue to be clearer about using annotations.

Hmm.. I will check as well. Nonetheless it'll be good to update the description and mention about the annotations as well. At least the current limitation of fixed pod VM size will be get fixed

@stevenhorsman stevenhorsman changed the title libvirt: Add podvm instance size/type support for libvirt libvirt: Add podvm instance size support for libvirt May 24, 2024
@stevenhorsman stevenhorsman changed the title libvirt: Add podvm instance size support for libvirt libvirt: Add podvm instance cpu/mem size support for libvirt May 24, 2024
savitrilh added a commit to savitrilh/cloud-api-adaptor that referenced this issue Oct 14, 2024
@savitrilh
libvirt: Add podvm instance cpu and mem size support for libvirt
628f293 
Use io.katacontainers.config.hypervisor.default_vcpus and io.katacontainers.config.hypervisor.default_memory annotations to set the libvirt podvm instance. Use the default values if no annotations are provided.

Fixes: confidential-containers#1650

Signed-off-by : savitrilh <[email protected]>
@savitrilh savitrilh linked a pull request Oct 14, 2024 that will close this issue
Draft
savitrilh added a commit to savitrilh/cloud-api-adaptor that referenced this issue Oct 14, 2024
@savitrilh
libvirt: Add podvm instance cpu and mem size support for libvirt
5c98498 
Add podvm instance cpu and mem size support for libvirt

Fixes: confidential-containers#1650

Signed-off-by : savitrilh <[email protected]>
savitrilh added a commit to savitrilh/cloud-api-adaptor that referenced this issue Oct 14, 2024
@savitrilh
libvirt: Add podvm instance cpu and mem size support for libvirt
951f560 
Add podvm instance cpu and mem size support for libvirt

Fixes: confidential-containers#1650

Signed-off-by : SAVITRI HUNASHEEKATTI <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
provider/libvirt
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

libvirt: Add podvm instance cpu and mem size support for libvirt savitrilh/cloud-api-adaptor
3 participants
@bpradipt @stevenhorsman @savitrilh