[KVM] CPU Features for System VMs #10964
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
@blueorangutan package |
|
@bernardodemarco a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress. |
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #10964 +/- ##
=========================================
Coverage 17.17% 17.17%
- Complexity 14985 14987 +2
=========================================
Files 5869 5869
Lines 521590 521591 +1
Branches 63485 63481 -4
=========================================
+ Hits 89562 89566 +4
+ Misses 421962 421959 -3
Partials 10066 10066
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
|
Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 13638 |
|
@blueorangutan test |
|
@DaanHoogland a [SL] Trillian-Jenkins test job (ol8 mgmt + kvm-ol8) has been kicked to run smoke tests |
|
@bernardodemarco , as per your description, would it not make sense to always apply |
@DaanHoogland, the backwards incompatibility lies in the fact that the
Yes, it would. I can update the PR tomorrow to reflect this. What are your thoughts? |
|
[SF] Trillian test result (tid-13482)
|
I do not know what would be wisdom here.
intiutively, I’d just apply the user VM settings to systemVMs as well. |
+1 |
Ok, nice. ASAP I'll change the PR to address that |
bernardodemarco
force-pushed
the
cpu-features-for-systemvms
branch
from
e36ea22 to
84148e6
Compare
@DaanHoogland, @weizhouapache, done! |
|
@blueorangutan package |
|
@bernardodemarco a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress. |
|
Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 13770 |
|
@blueorangutan test |
|
@DaanHoogland a [SL] Trillian-Jenkins test job (ol8 mgmt + kvm-ol8) has been kicked to run smoke tests |
|
[SF] Trillian test result (tid-13522)
|
|
@DaanHoogland, thanks for running the integration tests! I've taken a quick look at the errors and the Management Server logs. It seems that they are related to environment issues: Could we rerun the tests? |
Let’s first try the healtcheck PR. |
|
@blueorangutan package |
|
@JoaoJandre a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress. |
|
Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 14046 |
|
@blueorangutan test |
|
@DaanHoogland a [SL] Trillian-Jenkins test job (ol8 mgmt + kvm-ol8) has been kicked to run smoke tests |
|
@sureshanaparti a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress. |
|
Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 14339 |
|
This pull request has merge conflicts. Dear author, please fix the conflicts and sync your branch with the base branch. |
bernardodemarco
force-pushed
the
cpu-features-for-systemvms
branch
from
d664399 to
739c4e7
Compare
|
@blueorangutan package |
|
@bernardodemarco a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress. |
|
Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 14440 |
rosi-shapeblue
suggested changes
Aug 13, 2025
There was a problem hiding this comment.
PR Status: Verification failed
Test 1: Agent Configuration Loading -> PASSED -> Agent correctly loads and parses custom CPU features
Configuration change:
[root@ref-trl-9225-k-Mol8-rositsa-kyuchukova-kvm1 ~] echo "guest.cpu.features=vmx" >> /etc/cloudstack/agent/agent.properties
[root@ref-trl-9225-k-Mol8-rositsa-kyuchukova-kvm1 ~] systemctl restart cloudstack-agent
Agent Logs:
[root@ref-trl-9225-k-Mol8-rositsa-kyuchukova-kvm1 ~] grep -A5 -B5 "guest.cpu.features" /var/log/cloudstack/agent/agent.log
2025-08-13 10:46:01,288 DEBUG [agent.properties.AgentPropertiesFileHandler] (main:[]) (logid:) Property [guest.cpu.features] was altered. Now using the value [vmx].
Configuration Verification:
[root@ref-trl-9225-k-Mol8-rositsa-kyuchukova-kvm2 ~]# cat /etc/cloudstack/agent/agent.properties | grep "guest.cpu.features"
guest.cpu.features=vmx
Test 2: System VM CPU Feature Application -> FAILED -> Result: Custom feature completely absent from system VM configuration despite being configured on host
Host Configuration:
[root@ref-trl-9225-k-Mol8-rositsa-kyuchukova-kvm2 ~]# cat /etc/cloudstack/agent/agent.properties | grep "guest.cpu.features"
guest.cpu.features=vmx
[root@ref-trl-9225-k-Mol8-rositsa-kyuchukova-kvm2 ~]# systemctl restart cloudstack-agent
Force System VM Recreation:
(localcloud) 🐱 > destroy systemvm id=8b39af83-0624-4adc-a8ca-0c1b362c256b
{
"systemvm": {
"arch": "x86_64",
"created": "2025-08-13T11:20:29+0000",
"dns1": "10.0.32.1",
"dns2": "8.8.8.8",
"hasannotations": false,
"hostcontrolstate": "Enabled",
"hostid": "76a50ddc-1967-4360-b0b6-88e639ff549e",
"hostname": "ref-trl-9225-k-Mol8-rositsa-kyuchukova-kvm2",
"hypervisor": "KVM",
"id": "8b39af83-0624-4adc-a8ca-0c1b362c256b",
"isdynamicallyscalable": false,
"name": "v-32-VM",
"podid": "bfc2fb12-0a2f-4f7d-b5ad-842820702ce4",
"podname": "Pod1",
"serviceofferingid": "1e02d8ea-5e35-42b3-aaaa-11b3abc9cd12",
"serviceofferingname": "System Offering For Console Proxy",
"state": "Running",
"systemvmtype": "consoleproxy",
"templateid": "ba5be1e1-778a-11f0-886f-1e00b300019d",
"templatename": "SystemVM Template (KVM)",
"zoneid": "14c975d3-5239-4592-b010-8198d0fc884a",
"zonename": "ref-trl-9225-k-Mol8-rositsa-kyuchukova"
}
}
- CloudStack auto-recreated as v-33-VM
System VM CPU Configuration Check:
[root@ref-trl-9225-k-Mol8-rositsa-kyuchukova-kvm2 ~] virsh dumpxml v-33-VM | grep -A 15 "<cpu"
<cputune>
<shares>250</shares>
</cputune>
<resource>
<partition>/machine</partition>
</resource>
<sysinfo type='smbios'>
<system>
<entry name='manufacturer'>Apache Software Foundation</entry>
<entry name='product'>CloudStack KVM Hypervisor</entry>
<entry name='serial'>8b39af83-0624-4adc-a8ca-0c1b362c256b</entry>
<entry name='uuid'>8b39af83-0624-4adc-a8ca-0c1b362c256b</entry>
</system>
</sysinfo>
<os>
<type arch='x86_64' machine='pc-i440fx-rhel7.6.0'>hvm</type>
--
<cpu mode='custom' match='exact' check='full'>
<model fallback='forbid'>qemu64</model>
<topology sockets='1' dies='1' cores='1' threads='1'/>
<feature policy='require' name='x2apic'/>
<feature policy='require' name='hypervisor'/>
<feature policy='require' name='lahf_lm'/>
<feature policy='disable' name='svm'/>
</cpu>
<clock offset='utc'>
<timer name='kvmclock'/>
</clock>
<on_poweroff>destroy</on_poweroff>
<on_reboot>restart</on_reboot>
<on_crash>destroy</on_crash>
<devices>
<emulator>/usr/libexec/qemu-kvm</emulator>
Explicit Search for Custom Feature:
bash[root@ref-trl-9225-k-Mol8-rositsa-kyuchukova-kvm2 ~]# virsh dumpxml v-33-VM | grep vmx
[root@ref-trl-9225-k-Mol8-rositsa-kyuchukova-kvm2 ~]#
[root@ref-trl-9225-k-Mol8-rositsa-kyuchukova-kvm2 ~]# virsh dumpxml v-34-VM | grep -A 15 "<cpu"
<cputune>
<shares>250</shares>
</cputune>
<resource>
<partition>/machine</partition>
</resource>
<sysinfo type='smbios'>
<system>
<entry name='manufacturer'>Apache Software Foundation</entry>
<entry name='product'>CloudStack KVM Hypervisor</entry>
<entry name='serial'>552ee123-3101-461f-9fc3-25fdeb901b8a</entry>
<entry name='uuid'>552ee123-3101-461f-9fc3-25fdeb901b8a</entry>
</system>
</sysinfo>
<os>
<type arch='x86_64' machine='pc-i440fx-rhel7.6.0'>hvm</type>
--
<cpu mode='custom' match='exact' check='full'>
<model fallback='forbid'>qemu64</model>
<topology sockets='1' dies='1' cores='1' threads='1'/>
<feature policy='require' name='x2apic'/>
<feature policy='require' name='hypervisor'/>
<feature policy='require' name='lahf_lm'/>
<feature policy='disable' name='svm'/>
</cpu>
<clock offset='utc'>
<timer name='kvmclock'/>
</clock>
<on_poweroff>destroy</on_poweroff>
<on_reboot>restart</on_reboot>
<on_crash>destroy</on_crash>
<devices>
<emulator>/usr/libexec/qemu-kvm</emulator>
Test 3: User VM Deployment -> FAILED -> Result: User VM deployment is broken due to malformed CPU XML
VM Deployment
(localcloud) 🐱 > deploy virtualmachine templateid=ba5ce9aa-778a-11f0-886f-1e00b300019d serviceofferingid=75260ae4-9971-4462-b1db-0254ef57bbee zoneid=14c975d3-5239-4592-b010-8198d0fc884a networkofferingid=07029b1d-d664-467d-8dd5-7c4351d70683
{
"account": "admin",
"accountid": "07b7b20e-778b-11f0-886f-1e00b300019d",
"cmd": "org.apache.cloudstack.api.command.admin.vm.DeployVMCmdByAdmin",
"completed": "2025-08-13T10:28:34+0000",
"created": "2025-08-13T10:28:24+0000",
"domainid": "ba55511d-778a-11f0-886f-1e00b300019d",
"domainpath": "ROOT",
"jobid": "12dd89bc-fc59-4bb5-a302-612d9400a745",
"jobinstanceid": "d8273d41-1ac3-4273-9237-71d2c17283a3",
"jobinstancetype": "VirtualMachine",
"jobprocstatus": 0,
"jobresult": {
"errorcode": 530,
"errortext": "Unable to start a VM [d8273d41-1ac3-4273-9237-71d2c17283a3] due to [Unable to create a deployment for VM instance {\"id\":28,\"instanceName\":\"i-2-28-VM\",\"state\":\"Starting\",\"type\":\"User\",\"uuid\":\"d8273d41-1ac3-4273-9237-71d2c17283a3\"}]."
},
"jobresultcode": 530,
"jobresulttype": "object",
"jobstatus": 2,
"userid": "07b8fd70-778b-11f0-886f-1e00b300019d"
}
🙈 Error: async API failed for job 12dd89bc-fc59-4bb5-a302-612d9400a745
Management Server Logs Evidence:
[root@ref-trl-9225-k-Mol8-rositsa-kyuchukova-mgmt1 ~]# tail -n 100 /var/log/cloudstack/management/management-server.log | grep -i "cpu\|feature\|vmx"
2025-08-13 10:35:56,208 DEBUG [c.c.a.t.Request] (AgentManager-Handler-19:[]) (logid:) Seq 2-673569619268599828: Processing: { Ans: , MgmtId: 32988351955357, via: 2, Ver: v1, Flags: 10, [{"com.cloud.agent.api.StartAnswer":{"vm":{"id":"31","name":"i-2-31-VM","state":"Starting","type":"User"},"result":"false","details":"XML error: Non-empty feature list specified without CPU model","wait":"0","bypassHostMaintenance":"false"}}] }
2025-08-13 10:35:56,211 INFO [c.c.v.ClusteredVirtualMachineManagerImpl] (Work-Job-Executor-87:[ctx-328980d2, job-137/job-139, ctx-e9372b0d]) (logid:bfd1c45f) Unable to start VM on Host {"id":2,"name":"ref-trl-9225-k-Mol8-rositsa-kyuchukova-kvm2","type":"Routing","uuid":"76a50ddc-1967-4360-b0b6-88e639ff549e"} due to XML error: Non-empty feature list specified without CPU model
2025-08-13 10:35:56,940 INFO [c.c.v.ClusteredVirtualMachineManagerImpl] (Work-Job-Executor-87:[ctx-328980d2, job-137/job-139, ctx-e9372b0d]) (logid:bfd1c45f) Unable to start VM on Host {"id":1,"name":"ref-trl-9225-k-Mol8-rositsa-kyuchukova-kvm1","type":"Routing","uuid":"21a79fcc-ce3c-483a-b72c-fee78e2ab1ab"} due to XML error: Non-empty feature list specified without CPU model
Agent Logs Evidence
[root@ref-trl-9225-k-Mol8-rositsa-kyuchukova-kvm1 ~]# grep -i "configure\|cpu\|feature" /var/log/cloudstack/agent/agent.log | grep -v "Retrieved statistics\|Calculated metrics" | tail -20
2025-08-13 10:35:56,850 WARN [resource.wrapper.LibvirtStartCommandWrapper] (AgentRequest-Handler-2:[]) (logid:bfd1c45f) LibvirtException org.libvirt.LibvirtException: XML error: Non-empty feature list specified without CPU model
Test 4: Host CPU Capability Verification -> PASSED -> Host supports vmx and other CPU features used in testing
[root@ref-trl-9225-k-Mol8-rositsa-kyuchukova-kvm1 ~]# virsh capabilities | grep -A 50 "<cpu>" | grep "feature name"
Output:
xml <feature name='vme'/>
<feature name='ss'/>
<feature name='vmx'/>
<feature name='osxsave'/>
<feature name='f16c'/>
<feature name='rdrand'/>
<feature name='hypervisor'/>
<feature name='arat'/>
<feature name='tsc_adjust'/>
<feature name='mpx'/>
<feature name='avx512f'/>
<feature name='avx512dq'/>
<feature name='clflushopt'/>
<feature name='clwb'/>
<feature name='avx512cd'/>
<feature name='avx512bw'/>
<feature name='avx512vl'/>
<feature name='pku'/>
<feature name='ospke'/>
<feature name='md-clear'/>
<feature name='stibp'/>
<feature name='arch-capabilities'/>
<feature name='ssbd'/>
<feature name='xsaveopt'/>
<feature name='xsavec'/>
<feature name='xsaves'/>
<feature name='pdpe1gb'/>
<feature name='abm'/>
<feature name='invtsc'/>
<feature name='rdctl-no'/>
<feature name='ibrs-all'/>
<feature name='skip-l1dfl-vmentry'/>
<feature name='mds-no'/>
Test 5: CPU Feature Processing Debug -> NO EVIDENCE -> No debug logs showing CPU feature processing for any VM type
[root@ref-trl-9225-k-Mol8-rositsa-kyuchukova-kvm1 ~]# grep -i "setFeatures\|createCpuModeDef" /var/log/cloudstack/agent/agent.log
[root@ref-trl-9225-k-Mol8-rositsa-kyuchukova-kvm1 ~]#
@rosi-shapeblue, thanks for your review! Could you further explain what verification failed? |
@bernardodemarco - I was still updating my comment; could you please check it #10964 (review) and let me know if any questions arise? |
|
@rosi-shapeblue, thanks very much for testing! Basically, Apache CloudStack allows operators to configure the CPU model that is exposed to VMs in KVM through three properties that can be defined in The
The guest.cpu.mode=custom
guest.cpu.model=Skylake-Server
guest.cpu.features=aes mmx avxAs can be noticed from the Agent logs that have been provided: 2025-08-13 10:35:56,850 WARN [resource.wrapper.LibvirtStartCommandWrapper] (AgentRequest-Handler-2:[]) (logid:bfd1c45f) LibvirtException org.libvirt.LibvirtException: XML error: Non-empty feature list specified without CPU modelLibvirt complains about specifying a list of flags without a CPU model. Therefore, to apply CPU flags in KVM, it is required to meet one of the following requirements:
Otherwise, Libvirt will complain that a CPU model has not been specified. |
This comment was marked as duplicate.
This comment was marked as duplicate.
Sorry, something went wrong.
rosi-shapeblue
approved these changes
Aug 15, 2025
There was a problem hiding this comment.
LGTM - verification passed
Test Case 1: Baseline (Mode A: host-model + flags); Status: PASS
1. Setup
[root@ref-trl-9239-k-Mol8-rositsa-kyuchukova-kvm1 ~]# sudo sed -i '/^guest\.cpu\.model/d' /etc/cloudstack/agent/agent.properties
[root@ref-trl-9239-k-Mol8-rositsa-kyuchukova-kvm1 ~]# sudo sed -i '/^guest\.cpu\.mode=.*/d' /etc/cloudstack/agent/agent.properties
[root@ref-trl-9239-k-Mol8-rositsa-kyuchukova-kvm1 ~]# echo 'guest.cpu.mode=host-model' | sudo tee -a /etc/cloudstack/agent/agent.properties
guest.cpu.mode=host-model
[root@ref-trl-9239-k-Mol8-rositsa-kyuchukova-kvm1 ~]# echo 'guest.cpu.features=-hle -rtm -mpx' | sudo tee -a /etc/cloudstack/agent/agent.properties
guest.cpu.features=-hle -rtm -mpx
[root@ref-trl-9239-k-Mol8-rositsa-kyuchukova-kvm1 ~]# sudo systemctl restart cloudstack-agent
[root@ref-trl-9239-k-Mol8-rositsa-kyuchukova-kvm1 ~]#
2. Recreate System VMs
(localcloud) 🐱 > list systemvms | jq -r '.systemvm[].id' | xargs -r -n1 -I{} cmk destroy systemvm id={}
{
"systemvm": {
"arch": "x86_64",
"created": "2025-08-14T17:13:51+0000",
"dns1": "10.0.32.1",
"dns2": "8.8.8.8",
"hasannotations": false,
"hostcontrolstate": "Enabled",
"hostid": "b6384882-f603-4425-bba1-0c5527e0c745",
"hostname": "ref-trl-9239-k-Mol8-rositsa-kyuchukova-kvm1",
"hypervisor": "KVM",
"id": "db729706-6e27-4dbf-ba2f-3d4fd73f1477",
"isdynamicallyscalable": false,
"name": "v-12-VM",
"podid": "20565719-3178-4595-8063-7d083de87802",
"podname": "Pod1",
"serviceofferingid": "714b8119-602d-4337-a85f-b77fb2b16629",
"serviceofferingname": "System Offering For Console Proxy",
"state": "Running",
"systemvmtype": "consoleproxy",
"templateid": "484370e0-7849-11f0-84c2-1e006c00031c",
"templatename": "SystemVM Template (KVM)",
"zoneid": "a4fb4f58-511f-4998-a7e2-a6604f8f1374",
"zonename": "ref-trl-9239-k-Mol8-rositsa-kyuchukova"
}
}
{
"systemvm": {
"arch": "x86_64",
"created": "2025-08-14T17:14:21+0000",
"dns1": "10.0.32.1",
"dns2": "8.8.8.8",
"hasannotations": false,
"hostcontrolstate": "Enabled",
"hostid": "b6384882-f603-4425-bba1-0c5527e0c745",
"hostname": "ref-trl-9239-k-Mol8-rositsa-kyuchukova-kvm1",
"hypervisor": "KVM",
"id": "94d54534-f531-4e60-ac92-0a9ad02654d7",
"isdynamicallyscalable": false,
"name": "s-13-VM",
"podid": "20565719-3178-4595-8063-7d083de87802",
"podname": "Pod1",
"serviceofferingid": "1ab28e97-a225-466c-9b3e-bb8c1d7a43f8",
"serviceofferingname": "System Offering For Secondary Storage VM",
"state": "Running",
"systemvmtype": "secondarystoragevm",
"templateid": "484370e0-7849-11f0-84c2-1e006c00031c",
"templatename": "SystemVM Template (KVM)",
"zoneid": "a4fb4f58-511f-4998-a7e2-a6604f8f1374",
"zonename": "ref-trl-9239-k-Mol8-rositsa-kyuchukova"
}
}
3. Verification
[root@ref-trl-9239-k-Mol8-rositsa-kyuchukova-kvm1 ~]# for n in $(virsh list --name | egrep '^(v-|s-)'); do
> echo "== $n ==";
> virsh dumpxml "$n" | sed -n '/<cpu /,/<\/cpu>/p' | egrep "cpu mode=|<model |feature.*(hle|rtm|mpx)";
> echo;
> done
== v-14-VM ==
<cpu mode='custom' match='exact' check='full'>
<model fallback='forbid'>Cascadelake-Server</model>
<feature policy='disable' name='hle'/>
<feature policy='disable' name='rtm'/>
<feature policy='disable' name='mpx'/>
== s-15-VM ==
<cpu mode='custom' match='exact' check='full'>
<model fallback='forbid'>Cascadelake-Server</model>
<feature policy='disable' name='hle'/>
<feature policy='disable' name='rtm'/>
<feature policy='disable' name='mpx'/>Status: Pass
- mode='custom'
- Cascadelake-Server present
- Features hle, rtm, mpx = disabled
Test Case 2: Mode B (host-passthrough + flags); Status: PASS
1. Setup
[root@ref-trl-9239-k-Mol8-rositsa-kyuchukova-kvm1 ~]# sudo sed -i 's/^guest\.cpu\.mode=.*/guest.cpu.mode=host-passthrough/' /etc/cloudstack/agent/agent.properties
[root@ref-trl-9239-k-Mol8-rositsa-kyuchukova-kvm1 ~]# grep "^guest.cpu.mode" /etc/cloudstack/agent/agent.properties
guest.cpu.mode=host-passthrough
[root@ref-trl-9239-k-Mol8-rositsa-kyuchukova-kvm1 ~]# systemctl restart cloudstack-agent
2. Recreate System VMs
(localcloud) 🐱 > list systemvms | jq -r '.systemvm[].id' | xargs -r -n1 -I{} cmk destroy systemvm id={}
{
"systemvm": {
"arch": "x86_64",
"created": "2025-08-14T17:19:21+0000",
"dns1": "10.0.32.1",
"dns2": "8.8.8.8",
"hasannotations": false,
"hostcontrolstate": "Enabled",
"hostid": "b6384882-f603-4425-bba1-0c5527e0c745",
"hostname": "ref-trl-9239-k-Mol8-rositsa-kyuchukova-kvm1",
"hypervisor": "KVM",
"id": "9ab9f005-77da-48f7-a92a-29fc53745b1f",
"isdynamicallyscalable": false,
"name": "v-14-VM",
"podid": "20565719-3178-4595-8063-7d083de87802",
"podname": "Pod1",
"serviceofferingid": "714b8119-602d-4337-a85f-b77fb2b16629",
"serviceofferingname": "System Offering For Console Proxy",
"state": "Running",
"systemvmtype": "consoleproxy",
"templateid": "484370e0-7849-11f0-84c2-1e006c00031c",
"templatename": "SystemVM Template (KVM)",
"zoneid": "a4fb4f58-511f-4998-a7e2-a6604f8f1374",
"zonename": "ref-trl-9239-k-Mol8-rositsa-kyuchukova"
}
}
{
"systemvm": {
"arch": "x86_64",
"created": "2025-08-14T17:19:21+0000",
"dns1": "10.0.32.1",
"dns2": "8.8.8.8",
"hasannotations": false,
"hostcontrolstate": "Enabled",
"hostid": "b6384882-f603-4425-bba1-0c5527e0c745",
"hostname": "ref-trl-9239-k-Mol8-rositsa-kyuchukova-kvm1",
"hypervisor": "KVM",
"id": "627b53de-bbad-4cbd-a579-58289e638492",
"isdynamicallyscalable": false,
"name": "s-15-VM",
"podid": "20565719-3178-4595-8063-7d083de87802",
"podname": "Pod1",
"serviceofferingid": "1ab28e97-a225-466c-9b3e-bb8c1d7a43f8",
"serviceofferingname": "System Offering For Secondary Storage VM",
"state": "Running",
"systemvmtype": "secondarystoragevm",
"templateid": "484370e0-7849-11f0-84c2-1e006c00031c",
"templatename": "SystemVM Template (KVM)",
"zoneid": "a4fb4f58-511f-4998-a7e2-a6604f8f1374",
"zonename": "ref-trl-9239-k-Mol8-rositsa-kyuchukova"
}
}
3. Verification
[root@ref-trl-9239-k-Mol8-rositsa-kyuchukova-kvm1 ~]# for n in $(virsh list --name | egrep '^(v-|s-)'); do
> echo "== $n ==";
> virsh dumpxml "$n" | sed -n '/<cpu /,/<\/cpu>/p' | egrep "cpu mode=|feature.*(hle|rtm|mpx)";
> echo;
> done
== v-16-VM ==
<cpu mode='host-passthrough' check='none' migratable='on'>
<feature policy='disable' name='hle'/>
<feature policy='disable' name='rtm'/>
<feature policy='disable' name='mpx'/>
== s-17-VM ==
<cpu mode='host-passthrough' check='none' migratable='on'>
<feature policy='disable' name='hle'/>
<feature policy='disable' name='rtm'/>
<feature policy='disable' name='mpx'/>
[root@ref-trl-9239-k-Mol8-rositsa-kyuchukova-kvm1 ~]# Test Result: Pass
- mode='host-passthrough'
- Features hle, rtm, mpx = disabled
Test Case 3: Mode C (custom + explicit model + flags); Status: PASS
1. Setup
[root@ref-trl-9239-k-Mol8-rositsa-kyuchukova-kvm1 ~]# sudo sed -i '/^guest\.cpu\./d' /etc/cloudstack/agent/agent.properties
[root@ref-trl-9239-k-Mol8-rositsa-kyuchukova-kvm1 ~]# sudo tee -a /etc/cloudstack/agent/agent.properties >/dev/null <<'EOF'
> guest.cpu.mode=custom
> guest.cpu.model=Westmere
> guest.cpu.features=-hle -rtm -mpx
> EOF
[root@ref-trl-9239-k-Mol8-rositsa-kyuchukova-kvm1 ~]# sudo systemctl restart libvirtd
[root@ref-trl-9239-k-Mol8-rositsa-kyuchukova-kvm1 ~]# sudo systemctl restart cloudstack-agent
2. Recreate System VMs
(localcloud) 🐱 > list systemvms | jq -r '.systemvm[].id' | xargs -r -n1 -I{} cmk destroy systemvm id={}
{
"systemvm": {
"arch": "x86_64",
"created": "2025-08-14T17:52:51+0000",
"dns1": "10.0.32.1",
"dns2": "8.8.8.8",
"hasannotations": false,
"hostcontrolstate": "Enabled",
"hostid": "b6384882-f603-4425-bba1-0c5527e0c745",
"hostname": "ref-trl-9239-k-Mol8-rositsa-kyuchukova-kvm1",
"hypervisor": "KVM",
"id": "cd4fa119-0a75-4512-9a83-8921cf0bfc99",
"isdynamicallyscalable": false,
"name": "v-16-VM",
"podid": "20565719-3178-4595-8063-7d083de87802",
"podname": "Pod1",
"serviceofferingid": "714b8119-602d-4337-a85f-b77fb2b16629",
"serviceofferingname": "System Offering For Console Proxy",
"state": "Running",
"systemvmtype": "consoleproxy",
"templateid": "484370e0-7849-11f0-84c2-1e006c00031c",
"templatename": "SystemVM Template (KVM)",
"zoneid": "a4fb4f58-511f-4998-a7e2-a6604f8f1374",
"zonename": "ref-trl-9239-k-Mol8-rositsa-kyuchukova"
}
}
{
"systemvm": {
"arch": "x86_64",
"created": "2025-08-14T17:53:21+0000",
"dns1": "10.0.32.1",
"dns2": "8.8.8.8",
"hasannotations": false,
"hostcontrolstate": "Enabled",
"hostid": "b6384882-f603-4425-bba1-0c5527e0c745",
"hostname": "ref-trl-9239-k-Mol8-rositsa-kyuchukova-kvm1",
"hypervisor": "KVM",
"id": "c52e61b1-94df-445b-a1bd-717ea4c591f4",
"isdynamicallyscalable": false,
"name": "s-17-VM",
"podid": "20565719-3178-4595-8063-7d083de87802",
"podname": "Pod1",
"serviceofferingid": "1ab28e97-a225-466c-9b3e-bb8c1d7a43f8",
"serviceofferingname": "System Offering For Secondary Storage VM",
"state": "Running",
"systemvmtype": "secondarystoragevm",
"templateid": "484370e0-7849-11f0-84c2-1e006c00031c",
"templatename": "SystemVM Template (KVM)",
"zoneid": "a4fb4f58-511f-4998-a7e2-a6604f8f1374",
"zonename": "ref-trl-9239-k-Mol8-rositsa-kyuchukova"
}
}
3. Verification
[root@ref-trl-9239-k-Mol8-rositsa-kyuchukova-kvm1 ~]# for n in $(virsh list --name | egrep '^(v-|s-)'); do
> echo "== $n ==";
> virsh dumpxml "$n" | sed -n '/<cpu /,/<\/cpu>/p';
> echo;
> done
== v-16-VM ==
<cpu mode='custom' match='exact' check='full'>
<model fallback='forbid'>Westmere</model>
<topology sockets='1' dies='1' cores='1' threads='1'/>
<feature policy='disable' name='hle'/>
<feature policy='disable' name='rtm'/>
<feature policy='disable' name='mpx'/>
<feature policy='require' name='vme'/>
<feature policy='require' name='pclmuldq'/>
<feature policy='require' name='x2apic'/>
<feature policy='require' name='hypervisor'/>
<feature policy='require' name='arat'/>
</cpu>
== s-17-VM ==
<cpu mode='custom' match='exact' check='full'>
<model fallback='forbid'>Westmere</model>
<topology sockets='1' dies='1' cores='1' threads='1'/>
<feature policy='disable' name='hle'/>
<feature policy='disable' name='rtm'/>
<feature policy='disable' name='mpx'/>
<feature policy='require' name='vme'/>
<feature policy='require' name='pclmuldq'/>
<feature policy='require' name='x2apic'/>
<feature policy='require' name='hypervisor'/>
<feature policy='require' name='arat'/>
</cpu>
[root@ref-trl-9239-k-Mol8-rositsa-kyuchukova-kvm1 ~]#
Test Result: Pass
- mode='host-passthrough'
<model>Westmere</model>- Disabled features: hle, rtm, mpx
- Required features: vme, pclmuldq, x2apic, hypervisor, arat
Test Case 4: Negative Control (flags set without mode/model); Status: PASS
1. Setup (intentionally wrong)
# Remove any CPU mode/model, leave only features
sudo sed -i '/^guest\.cpu\./d' /etc/cloudstack/agent/agent.properties
echo 'guest.cpu.features=-hle -rtm -mpx' | sudo tee -a /etc/cloudstack/agent/agent.properties >/dev/null
sudo systemctl restart cloudstack-agent
2. Recreate System VMs
list systemvms | jq -r '.systemvm[]?.id' | xargs -r -n1 -I{} destroy systemvm id={}
Expected Result
Libvirt rejects the domain definition: flags are specified without a CPU model; System VMs fail to start and get removed.
Actual Result
Agent raised the model/flags validation error:
LibvirtException: XML error: Non-empty feature list specified without CPU model
System VMs enter create→stop→remove loop; agent can’t find a defined domain afterward:
2025-08-14 19:09:27,093 DEBUG ... Failed to get vm :Domain not found: no domain with matching name 'v-80-VM'
2025-08-14 19:09:27,094 DEBUG ... VM v-80-VM doesn't exist, no need to stop it
...
2025-08-14 19:09:27,308 DEBUG ... Failed to get dom xml: org.libvirt.LibvirtException: Domain not found: no domain with matching name 's-81-VM'
2025-08-14 19:09:27,447 DEBUG ... VM s-81-VM doesn't exist, no need to stop it
Test Result: Pass (negative control)
- Libvirt refuses CPU flags without a model, as designed.
- System VMs do not remain running and are removed.
Test Case 5: Unsupported feature (require a feature the host lacks); Status: PASS
1. Setup
Use a newer baseline (Cascade Lake) that implies features the host lacks (e.g., avx512vnni, xgetbv1).
The host advertises Cascade Lake models among supported CPU models:
[root@ref-trl-9239-k-Mol8-rositsa-kyuchukova-kvm1 ~]# virsh cpu-models x86_64 | egrep -i 'westmere|sandy|ivy|haswell|broadwell|skylake|cascade|icelake|cooper|sapphirerapids'
Westmere
Westmere-IBRS
SandyBridge
SandyBridge-IBRS
IvyBridge
IvyBridge-IBRS
Haswell-noTSX
Haswell-noTSX-IBRS
Haswell
Haswell-IBRS
Broadwell-noTSX
Broadwell-noTSX-IBRS
Broadwell
Broadwell-IBRS
Skylake-Client
Skylake-Client-IBRS
Skylake-Client-noTSX-IBRS
Skylake-Server
Skylake-Server-IBRS
Skylake-Server-noTSX-IBRS
Cascadelake-Server
Cascadelake-Server-noTSX
Icelake-Client
Icelake-Client-noTSX
Icelake-Server
Icelake-Server-noTSX
Cooperlake
But capabilities query doesn't surface Cascade Lake details on this host (a hint of mismatch):
2. Recreate System VMs
- While ACS was recreating, QEMU logs for new VMs show immediate failure:
== v-18-VM.log ==
2025-08-14 18:28:23.270+0000: shutting down, reason=failed
2025-08-14 18:28:53.611+0000: shutting down, reason=failed
== v-20-VM.log ==
2025-08-14 18:29:23.302+0000: shutting down, reason=failed
2025-08-14 18:29:53.191+0000: shutting down, reason=failed
... (and similarly for v-22, v-24, v-26, v-28, v-30)
- Error surfaced by libvirt: Libvirt explicitly reports the host is missing required features for the requested model:
journalctl -u libvirtd | grep -i cpu
the CPU is incompatible with host CPU: Host CPU does not provide required features: avx512vnni, xgetbv1
# (repeated at 18:28, 18:29, 18:30, 18:31, 18:32, 18:33, 18:34 ...)
4. Cleanup/Removal behavior (ACS reaction)
Since the domains never actually start, ACS tears them down; agent can’t find the domain afterwards.
2025-08-14 19:09:27,093 DEBUG ... Failed to get vm :Domain not found: no domain with matching name 'v-80-VM'
2025-08-14 19:09:27,094 DEBUG ... VM v-80-VM doesn't exist, no need to stop it
...
2025-08-14 19:09:27,308 DEBUG ... Failed to get dom xml: org.libvirt.LibvirtException: Domain not found: no domain with matching name 's-81-VM'
2025-08-14 19:09:27,447 DEBUG ... VM s-81-VM doesn't exist, no need to stop it
Test Result: Pass (negative)
- Platform correctly rejects a CPU configuration that requires unsupported features (avx512vnni, xgetbv1), VMs fail to boot, and ACS removes them as expected.
Test Case 6: Deploy VMs; Status: PASS
Option A: host-model + flags (safe default)
1. Setup
[root@ref-trl-9239-k-Mol8-rositsa-kyuchukova-kvm1 ~]# sudo sed -i '/^guest\.cpu\./d' /etc/cloudstack/agent/agent.properties
[root@ref-trl-9239-k-Mol8-rositsa-kyuchukova-kvm1 ~]# echo 'guest.cpu.mode=host-model' | sudo tee -a /etc/cloudstack/agent/agent.properties
guest.cpu.mode=host-model
[root@ref-trl-9239-k-Mol8-rositsa-kyuchukova-kvm1 ~]# echo 'guest.cpu.features=-hle -rtm -mpx' | sudo tee -a /etc/cloudstack/agent/agent.properties
guest.cpu.features=-hle -rtm -mpx
[root@ref-trl-9239-k-Mol8-rositsa-kyuchukova-kvm1 ~]# sudo systemctl restart cloudstack-agent
2. Deploy VM and check CPU xml
[root@ref-trl-9239-k-Mol8-rositsa-kyuchukova-kvm1 ~]# for n in $(virsh list --name | egrep '^(v-|s-|i-)'); do
> echo "== $n ==";
> virsh dumpxml "$n" | sed -n '/<cpu /,/<\/cpu>/p' | egrep "cpu mode=|<model |feature.*(hle|rtm|mpx)";
> echo;
> done
== i-2-1504-VM ==
<cpu mode='custom' match='exact' check='full'>
<model fallback='forbid'>Cascadelake-Server</model>
<feature policy='disable' name='hle'/>
<feature policy='disable' name='rtm'/>
<feature policy='disable' name='mpx'/>Test Result: Pass - host-model resolved to custom + Cascadelake-Server; hle/rtm/mpx disabled.
Option B: host-passthrough + flags (max perf)
1. Setup
[root@ref-trl-9239-k-Mol8-rositsa-kyuchukova-kvm1 ~]# sudo sed -i '/^guest\.cpu\./d' /etc/cloudstack/agent/agent.properties
[root@ref-trl-9239-k-Mol8-rositsa-kyuchukova-kvm1 ~]# echo 'guest.cpu.mode=host-passthrough' | sudo tee -a /etc/cloudstack/agent/agent.properties
guest.cpu.mode=host-passthrough
[root@ref-trl-9239-k-Mol8-rositsa-kyuchukova-kvm1 ~]# echo 'guest.cpu.features=-hle -rtm -mpx' | sudo tee -a /etc/cloudstack/agent/agent.properties
guest.cpu.features=-hle -rtm -mpx
[root@ref-trl-9239-k-Mol8-rositsa-kyuchukova-kvm1 ~]# sudo systemctl restart cloudstack-agent
[root@ref-trl-9239-k-Mol8-rositsa-kyuchukova-kvm1 ~]#
2. Deploy VM and check CPU xml
[root@ref-trl-9239-k-Mol8-rositsa-kyuchukova-kvm1 ~]# for n in $(virsh list --name | egrep '^(v-|s-|i-)'); do
> echo "== $n ==";
> virsh dumpxml "$n" | sed -n '/<cpu /,/<\/cpu>/p' | egrep "cpu mode=|<model |feature.*(hle|rtm|mpx)";
> echo;
> done
== i-2-1505-VM ==
<cpu mode='host-passthrough' check='none' migratable='on'>
<feature policy='disable' name='hle'/>
<feature policy='disable' name='rtm'/>
<feature policy='disable' name='mpx'/>Test Result: Pass - host-passthrough; hle/rtm/mpx disabled.
Option C: custom baseline
1. Setup
[root@ref-trl-9239-k-Mol8-rositsa-kyuchukova-kvm1 ~]# sudo sed -i '/^guest\.cpu\./d' /etc/cloudstack/agent/agent.properties
[root@ref-trl-9239-k-Mol8-rositsa-kyuchukova-kvm1 ~]# cat <<'EOF' | sudo tee -a /etc/cloudstack/agent/agent.properties >/dev/null
> guest.cpu.mode=custom
> guest.cpu.model=Westmere
> guest.cpu.features=-hle -rtm -mpx
> EOF
[root@ref-trl-9239-k-Mol8-rositsa-kyuchukova-kvm1 ~]# sudo systemctl restart cloudstack-agent
2. Deploy VM and check CPU xml
[root@ref-trl-9239-k-Mol8-rositsa-kyuchukova-kvm1 ~]# for n in $(virsh list --name | egrep '^(v-|s-|i-)'); do
> echo "== $n ==";
> virsh dumpxml "$n" | sed -n '/<cpu /,/<\/cpu>/p' | egrep "cpu mode=|<model |feature.*(hle|rtm|mpx)";
> echo;
> done
== i-2-1506-VM ==
<cpu mode='custom' match='exact' check='full'>
<model fallback='forbid'>Westmere</model>
<feature policy='disable' name='hle'/>
<feature policy='disable' name='rtm'/>
<feature policy='disable' name='mpx'/>Test Result: Pass - custom + Westmere; hle/rtm/mpx disabled.
Test Case 7: Per-VM override sanity; Status PASS
1. Locate the target VM (capture current state & IDs)
(localcloud) 🐱 > list virtualmachines id=1ad45a2f-d96e-4dc9-be52-72fb8d033d5b
{
"count": 1,
"virtualmachine": [
{
"account": "admin",
"affinitygroup": [],
"arch": "x86_64",
"cpunumber": 1,
"cpuspeed": 500,
"cpuused": "17.22%",
"created": "2025-08-15T07:14:18+0000",
"deleteprotection": false,
"details": {
"cpuOvercommitRatio": "2.0"
},
"diskioread": 0,
"diskiowrite": 7,
"diskkbsread": 0,
"diskkbswrite": 40,
"displayname": "VM-1ad45a2f-d96e-4dc9-be52-72fb8d033d5b",
"displayvm": true,
"domain": "ROOT",
"domainid": "483d23ca-7849-11f0-84c2-1e006c00031c",
"domainpath": "/",
"guestosid": "486b98a6-7849-11f0-84c2-1e006c00031c",
"haenable": false,
"hasannotations": false,
"hostcontrolstate": "Enabled",
"hostid": "b6384882-f603-4425-bba1-0c5527e0c745",
"hostname": "ref-trl-9239-k-Mol8-rositsa-kyuchukova-kvm1",
"hypervisor": "KVM",
"id": "1ad45a2f-d96e-4dc9-be52-72fb8d033d5b",
"instancename": "i-2-1506-VM",
"isdynamicallyscalable": false,
"lastupdated": "2025-08-15T07:14:22+0000",
"memory": 512,
"memoryintfreekbs": -1,
"memorykbs": 524288,
"memorytargetkbs": 524288,
"name": "VM-1ad45a2f-d96e-4dc9-be52-72fb8d033d5b",
"networkkbsread": 0,
"networkkbswrite": 0,
"nic": [
{
"broadcasturi": "vlan://2619",
"deviceid": "0",
"extradhcpoption": [],
"id": "265a5f32-e248-4bb6-b030-9fdc9ec138df",
"isdefault": true,
"isolationuri": "vlan://2619",
"macaddress": "02:01:00:cc:00:06",
"networkid": "e79f511a-2178-4b50-b5e4-42ba1206af6c",
"networkname": "test",
"secondaryip": [],
"traffictype": "Guest",
"type": "L2"
}
],
"osdisplayname": "CentOS 5.5 (64-bit)",
"ostypeid": "486b98a6-7849-11f0-84c2-1e006c00031c",
"passwordenabled": false,
"pooltype": "NetworkFilesystem",
"receivedbytes": 0,
"rootdeviceid": 0,
"rootdevicetype": "ROOT",
"securitygroup": [],
"sentbytes": 0,
"serviceofferingid": "a2da53cc-5df0-440e-9562-66ca15a1220c",
"serviceofferingname": "Small Instance",
"state": "Running",
"tags": [],
"templatedisplaytext": "CentOS 5.5(64-bit) no GUI (KVM)",
"templateformat": "QCOW2",
"templateid": "48447311-7849-11f0-84c2-1e006c00031c",
"templatename": "CentOS 5.5(64-bit) no GUI (KVM)",
"templatetype": "BUILTIN",
"userid": "9a5fe895-7849-11f0-84c2-1e006c00031c",
"username": "admin",
"zoneid": "a4fb4f58-511f-4998-a7e2-a6604f8f1374",
"zonename": "ref-trl-9239-k-Mol8-rositsa-kyuchukova"
}
]
}- Apply per-VM CPU override details
(localcloud) 🐱 > update virtualmachine id=1ad45a2f-d96e-4dc9-be52-72fb8d033d5b details[0].key=guest.cpu.mode details[0].value=custom details[1].key=guest.cpu.model details[1].value=Westmere details[2].key=guest.cpu.features details[2].value="-hle -rtm -mpx"
{
"virtualmachine": {
"account": "admin",
"affinitygroup": [],
"arch": "x86_64",
"cpunumber": 1,
"cpuspeed": 500,
"cpuused": "17.58%",
"created": "2025-08-15T07:14:18+0000",
"deleteprotection": false,
"details": {
"key": "guest.cpu.mode",
"value": "custom"
},
"diskioread": 0,
"diskiowrite": 4,
"diskkbsread": 0,
"diskkbswrite": 24,
"displayname": "VM-1ad45a2f-d96e-4dc9-be52-72fb8d033d5b",
"displayvm": true,
"domain": "ROOT",
"domainid": "483d23ca-7849-11f0-84c2-1e006c00031c",
"domainpath": "/",
"guestosid": "486b98a6-7849-11f0-84c2-1e006c00031c",
"haenable": false,
"hasannotations": false,
"hypervisor": "KVM",
"id": "1ad45a2f-d96e-4dc9-be52-72fb8d033d5b",
"instancename": "i-2-1506-VM",
"isdynamicallyscalable": false,
"lastupdated": "2025-08-15T07:35:24+0000",
"memory": 512,
"memoryintfreekbs": -1,
"memorykbs": 524288,
"memorytargetkbs": 524288,
"name": "VM-1ad45a2f-d96e-4dc9-be52-72fb8d033d5b",
"networkkbsread": 0,
"networkkbswrite": 0,
"nic": [
{
"deviceid": "0",
"extradhcpoption": [],
"id": "265a5f32-e248-4bb6-b030-9fdc9ec138df",
"isdefault": true,
"macaddress": "02:01:00:cc:00:06",
"networkid": "e79f511a-2178-4b50-b5e4-42ba1206af6c",
"networkname": "test",
"secondaryip": [],
"traffictype": "Guest",
"type": "L2"
}
],
"osdisplayname": "CentOS 5.5 (64-bit)",
"ostypeid": "486b98a6-7849-11f0-84c2-1e006c00031c",
"passwordenabled": false,
"pooltype": "NetworkFilesystem",
"receivedbytes": 0,
"rootdeviceid": 0,
"rootdevicetype": "ROOT",
"securitygroup": [],
"sentbytes": 0,
"serviceofferingid": "a2da53cc-5df0-440e-9562-66ca15a1220c",
"serviceofferingname": "Small Instance",
"state": "Stopped",
"tags": [],
"templatedisplaytext": "CentOS 5.5(64-bit) no GUI (KVM)",
"templateformat": "QCOW2",
"templateid": "48447311-7849-11f0-84c2-1e006c00031c",
"templatename": "CentOS 5.5(64-bit) no GUI (KVM)",
"templatetype": "BUILTIN",
"userid": "9a5fe895-7849-11f0-84c2-1e006c00031c",
"username": "admin",
"zoneid": "a4fb4f58-511f-4998-a7e2-a6604f8f1374",
"zonename": "ref-trl-9239-k-Mol8-rositsa-kyuchukova"
}
}3. Start the VM (apply the override on boot)
4. Verify on the KVM host: VM CPU XML reflects the override
[root@ref-trl-9239-k-Mol8-rositsa-kyuchukova-kvm1 ~]# virsh dumpxml i-2-1506-VM | sed -n '/<cpu /,/<\/cpu>/p'
<cpu mode='custom' match='exact' check='full'>
<model fallback='forbid'>Westmere</model>
<topology sockets='1' dies='1' cores='1' threads='1'/>
<feature policy='disable' name='hle'/>
<feature policy='disable' name='rtm'/>
<feature policy='disable' name='mpx'/>
<feature policy='require' name='vme'/>
<feature policy='require' name='pclmuldq'/>
<feature policy='require' name='x2apic'/>
<feature policy='require' name='hypervisor'/>
<feature policy='require' name='arat'/>
</cpu>
[root@ref-trl-9239-k-Mol8-rositsa-kyuchukova-kvm1 ~]#
Test Result Per-VM override took effect (mode=custom, model=Westmere, hle/rtm/mpx disabled) and is visible in the VM’s libvirt XML.
Test Case 8: Live migration between hosts in the same mode; Status: PASS
1. Hosts Setup
-
Make sure Host A has the same config as Host B
-
Host A (kvm1):
[root@ref-trl-9239-k-Mol8-rositsa-kyuchukova-kvm1 ~]# sudo egrep -n '^(guest\.cpu\.)' /etc/cloudstack/agent/agent.properties || echo "Host-A: no guest.cpu.* lines"
9:guest.cpu.mode=custom
11:guest.cpu.model=Westmere
25:guest.cpu.features=-hle -rtm -mpx
- Host B (kvm2):
[root@ref-trl-9239-k-Mol8-rositsa-kyuchukova-kvm2 ~]# sudo egrep -n '^(guest\.cpu\.)' /etc/cloudstack/agent/agent.properties
9:guest.cpu.mode=custom
11:guest.cpu.model=Westmere
25:guest.cpu.features=-hle -rtm -mpx
2. Live-migrate Instance from Host A to Host B
(localcloud) 🐱 > migrateVirtualMachine virtualmachineid=1ad45a2f-d96e-4dc9-be52-72fb8d033d5b hostid=d63404e1-a054-41df-92fe-06a18cb1286e
{
"virtualmachine": {
"account": "admin",
"affinitygroup": [],
"arch": "x86_64",
"cpunumber": 1,
"cpuspeed": 500,
"cpuused": "17.33%",
"created": "2025-08-15T07:14:18+0000",
"deleteprotection": false,
"details": {
"cpuOvercommitRatio": "2.0",
"key": "guest.cpu.mode",
"value": "custom"
},
"diskioread": 0,
"diskiowrite": 4,
"diskkbsread": 0,
"diskkbswrite": 24,
"displayname": "VM-1ad45a2f-d96e-4dc9-be52-72fb8d033d5b",
"displayvm": true,
"domain": "ROOT",
"domainid": "483d23ca-7849-11f0-84c2-1e006c00031c",
"domainpath": "/",
"guestosid": "486b98a6-7849-11f0-84c2-1e006c00031c",
"haenable": false,
"hasannotations": false,
"hostcontrolstate": "Enabled",
"hostid": "d63404e1-a054-41df-92fe-06a18cb1286e",
"hostname": "ref-trl-9239-k-Mol8-rositsa-kyuchukova-kvm2",
"hypervisor": "KVM",
"id": "1ad45a2f-d96e-4dc9-be52-72fb8d033d5b",
"instancename": "i-2-1506-VM",
"isdynamicallyscalable": false,
"jobid": "8a9ec386-9cbb-467a-aaa4-f3454b4ae736",
"jobstatus": 0,
"lastupdated": "2025-08-15T08:07:01+0000",
"memory": 512,
"memoryintfreekbs": -1,
"memorykbs": 524288,
"memorytargetkbs": 524288,
"name": "VM-1ad45a2f-d96e-4dc9-be52-72fb8d033d5b",
"networkkbsread": 0,
"networkkbswrite": 0,
"nic": [
{
"broadcasturi": "vlan://2619",
"deviceid": "0",
"extradhcpoption": [],
"id": "265a5f32-e248-4bb6-b030-9fdc9ec138df",
"isdefault": true,
"isolationuri": "vlan://2619",
"macaddress": "02:01:00:cc:00:06",
"networkid": "e79f511a-2178-4b50-b5e4-42ba1206af6c",
"networkname": "test",
"secondaryip": [],
"traffictype": "Guest",
"type": "L2"
}
],
"osdisplayname": "CentOS 5.5 (64-bit)",
"ostypeid": "486b98a6-7849-11f0-84c2-1e006c00031c",
"passwordenabled": false,
"pooltype": "NetworkFilesystem",
"receivedbytes": 0,
"rootdeviceid": 0,
"rootdevicetype": "ROOT",
"securitygroup": [],
"sentbytes": 0,
"serviceofferingid": "a2da53cc-5df0-440e-9562-66ca15a1220c",
"serviceofferingname": "Small Instance",
"state": "Running",
"tags": [],
"templatedisplaytext": "CentOS 5.5(64-bit) no GUI (KVM)",
"templateformat": "QCOW2",
"templateid": "48447311-7849-11f0-84c2-1e006c00031c",
"templatename": "CentOS 5.5(64-bit) no GUI (KVM)",
"templatetype": "BUILTIN",
"userid": "9a5fe895-7849-11f0-84c2-1e006c00031c",
"username": "admin",
"zoneid": "a4fb4f58-511f-4998-a7e2-a6604f8f1374",
"zonename": "ref-trl-9239-k-Mol8-rositsa-kyuchukova"
}
}- Instance is migrated on Host B
[root@ref-trl-9239-k-Mol8-rositsa-kyuchukova-kvm2 ~]# virsh list --all | grep -w i-2-1506-VM
1 i-2-1506-VM running
3. Confirm the CPU XML still matches (mode/model/flags)
[root@ref-trl-9239-k-Mol8-rositsa-kyuchukova-kvm2 ~]# virsh dumpxml i-2-1506-VM | sed -n '/<cpu[ >]/,/<\/cpu>/p'
<cpu mode='custom' match='exact' check='full'>
<model fallback='forbid'>Westmere</model>
<topology sockets='1' dies='1' cores='1' threads='1'/>
<feature policy='disable' name='hle'/>
<feature policy='disable' name='rtm'/>
<feature policy='disable' name='mpx'/>
<feature policy='require' name='vme'/>
<feature policy='require' name='pclmuldq'/>
<feature policy='require' name='x2apic'/>
<feature policy='require' name='hypervisor'/>
<feature policy='require' name='arat'/>
</cpu>
@bernardodemarco have you raised any doc PR with these details? if not, can you create one. thanks. |
|
@rosi-shapeblue, thanks for testing this one!
@sureshanaparti, sure, I'll try to open a doc PR ASAP |
|
@sureshanaparti , this being a non-LTS release, should we merge anyway? (in spite of missing/while waiting on docs.) |
thanks @bernardodemarco |
github-project-automation
bot
moved this from In Progress
to Done
in Apache CloudStack 4.21.0
dhslove
pushed a commit
to ablecloud-team/ablestack-cloud
that referenced
this pull request
Sep 4, 2025
* CPU features for System VMs * Apply guest.cpu.features for System VMs
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Currently, when defining the CPU configuration of VMs with KVM, the Apache CloudStack Agent executes the following workflow:
cloudstack/plugins/hypervisors/kvm/src/main/java/com/cloud/hypervisor/kvm/resource/LibvirtComputingResource.java
Lines 2980 to 2993 in 41b4f0a
As can be noticed, the CPU features are only considered for end-user VMs; they are completely ignored for system VMs. This can lead to system VMs deployment inconsistencies and errors. For instance, when it is required to disable CPU flags for a given CPU model, because the host CPU does not support such flags, an error similar to the following will be returned by Libvirt when trying to deploy system VMs:
Error while deploying VM. org.libvirt.LibvirtException: the CPU is incompatible with host CPU: Host CPU does not provide required features: hle, rtm, avx512-bf16, taa-noTherefore, this PR proposes to add a new property, calledsystemvm.guest.cpu.features, to define CPU features for system VMs.(Edit) Therefore, this PR proposes to consider the CPU features defined in the
guest.cpu.featuresproperty when provisioning system VMs.Types of changes
Feature/Enhancement Scale or Bug Severity
Feature/Enhancement Scale
Screenshots (if appropriate):
How Has This Been Tested?
agent.propertiesof the KVM hosts:virsh dumpxml --domain r-15-VMvirsh dumpxml --domain i-2-14-VM