I hope you know that OpenStack, offers a valuable feature called “live migration” that allows to move virtual machines (VMs) from one physical host to another without causing downtime. It is an essential capability for maintaining high availability and optimizing resource usage in cloud environments. However, while configuring and performing live migration, encountered an error “Unacceptable CPU info: CPU doesn’t have compatibility“. Below is the detailed error from /var/log/nova/nova-compute.log
:
ERROR oslo_messaging.rpc.server Traceback (most recent call last): ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming ERROR oslo_messaging.rpc.server res = self.dispatcher.dispatch(message) ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 274, in dispatch ERROR oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args) ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 194, in _do_dispatch ERROR oslo_messaging.rpc.server result = func(ctxt, **new_args) ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/nova/exception_wrapper.py", line 79, in wrapped ERROR oslo_messaging.rpc.server function_name, call_dict, binary, tb) ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ ERROR oslo_messaging.rpc.server self.force_reraise() ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise ERROR oslo_messaging.rpc.server six.reraise(self.type_, self.value, self.tb) ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/nova/exception_wrapper.py", line 69, in wrapped ERROR oslo_messaging.rpc.server return f(self, context, *args, **kw) ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/nova/compute/utils.py", line 1372, in decorated_function ERROR oslo_messaging.rpc.server return function(self, context, *args, **kwargs) ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 219, in decorated_function ERROR oslo_messaging.rpc.server kwargs['instance'], e, sys.exc_info()) ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ ERROR oslo_messaging.rpc.server self.force_reraise() ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise ERROR oslo_messaging.rpc.server six.reraise(self.type_, self.value, self.tb) ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 207, in decorated_function ERROR oslo_messaging.rpc.server return function(self, context, *args, **kwargs) ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 6796, in check_can_live_migrate_destination ERROR oslo_messaging.rpc.server block_migration, disk_over_commit) ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 8114, in check_can_live_migrate_destination ERROR oslo_messaging.rpc.server self._compare_cpu(None, source_cpu_info, instance) ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 8439, in _compare_cpu ERROR oslo_messaging.rpc.server raise exception.InvalidCPUInfo(reason=m % {'ret': ret, 'u': u}) ERROR oslo_messaging.rpc.server InvalidCPUInfo: Unacceptable CPU info: CPU doesn't have compatibility. ERROR oslo_messaging.rpc.server ERROR oslo_messaging.rpc.server 0 ERROR oslo_messaging.rpc.server ERROR oslo_messaging.rpc.server Refer to http://libvirt.org/html/libvirt-libvirt-host.html#virCPUCompareResult
How to solve Openstack Live Migration error with Unacceptable CPU info?
The cause of the error is due to difference in CPU model/architecture between the source host and the destination host.
How to verify CPU info between 2-nodes?
Step 1: Run the following command on the source host where VM currently resides to create an XML with the host physical machine’s CPU model information:
# virsh capabilities [sourceHostName]-full.xml
Step 2: Copy the [sourceHostname-full.xml
file to the Destination Host where VM needs to be migrated and run the following command to compare whether the CPU description from source host is compatible with the destination host CPU description:
# virsh cpu-compare [SourceHostname].xml CPU described in test.xml is incompatible with host CPU
Step 3: You can also verify by viewing the XML file.
# vim [SourceHostname].xml <capabilities> <host> <uuid>4c4c4544-0030-5810-8033-b6c04f515631</uuid> <cpu> <arch>x86_64</arch> <model>SandyBridge-IBRS</model> <vendor>Intel</vendor> # vim [DestinationHostname].xml <capabilities> <host> <uuid>31333735-3232-4753-4832-3032584b5433</uuid> <cpu> <arch>x86_64</arch> <model>Opteron_G3</model> <vendor>AMD</vendor>
The above comparison clearly specify that there is a mismatch in the CPU architecture and is below:
Source Host: Intel - SandyBridge-IBRS Destination Host: AMD - Opteron_G3
Conclusion
The table below illustrates the circumstances in which a Live Migration or Cold Migration is feasible according to Werner Fischer’s analysis.
Live Migration | Cold Migration | |
---|---|---|
Intel -> AMD | – | ✔ |
AMD -> Intel | – | ✔ |
Intel Xeon Scalable (Skylake) -> Intel Xeon Scalable (Skylake) | ✔ | ✔ |
Intel E5 2600 v4 (Broadwell) -> Intel Xeon Scalable (Skylake) | (only with limitation of CPU functions, e.g. EVC) | ✔ |
AMD EPYC (Zen) -> AMD EPYC (Zen) | ✔ | ✔ |
AMD Opteron (Piledriver) -> AMD EPYC (Zen) | (only with limitation of CPU functions, e.g. EVC) | ✔ |
In my situation, it is evident that conducting a Live Migration between Intel and AMD CPU architectures is not feasible. Nevertheless, I was able to successfully demonstrate a Cold Migration, where the VM is first shut down before migration and then started on the new host after the migration process.