OpenStack compute Error: Though the compute hosts had nova-compute service listening, those nodes were not displayed when ‘nova hypervisor-list‘ command was executed.
# nova hypervisor-list +----+----------------------------+-------+---------+ | ID | Hypervisor hostname | State | Status | +----+----------------------------+-------+---------+
But the command nova service-list didn’t have any problem in displaying the compute hosts. Well, the issue might sound little strange, but if you are stuck with a similar problem in OpenStack, then here’s the solution.
Solution:
How to fix OpenStack Compute Error – No compute node record for host?
Step 1: Since the compute hosts are not listed, the issue is related to nova-compute service. So I started to debug the issue right from the logs of nova @ compute nodes.
[compute-node]# cd /var/log/nova
[compute-node]# tailf nova-compute.log
Sample output:
ERROR nova.compute.manager [req-1d2c4e83-a57d-4e66-b8b4-81da4772b0d2 - - - - -] No compute node record for host cloudsecurity5 INFO nova.compute.resource_tracker [req-1d2c4e83-a57d-4e66-b8b4-81da4772b0d2 - - - - -] Auditing locally available compute resources for node cloudsecurity5 ERROR nova.compute.manager [req-1d2c4e83-a57d-4e66-b8b4-81da4772b0d2 - - - - -] Error updating resources for node cloudsecurity5
The above error indicates that the OpenStack scheduler is not able to find suitable compute host – probably due to invalid hypervisor or nova-compute service is not up or other configuration issue.
Step 2: Verify the configurations of Nova Compute
[compute-node]# vim /etc/nova-compute.conf
and look out for the below entries under [DEFAULT] section.
[DEFAULT] compute_driver=libvirt.LibvirtDriver [libvirt] virt_type=kvm
Also ensure /etc/nova/nova.conf has all necessary configurations to run a compute host.
The central part of nova-compute service is the virtualization driver that interface with hypervisor. OpenStack supports many compute drivers such as libvirt.LibvirtDriver for KVM , xenapi.XenAPIDriver for Xen, vmwareapi.VMwareVCDriver for VMware and hyperv.HyperVDrive for HyperV. In my case, I was using libvirt compute driver (lookout for compute_driver attribute in the above snapshot).
Step 3: Ensure libvirt daemon is running.
[compute-node]# /etc/init.d/libvirt-bin status
Fortunately, the service libvirt-bin was active, but I did saw few errors at the end of the status output as shown below:
cloudsecurity5 libvirtd[58988]: hostname: cloudsecurity5.test.in cloudsecurity5 libvirtd[58988]: End of file while reading data: Input/output error
Though libvirt-bin daemon was running, it still indicates an issue while performing an Input/Output operations.
Hence, I decided to restart libvirt-bin and its related daemon called libvirt-guests.
[compute-node]# /etc/init.d/libvirt-guests restart
[compute-node]# /etc/init.d/libvirt-bin restart
Once done, confirm if the daemon is running without any error.
[compute-node]# /etc/init.d/libvirt-bin status
Sample output at the end of the service status message:
cloudsecurity2 systemd[1]: Starting Virtualization daemon... cloudsecurity2 systemd[1]: Started Virtualization daemon. cloudsecurity2 dnsmasq[2960]: read /etc/hosts - 9 addresses cloudsecurity2 dnsmasq[2960]: read /var/lib/libvirt/dnsmasq/def cloudsecurity2 dnsmasq-dhcp[2960]: read /var/lib/libvirt/dnsmas
Step 4: Restart nova-compute service as well.
[compute-node]# /etc/init.d/nova-compute restart
In my case, libvirt daemon’s Input/Output error was the culprit. The issue is solved and now the compute nodes are registered properly as shown below:
# nova hypervisor-list +----+----------------------------+-------+---------+ | ID | Hypervisor hostname | State | Status | +----+----------------------------+-------+---------+ | 1 | cloudsecurity5.test.in | up | enabled | +----+----------------------------+-------+---------+
