Nova service controls where the instances are started within OpenStack cloud environment. It means, the nova-scheduler decides which hypervisor will run a particular instance and this behavior is controlled using two filters (Affinity/anti-affinity group policies) – namely, ServerGroupAntiAffinityFilter and ServerGroupAffinityFilter. The ServerGroupAffinityFilter allows a user to run group of instances on the same hypervisor/host and ServerGroupAntiAffinityFilter will ensure all the requested instances are started on different hypevisors/hosts.
These filters are enabled by default in /etc/nova/nova.conf as shown below:
scheduler_driver_task_period = 60 scheduler_driver = nova.scheduler.filter_scheduler.FilterScheduler scheduler_available_filters = nova.scheduler.filters.all_filters scheduler_default_filters = RetryFilter, AvailabilityZoneFilter, RamFilter, ComputeFilter, ComputeCapabilitiesFilter, ImagePropertiesFilter, ServerGroupAntiAffinityFilter, ServerGroupAffinityFilter
In case, if you don’t find those filters, then edit nova.conf, add the above lines and restart nova-scheduler and nova-conductor services.
# /etc/init.d/nova-scheduler restart
# /etc/init.d/nova-conductor restart
Note: Restarting nova-scheduler service alone is enough on some versions of OpenStack, but in OpenStack Mitaka, I need to restart nova-conductor service as well.
To work with these filters, the user needs to create a server group, assign policy to the group and add instances to that group.
Note: The below commands were executed on OpenStack Mitaka setup.
How to use ServerGroupAffinityFilter policy?
Step 1: Create a server group
# nova server-group-create <Group_Name> affinity
# nova server-group-create affinityHost affinity
+--------------------------------------+--------------+------------------------- ---------+----------------------------------+---------------+---------+--------- -+
| Id | Name | Project Id | User Id | Policies | Members | Metadata |
+--------------------------------------+--------------+------------------------- ---------+----------------------------------+---------------+---------+--------- -+
| 565948b0-076e-4de0-97dd-11d58862b64e | affinityHost | 9563fa5bb6dd479aa9f18048 cd35ca36 | d4053faa2c374045bd37d96ccb7f1b6c | [u'affinity'] | [] | {} |
+--------------------------------------+--------------+------------------------- ---------+----------------------------------+---------------+---------+--------- -+Ensure that the group is created using the below command:
# nova server-group-get 565948b0-076e-4de0-97dd-11d58862b64e
+--------------------------------------+--------------+----------------------------------+----------------------------------+---------------+---------+----------+
| Id | Name | Project Id | User Id | Policies | Members | Metadata |
+--------------------------------------+--------------+----------------------------------+----------------------------------+---------------+---------+----------+
| 565948b0-076e-4de0-97dd-11d58862b64e | affinityHost | 9563fa5bb6dd479aa9f18048cd35ca36 | d4053faa2c374045bd37d96ccb7f1b6c | [u'affinity'] | [] | {} |
+--------------------------------------+--------------+----------------------------------+----------------------------------+---------------+---------+----------+Step 2: Launch instances within the group created in step 1
# nova boot --flavor 1 --image cirros --security-groups default --nic net-id=9381abae-e0c5-4844-b605-da6b04ce9a36 --hint group=565948b0-076e-4de0-97dd-11d58862b64e --max 4 affinityVM
Lookout for –hint group option with Id of the group created in step 1. Also, the above command requests for 4 VMs using the –max option.
Step 3: Check if all the requested VMs are launched on the same host.
for i in `seq 1 4`
> do
> openstack server show affinityVM-${i} -c hostId -f shell
> done
hostid="1148bff316258766a721cef177307ce597d80b0e2d95b7603bbfb16d"
hostid="1148bff316258766a721cef177307ce597d80b0e2d95b7603bbfb16d"
hostid="1148bff316258766a721cef177307ce597d80b0e2d95b7603bbfb16d"
hostid="1148bff316258766a721cef177307ce597d80b0e2d95b7603bbfb16d"The above command says all the VMs were started on same host.
Let’s now work with Anti-Affinity Group policy.
How to use ServerGroupAntiAffinityFilter policy?
Using ServerGroupAntiAffinityFilter, the user can ensure that the VMs are started on different hypervisors. For e.g., consider the situation where you don’t want all of your services hosted on same machine, as the failure of host machine will affect all of your services. Using ServerGroupAntiAffinityFilter ensures that all of your services are disturbed in different physical hosts.
Step 1: Create group with Antiaffinity policy
# nova server-group-create anti-affinityHost anti-affinity
+--------------------------------------+-------------------+----------------------------------+----------------------------------+--------------------+---------+----------+
| Id | Name | Project Id | User Id | Policies | Members | Metadata |
+--------------------------------------+-------------------+----------------------------------+----------------------------------+--------------------+---------+----------+
| a1c8220a-275f-4ff1-8ce1-6fbb54f7cf46 | anti-affinityHost | 9563fa5bb6dd479aa9f18048cd35ca36 | d4053faa2c374045bd37d96ccb7f1b6c | [u'anti-affinity'] | [] | {} |
+--------------------------------------+-------------------+----------------------------------+----------------------------------+--------------------+---------+----------+Step 2: Launch instances within the anti-affinity group.
# nova boot --flavor 1 --image cirros --security-groups default --nic net-id=9381abae-e0c5-4844-b605-da6b04ce9a36 --hint group=a1c8220a-275f-4ff1-8ce1-6fbb54f7cf46 --max 3 anti-affinityVM
Lookout for –-hint group option with Id of the group created in step 1. Also, –max requests 3 VMs and all are expected to be launched on different hosts.
Step 3: Verify VMs are started on different hosts/hypervisors.
# for i in `seq 1 3`; do openstack server show anti-affinityVM-${i} -c hostId -f shell; done
hostid="09fac9c46a3dc5f6219e2af3e3a7da6dad7de843cfba7d8d4feb9054"
hostid="1148bff316258766a721cef177307ce597d80b0e2d95b7603bbfb16d"
hostid="4e488190b3d18a501dd719c051771636b6f73f6fd292f00e8622b185"The above command confirms that the all three VMs were started on different hosts.
In case, if there are no resources available to satisfy anti-affinity policy, then nova boot command will fail with the below error message:
# nova boot --flavor 1 --image cirros --security-groups default --nic net-id=9381abae-e0c5-4844-b605-da6b04ce9a36 --hint group=a1c8220a-275f-4ff1-8ce1-6fbb54f7cf46 anti-affinityVM
Sample output:
"No valid host was found. There are not enough hosts available."
As my setup has only three compute hosts, the above command is unable to find valid hosts for starting a new VM.
