Measurements

Introdcution

Motivation

The motivation for these tests, is to show how much more flexible the new API verb is but also to uncover potential `gotchas`. The section about the fake stats shows pretty good, that by not properly measuring the requested data. e.g. by blindly always requesting all changes, the usage of the API can cause more traffic on the wire than expected.

What was Measured

The measurements performed here have been using the body sizes of the data transmitted over the wire in response to the used vdsm API. This section of the document describes how the simulation of the engine was performed and which API calls have been measured.

All tests performed here are simulating a time frame of 15 minutes.

Description

list/getAllVMStats

Current represents how we are currently polling VDSM for the data of Virtual Machines. VDSM is polled every 3 seconds where there is called the ‘list’ API call and then every 5. time getAllVMStats is called. The list API call just sends the VM Ids and the status of the VM to the caller. ‘getAllVMStats’ contains various items, such as statistics, guest agent repoted data and various configuration and status fields.

The sequence currently looks like this:

1. getAllVMStats
2. list
3. list
4. list
5. list
6. getAllVMStats
7. list
8. …

queryVms

This test is using the proposed queryVms API call.

The test has been performed by calling 300 times queryVms and remembering the ‘queryStamp’ return value and passing it as the ‘changedSince’ parameter value with the next call. The first call did not have the queryStamp value therefore the full data set has been returned. This was done to simulate properly and fairly simulate the first 15 minutes of the engine querying the hosts.

queryVms 4 status/1 all

This test is using the proposed queryVms API call.

This test has been implemented to simulate the current way of querying VDSM on top of the new proposed API.

1. queryVms(changedSince=lastChanged)
2. queryVms(fields=’status’, changedSince=lastChanged)
3. queryVms(fields=’status’, changedSince=lastChanged)
4. queryVms(fields=’status’, changedSince=lastChanged)
5. queryVms(fields=’status’, changedSince=lastChanged)
6. queryVms(changedSince=lastChanged)
7. queryVms(fields=’status’, changedSince=lastChanged)
8. …

queryVms statistics every minute

This test is using the new proposed queryVms API call

This test assumes that certain statistics are only required every so often and for the sake of this test, the retrieval of the information has been scheduled to request this data statistics data every minute.

For this test the following fields have been categorized as statistics fields and are further refered to as `statsFields`

• disksUsage
• network
• disks
• memoryStats
• memUsage

Illustrative querying sequence:

• 1. queryVms(exclude=statsFields, changedSince=lastChanged) queryVms(fields=statsFields, changedSince=statsLastChanged)
• 1. queryVms(exclude=statsFields, changedSince=lastChanged)
• 1. queryVms(exclude=statsFields, changedSince=lastChanged)
• 1. …
• 1. queryVms(exclude=statsFields, changedSince=lastChanged)
• 1. queryVms(exclude=statsFields, changedSince=lastChanged) queryVms(fields=statsFields, changedSince=statsLastChanged)
• 1. queryVms(exclude=statsFields, changedSince=lastChanged)
• 1. …

queryVms statistics & status

This test is using the new proposed queryVms API call

This test has been implemented as a combination out of tests performed by `queryVms statistics every minute` and `queryVms 4 status/1 all`.

Illustrative querying sequence:

• 1. queryVms(exclude=statsFields, changedSince=lastChanged) queryVms(fields=statsFields, changedSince=statsLastChanged)
• 1. queryVms(fields=’status’, changedSince=lastChanged)
• 1. queryVms(fields=’status’, changedSince=lastChanged)
• 1. queryVms(fields=’status’, changedSince=lastChanged)
• 1. queryVms(fields=’status’, changedSince=lastChanged)
• 1. queryVms(exclude=statsFields, changedSince=lastChanged)
• 1. …
• 1. queryVms(fields=’status’, changedSince=lastChanged)
• 1. queryVms(exclude=statsFields, changedSince=lastChanged) queryVms(fields=statsFields, changedSince=statsLastChanged)
• 1. queryVms(fields=’status’, changedSince=lastChanged)
• 1. …

Note: The test here might have been optimized to NOT use separate calls when querying for the statistics since it overlaps with the general query, however for the sake of simplicity and potential difference on the engine side for the implementation this has been left this way.

Results

Idle VMs

A set of mixed (Windows/Linux) vms having no active users or any kind of services running. A third (33%) having of those VMs have guest agents.

6 VMs

100 VMs

1000 VMs

Fake Stats

A set of VMs where the statistics for ‘disksUsage’, ‘memUsage’, ‘cpuSys’, ‘cpuUser’, ‘network’ have been actively manipulated to be different on every request. (Basically what the fake vm stats hook does just made suitable for this test)

This test simulates heavy workload VMs.

6 VMs

100 VMs

1000 VMs

Windows VMs

Since the Windows virtual machines are producing more data reported by the guest agent due to the definition of the application list, as another sub test this simulates a fleet of Windows clones. Those Virtual Machines can be considered as a VM pool with 6 or 100 VMs. The VMs in this test have not been actively used, however actively reported their status and their the application list.

Please note that this is a test which is in comparison with the idle test, where ‘list/getAllVmStats’ would produce twice as much (even possibly even more data depending on what is installed on the windows machine) opposed to the current implementation of the application lists on Linux machines.

Just to give an idea about how significant this difference can be here a comparison of the query results:

6 VMs

100 VMs