Table of Contents

1. oVirt Upgrade Overview

This guide explains how to upgrade the following environments to oVirt 4.3:

  • Standalone manager, local database: Both the Data Warehouse database and the Engine database are installed on the Engine.

  • Standalone manager, remote database: Either the Data Warehouse database or the Engine database, or both, are on a separate machine.

  • Self-hosted engine, local database: Both the Data Warehouse database and the Engine database are installed on the Engine.

To save time when upgrading a Engine with local databases, use the ovirt-fast-forward-upgrade tool.

ovirt-fast-forward-upgrade supports local database environments only.

Alternatively, you can manually upgrade the Engine. For remote database environments, this path is required.

Plan any necessary downtime in advance. After you update the clusters' compatibility versions during the upgrade, a new hardware configuration is automatically applied to each virtual machine once it reboots. You must reboot any running or suspended VMs as soon as possible to apply the configuration changes.

Select the appropriate instructions for your environment from the following table. If your Engine and host versions differ (if you have previously upgraded the Engine but not the hosts), follow the instructions that match the Engine’s version.

Table 1. Supported Upgrade Paths
Current Engine version Target Engine version Relevant section

4.0

4.3

Local database environment:

Self-hosted engine, local database environment:

4.1

4.3

Local database environment:

Self-hosted engine, local database environment:

4.2

4.3

Local database environment:

Self-hosted engine, local database environment:

4.3.x

4.3.y

Fast-forward Upgrading a Local Database Environment

This section applies to both self-hosted engine and standalone Engine environments.

2. ovirt-fast-forward-upgrade Explained

You can upgrade the Engine to the latest version with the ovirt-fast-forward-upgrade tool. ovirt-fast-forward-upgrade detects the current version of the Engine and checks for available upgrades. If an upgrade is available, the tool upgrades the Engine to the next major version, and continues to upgrade the Engine until the latest version is installed.

During the upgrade process, the tool might pause and prompt you to manually complete some upgrade tasks, such as upgrading hosts or the cluster compatibility version. You can leave ovirt-fast-forward-upgrade paused while you complete these tasks.

ovirt-fast-forward-upgrade supports the following upgrade paths:

  • Upgrading a standalone Engine with local databases

  • Upgrading a self-hosted engine environment using local databases

ovirt-fast-forward-upgrade upgrades the Engine. You need to manually upgrade the hosts, and compatibility versions of clusters and data centers, and you need to reboot any running virtual machines. See Updating All Hosts in a Cluster to upgrade the hosts.

ovirt-fast-forward-upgrade does not support oVirt environments deployed and managed with Satellite.

3. Fast-forwarding oVirt from 4.0 to 4.3

oVirt 4.3 and later does not support 4.0 cluster compatibility version or data center compatibility version. So when you upgrade from oVirt 4.0, you must update the cluster and data center compatibility versions to at least 4.1 before you upgrade the Engine from 4.2 to 4.3. Then, if you want to use the 4.3 compatibility version, update the compatibility versions again after completing the Engine upgrades.

You must also update the hosts before updating the compatibility versions, but only need to do so once. The host repositories stay the same across oVirt versions, so the hosts will already be upgraded to the latest version after a single update.

Upgrading your environment from 4.0 to 4.3 involves the following steps:

3.1. Prerequisites

  • Plan for any necessary virtual machine downtime. After you update the clusters' compatibility versions during the upgrade, a new hardware configuration is automatically applied to each virtual machine once it reboots. You must reboot any running or suspended virtual machines as soon as possible to apply the configuration changes.

3.2. Upgrading to oVirt Engine 4.3 with ovirt-fast-forward-upgrade

Procedure
  1. Log in to the Engine machine and install the ovirt-fast-forward-upgrade tool:

    # yum install ovirt-fast-forward-upgrade
  2. Run the following command to upgrade the Engine, while creating a backup of the current version:

    # ovirt-fast-forward-upgrade --backup --backup-dir=/backup

    oVirt recommends using the --backup and --backup-dir options to create a backup of the current Engine. If a backup directory is not specified, the backup is saved in /tmp.

    The --backup option is a wrapper for the engine-backup tool and is equivalent to running the following command:

    # engine-backup --scope=all --mode=backup --file=file_name --log=log_file_name

    To restore your backup, run engine-backup in restore mode:

    # engine-backup --mode=restore

    See Backing Up and Restoring the oVirt Engine in the Administration Guide for details.

    Alternatively, to upgrade without creating a backup, run the following command:

    # ovirt-fast-forward-upgrade
  3. If there are errors, check the log: /var/log/ovirt-engine/ovirt-fast-forward-upgrade.log.

The tool enables the proper repositories and upgrades the Engine to 4.2.

If the tool detects that the cluster compatibility version is set to 3.6 or 4.0, the tool pauses and prompts you to manually upgrade the cluster compatibility version to at least 4.1.

If you try to continue without upgrading the cluster compatibility version to at least 4.1, ovirt-fast-forward-upgrade aborts.

You must now update the hosts before you can update the cluster and data center compatibility versions.

3.3. Updating Individual Hosts

Use the host upgrade manager to update individual hosts directly from the Administration Portal.

The upgrade manager only checks hosts with a status of Up or Non-operational, but not Maintenance.

Limitations
  • On oVirt Node, the update only preserves modified content in the /etc and /var directories. Modified data in other paths is overwritten during an update.

  • If migration is enabled at the cluster level, virtual machines are automatically migrated to another host in the cluster. Update a host when its usage is relatively low.

  • In a self-hosted engine environment, the Engine virtual machine can only migrate between self-hosted engine nodes in the same cluster. It cannot migrate to standard hosts.

  • The cluster must have sufficient memory reserved for its hosts to perform maintenance. Otherwise, virtual machine migrations will hang and fail. You can reduce the memory usage of host updates by shutting down some or all virtual machines before updating hosts.

  • Do not update all hosts at the same time, as one host must remain available to perform Storage Pool Manager (SPM) tasks.

  • You cannot migrate a pinned virtual machine (such as a virtual machine using a vGPU) to another host. Pinned virtual machines must be shut down before updating the host.

Procedure
  1. Ensure that the correct repositories are enabled. To view a list of currently enabled repositories, run yum repolist.

  2. In the Administration Portal, click Compute  Hosts and select the host to be updated.

  3. Click Installation  Check for Upgrade and click OK.

    Open the Notification Drawer (EventsIcon) and expand the Events section to see the result.

  4. If an update is available, click Installation  Upgrade.

  5. Click OK to update the host. Running virtual machines are migrated according to their migration policy. If migration is disabled for any virtual machines, you are prompted to shut them down.

    The details of the host are updated in Compute  Hosts and the status transitions through these stages:

    Maintenance > Installing > Reboot > Up

    If the update fails, the host’s status changes to Install Failed. From Install Failed you can click Installation  Upgrade again.

Repeat this procedure for each host in the oVirt environment.

You can now change the cluster compatibility version to 4.2.

3.4. Changing the Cluster Compatibility Version

oVirt clusters have a compatibility version. The cluster compatibility version indicates the features of oVirt supported by all of the hosts in the cluster. The cluster compatibility is set according to the version of the least capable host operating system in the cluster.

To change the cluster compatibility version, you must have first updated all the hosts in your cluster to a level that supports your desired compatibility level. Check if there is an icon next to the host indicating an update is available.

Procedure
  1. In the Administration Portal, click Compute  Clusters.

  2. Select the cluster to change and click Edit.

  3. On the General tab, change the Compatibility Version to the desired value.

  4. Click OK. The Change Cluster Compatibility Version confirmation dialog opens.

  5. Click OK to confirm.

An error message might warn that some virtual machines and templates are incorrectly configured. To fix this error, edit each virtual machine manually. The Edit Virtual Machine window provides additional validations and warnings that show what to correct. Sometimes the issue is automatically corrected and the virtual machine’s configuration just needs to be saved again. After editing each virtual machine, you will be able to change the cluster compatibility version.

3.5. Changing Virtual Machine Cluster Compatibility

After updating a cluster’s compatibility version, you must update the cluster compatibility version of all running or suspended virtual machines by rebooting them from the Administration Portal, or using the REST API, instead of from within the guest operating system. Virtual machines that require a reboot are marked with the pending changes icon (pendingchanges).

Although you can wait to reboot the virtual machines at a convenient time, rebooting immediately is highly recommended so that the virtual machines use the latest configuration. Any virtual machine that has not been rebooted runs with the previous configuration, and subsequent configuration changes made to the virtual machine might overwrite its pending cluster compatibility changes.

Procedure
  1. In the Administration Portal, click Compute  Virtual Machines.

  2. Check which virtual machines require a reboot. In the Vms: search bar, enter the following query:

    next_run_config_exists=True

    The search results show all virtual machines with pending changes.

  3. Select each virtual machine and click Reboot.

When the virtual machine starts, the new compatibility version is automatically applied.

You cannot change the cluster compatibility version of a virtual machine snapshot that is in preview. You must first commit or undo the preview.

You can now change the data center compatibility version to 4.2.

3.6. Changing the Data Center Compatibility Version

oVirt data centers have a compatibility version. The compatibility version indicates the version of oVirt with which the data center is intended to be compatible. All clusters in the data center must support the desired compatibility level.

To change the data center compatibility version, you must have first updated the compatibility version of all clusters and virtual machines in the data center.

Procedure
  1. In the Administration Portal, click Compute  Data Centers.

  2. Select the data center to change and click Edit.

  3. Change the Compatibility Version to the desired value.

  4. Click OK. The Change Data Center Compatibility Version confirmation dialog opens.

  5. Click OK to confirm.

3.7. Continue Upgrading the Engine to 4.3

Return to ovirt-fast-forward-upgrade and continue the process. The tool upgrades the Engine to 4.3.

You can now change the cluster compatibility version to 4.3.

3.8. Changing the Cluster Compatibility Version

oVirt clusters have a compatibility version. The cluster compatibility version indicates the features of oVirt supported by all of the hosts in the cluster. The cluster compatibility is set according to the version of the least capable host operating system in the cluster.

To change the cluster compatibility version, you must have first updated all the hosts in your cluster to a level that supports your desired compatibility level. Check if there is an icon next to the host indicating an update is available.

Procedure
  1. In the Administration Portal, click Compute  Clusters.

  2. Select the cluster to change and click Edit.

  3. On the General tab, change the Compatibility Version to the desired value.

  4. Click OK. The Change Cluster Compatibility Version confirmation dialog opens.

  5. Click OK to confirm.

An error message might warn that some virtual machines and templates are incorrectly configured. To fix this error, edit each virtual machine manually. The Edit Virtual Machine window provides additional validations and warnings that show what to correct. Sometimes the issue is automatically corrected and the virtual machine’s configuration just needs to be saved again. After editing each virtual machine, you will be able to change the cluster compatibility version.

3.9. Changing Virtual Machine Cluster Compatibility

After updating a cluster’s compatibility version, you must update the cluster compatibility version of all running or suspended virtual machines by rebooting them from the Administration Portal, or using the REST API, instead of from within the guest operating system. Virtual machines that require a reboot are marked with the pending changes icon (pendingchanges).

Although you can wait to reboot the virtual machines at a convenient time, rebooting immediately is highly recommended so that the virtual machines use the latest configuration. Any virtual machine that has not been rebooted runs with the previous configuration, and subsequent configuration changes made to the virtual machine might overwrite its pending cluster compatibility changes.

Procedure
  1. In the Administration Portal, click Compute  Virtual Machines.

  2. Check which virtual machines require a reboot. In the Vms: search bar, enter the following query:

    next_run_config_exists=True

    The search results show all virtual machines with pending changes.

  3. Select each virtual machine and click Reboot.

When the virtual machine starts, the new compatibility version is automatically applied.

You cannot change the cluster compatibility version of a virtual machine snapshot that is in preview. You must first commit or undo the preview.

You can now change the data center compatibility version to 4.3.

3.10. Changing the Data Center Compatibility Version

oVirt data centers have a compatibility version. The compatibility version indicates the version of oVirt with which the data center is intended to be compatible. All clusters in the data center must support the desired compatibility level.

To change the data center compatibility version, you must have first updated the compatibility version of all clusters and virtual machines in the data center.

Procedure
  1. In the Administration Portal, click Compute  Data Centers.

  2. Select the data center to change and click Edit.

  3. Change the Compatibility Version to the desired value.

  4. Click OK. The Change Data Center Compatibility Version confirmation dialog opens.

  5. Click OK to confirm.

3.11. Replacing SHA-1 Certificates with SHA-256 Certificates

oVirt 4.3 uses SHA-256 signatures, which provide a more secure way to sign SSL certificates than SHA-1. Newly installed systems do not require any special steps to enable oVirt’s public key infrastructure (PKI) to use SHA-256 signatures. However, for systems upgraded from 4.1 or earlier, one of the following is required:

Preventing Warning Messages from Appearing in the Browser

  1. Log in to the Engine machine as the root user.

  2. Check whether /etc/pki/ovirt-engine/openssl.conf includes the line default_md = sha256:

    # cat /etc/pki/ovirt-engine/openssl.conf

    If it still includes default_md = sha1, back up the existing configuration and change the default to sha256:

    # cp -p /etc/pki/ovirt-engine/openssl.conf /etc/pki/ovirt-engine/openssl.conf."$(date +"%Y%m%d%H%M%S")"
    # sed -i 's/^default_md = sha1/default_md = sha256/' /etc/pki/ovirt-engine/openssl.conf
  3. Define the certificate that should be re-signed:

    # names="apache"
  4. On the Engine, re-sign the Apache certificate:

    for name in $names; do
        subject="$(
            openssl \
                x509 \
                -in /etc/pki/ovirt-engine/certs/"${name}".cer \
                -noout \
                -subject \
            | sed \
                's;subject= \(.*\);\1;' \
        )"
       /usr/share/ovirt-engine/bin/pki-enroll-pkcs12.sh \
            --name="${name}" \
            --password=mypass \
            --subject="${subject}" \
            --san=DNS:"${ENGINE_FQDN}" \
            --keep-key
    done
  5. Restart the httpd service:

    # systemctl restart httpd
  6. Connect to the Administration Portal to confirm that the warning no longer appears.

  7. If you previously imported a CA or https certificate into the browser, find the certificate(s), remove them from the browser, and reimport the new CA certificate. Install the certificate authority according to the instructions provided by your browser. To get the certificate authority’s certificate, navigate to http://your-manager-fqdn/ovirt-engine/services/pki-resource?resource=ca-certificate&format=X509-PEM-CA, replacing your-manager-fqdn with the fully qualified domain name (FQDN).

Replacing All Signed Certificates with SHA-256

  1. Log in to the Engine machine as the root user.

  2. Check whether /etc/pki/ovirt-engine/openssl.conf includes the line default_md = sha256:

    # cat /etc/pki/ovirt-engine/openssl.conf

    If it still includes default_md = sha1, back up the existing configuration and change the default to sha256:

    # cp -p /etc/pki/ovirt-engine/openssl.conf /etc/pki/ovirt-engine/openssl.conf."$(date +"%Y%m%d%H%M%S")"
    # sed -i 's/^default_md = sha1/default_md = sha256/' /etc/pki/ovirt-engine/openssl.conf
  3. Re-sign the CA certificate by backing it up and creating a new certificate in ca.pem.new:

    # cp -p /etc/pki/ovirt-engine/private/ca.pem /etc/pki/ovirt-engine/private/ca.pem."$(date +"%Y%m%d%H%M%S")"
    # openssl x509 -signkey /etc/pki/ovirt-engine/private/ca.pem -in /etc/pki/ovirt-engine/ca.pem -out /etc/pki/ovirt-engine/ca.pem.new -days 3650 -sha256
  4. Replace the existing certificate with the new certificate:

    # mv /etc/pki/ovirt-engine/ca.pem.new /etc/pki/ovirt-engine/ca.pem
  5. Define the certificates that should be re-signed:

    # names="engine apache websocket-proxy jboss imageio-proxy"

    If you replaced the oVirt Engine SSL Certificate after the upgrade, run the following instead:

    # names="engine websocket-proxy jboss imageio-proxy"

    For more details see Replacing the oVirt Engine CA Certificate in the Administration Guide.

  6. On the Engine, re-sign the certificates:

    for name in $names; do
       subject="$(
            openssl \
                x509 \
                -in /etc/pki/ovirt-engine/certs/"${name}".cer \
                -noout \
                -subject \
            | sed \
                's;subject= \(.*\);\1;' \
            )"
         /usr/share/ovirt-engine/bin/pki-enroll-pkcs12.sh \
                --name="${name}" \
                --password=mypass \
                --subject="${subject}" \
                --san=DNS:"${ENGINE_FQDN}" \
                --keep-key
    done
  7. Restart the following services:

    # systemctl restart httpd
    # systemctl restart ovirt-engine
    # systemctl restart ovirt-websocket-proxy
    # systemctl restart ovirt-imageio-proxy
  8. Connect to the Administration Portal to confirm that the warning no longer appears.

  9. If you previously imported a CA or https certificate into the browser, find the certificate(s), remove them from the browser, and reimport the new CA certificate. Install the certificate authority according to the instructions provided by your browser. To get the certificate authority’s certificate, navigate to http://your-manager-fqdn/ovirt-engine/services/pki-resource?resource=ca-certificate&format=X509-PEM-CA, replacing your-manager-fqdn with the fully qualified domain name (FQDN).

  10. Enroll the certificates on the hosts. Repeat the following procedure for each host.

    1. In the Administration Portal, click Compute  Hosts.

    2. Select the host and click Management  Maintenance.

    3. Once the host is in maintenance mode, click Installation  Enroll Certificate.

    4. Click Management  Activate.

4. Fast-forwarding oVirt from 4.1 to 4.3

Upgrading your environment from 4.1 to 4.3 involves the following steps:

4.1. Prerequisites

  • Plan for any necessary virtual machine downtime. After you update the clusters' compatibility versions during the upgrade, a new hardware configuration is automatically applied to each virtual machine once it reboots. You must reboot any running or suspended virtual machines as soon as possible to apply the configuration changes.

4.2. Upgrading to oVirt Engine 4.3 with ovirt-fast-forward-upgrade

Procedure
  1. Log in to the Engine machine and install the ovirt-fast-forward-upgrade tool:

    # yum install ovirt-fast-forward-upgrade
  2. Run the following command to upgrade the Engine, while creating a backup of the current version:

    # ovirt-fast-forward-upgrade --backup --backup-dir=/backup

    oVirt recommends using the --backup and --backup-dir options to create a backup of the current Engine. If a backup directory is not specified, the backup is saved in /tmp.

    The --backup option is a wrapper for the engine-backup tool and is equivalent to running the following command:

    # engine-backup --scope=all --mode=backup --file=file_name --log=log_file_name

    To restore your backup, run engine-backup in restore mode:

    # engine-backup --mode=restore

    See Backing Up and Restoring the oVirt Engine in the Administration Guide for details.

    Alternatively, to upgrade without creating a backup, run the following command:

    # ovirt-fast-forward-upgrade
  3. If there are errors, check the log: /var/log/ovirt-engine/ovirt-fast-forward-upgrade.log.

You can now update the hosts.

4.3. Updating All Hosts in a Cluster

You can update all hosts in a cluster instead of updating hosts individually. This is particularly useful during upgrades to new versions of oVirt. See https://github.com/oVirt/ovirt-ansible-cluster-upgrade/blob/master/README.md for more information about the Ansible role used to automate the updates.

oVirt recommends updating one cluster at a time.

Limitations
  • On oVirt Node, the update only preserves modified content in the /etc and /var directories. Modified data in other paths is overwritten during an update.

  • If migration is enabled at the cluster level, virtual machines are automatically migrated to another host in the cluster.

  • In a self-hosted engine environment, the Engine virtual machine can only migrate between self-hosted engine nodes in the same cluster. It cannot migrate to standard hosts.

  • The cluster must have sufficient memory reserved for its hosts to perform maintenance. Otherwise, virtual machine migrations will hang and fail. You can reduce the memory usage of host updates by shutting down some or all virtual machines before updating hosts.

  • You cannot migrate a pinned virtual machine (such as a virtual machine using a vGPU) to another host. Pinned virtual machines are shut down during the update, unless you choose to skip that host instead.

Procedure
  1. In the Administration Portal, click Compute  Clusters and select the cluster.

  2. Click Upgrade.

  3. Select the hosts to update, then click Next.

  4. Configure the options:

    • Stop Pinned VMs shuts down any virtual machines that are pinned to hosts in the cluster, and is selected by default. You can clear this check box to skip updating those hosts so that the pinned virtual machines stay running, such as when a pinned virtual machine is running important services or processes and you do not want it to shut down at an unknown time during the update.

    • Upgrade Timeout (Minutes) sets the time to wait for an individual host to be updated before the cluster upgrade fails with a timeout. The default is 60. You can increase it for large clusters where 60 minutes might not be enough, or reduce it for small clusters where the hosts update quickly.

    • Check Upgrade checks each host for available updates before running the upgrade process. It is not selected by default, but you can select it if you need to ensure that recent updates are included, such as when you have configured the Engine to check for host updates less frequently than the default.

    • Reboot After Upgrade reboots each host after it is updated, and is selected by default. You can clear this check box to speed up the process if you are sure that there are no pending updates that require a host reboot.

    • Use Maintenance Policy sets the cluster’s scheduling policy to cluster_maintenance during the update. It is selected by default, so activity is limited and virtual machines cannot start unless they are highly available. You can clear this check box if you have a custom scheduling policy that you want to keep using during the update, but this could have unknown consequences. Ensure your custom policy is compatible with cluster upgrade activity before disabling this option.

  5. Click Next.

  6. Review the summary of the hosts and virtual machines that will be affected.

  7. Click Upgrade.

You can track the progress of host updates in the Compute  Hosts view, and in the Events section of the Notification Drawer (EventsIcon).

You can track the progress of individual virtual machine migrations in the Status column of the Compute  Virtual Machines view. In large environments, you may need to filter the results to show a particular group of virtual machines.

4.4. Changing the Cluster Compatibility Version

oVirt clusters have a compatibility version. The cluster compatibility version indicates the features of oVirt supported by all of the hosts in the cluster. The cluster compatibility is set according to the version of the least capable host operating system in the cluster.

To change the cluster compatibility version, you must have first updated all the hosts in your cluster to a level that supports your desired compatibility level. Check if there is an icon next to the host indicating an update is available.

Procedure
  1. In the Administration Portal, click Compute  Clusters.

  2. Select the cluster to change and click Edit.

  3. On the General tab, change the Compatibility Version to the desired value.

  4. Click OK. The Change Cluster Compatibility Version confirmation dialog opens.

  5. Click OK to confirm.

An error message might warn that some virtual machines and templates are incorrectly configured. To fix this error, edit each virtual machine manually. The Edit Virtual Machine window provides additional validations and warnings that show what to correct. Sometimes the issue is automatically corrected and the virtual machine’s configuration just needs to be saved again. After editing each virtual machine, you will be able to change the cluster compatibility version.

4.5. Changing Virtual Machine Cluster Compatibility

After updating a cluster’s compatibility version, you must update the cluster compatibility version of all running or suspended virtual machines by rebooting them from the Administration Portal, or using the REST API, instead of from within the guest operating system. Virtual machines that require a reboot are marked with the pending changes icon (pendingchanges).

Although you can wait to reboot the virtual machines at a convenient time, rebooting immediately is highly recommended so that the virtual machines use the latest configuration. Any virtual machine that has not been rebooted runs with the previous configuration, and subsequent configuration changes made to the virtual machine might overwrite its pending cluster compatibility changes.

Procedure
  1. In the Administration Portal, click Compute  Virtual Machines.

  2. Check which virtual machines require a reboot. In the Vms: search bar, enter the following query:

    next_run_config_exists=True

    The search results show all virtual machines with pending changes.

  3. Select each virtual machine and click Reboot.

When the virtual machine starts, the new compatibility version is automatically applied.

You cannot change the cluster compatibility version of a virtual machine snapshot that is in preview. You must first commit or undo the preview.

4.6. Changing the Data Center Compatibility Version

oVirt data centers have a compatibility version. The compatibility version indicates the version of oVirt with which the data center is intended to be compatible. All clusters in the data center must support the desired compatibility level.

To change the data center compatibility version, you must have first updated the compatibility version of all clusters and virtual machines in the data center.

Procedure
  1. In the Administration Portal, click Compute  Data Centers.

  2. Select the data center to change and click Edit.

  3. Change the Compatibility Version to the desired value.

  4. Click OK. The Change Data Center Compatibility Version confirmation dialog opens.

  5. Click OK to confirm.

4.7. Replacing SHA-1 Certificates with SHA-256 Certificates

oVirt 4.3 uses SHA-256 signatures, which provide a more secure way to sign SSL certificates than SHA-1. Newly installed systems do not require any special steps to enable oVirt’s public key infrastructure (PKI) to use SHA-256 signatures. However, for systems upgraded from 4.1 or earlier, one of the following is required:

Preventing Warning Messages from Appearing in the Browser

  1. Log in to the Engine machine as the root user.

  2. Check whether /etc/pki/ovirt-engine/openssl.conf includes the line default_md = sha256:

    # cat /etc/pki/ovirt-engine/openssl.conf

    If it still includes default_md = sha1, back up the existing configuration and change the default to sha256:

    # cp -p /etc/pki/ovirt-engine/openssl.conf /etc/pki/ovirt-engine/openssl.conf."$(date +"%Y%m%d%H%M%S")"
    # sed -i 's/^default_md = sha1/default_md = sha256/' /etc/pki/ovirt-engine/openssl.conf
  3. Define the certificate that should be re-signed:

    # names="apache"
  4. On the Engine, re-sign the Apache certificate:

    for name in $names; do
        subject="$(
            openssl \
                x509 \
                -in /etc/pki/ovirt-engine/certs/"${name}".cer \
                -noout \
                -subject \
            | sed \
                's;subject= \(.*\);\1;' \
        )"
       /usr/share/ovirt-engine/bin/pki-enroll-pkcs12.sh \
            --name="${name}" \
            --password=mypass \
            --subject="${subject}" \
            --san=DNS:"${ENGINE_FQDN}" \
            --keep-key
    done
  5. Restart the httpd service:

    # systemctl restart httpd
  6. Connect to the Administration Portal to confirm that the warning no longer appears.

  7. If you previously imported a CA or https certificate into the browser, find the certificate(s), remove them from the browser, and reimport the new CA certificate. Install the certificate authority according to the instructions provided by your browser. To get the certificate authority’s certificate, navigate to http://your-manager-fqdn/ovirt-engine/services/pki-resource?resource=ca-certificate&format=X509-PEM-CA, replacing your-manager-fqdn with the fully qualified domain name (FQDN).

Replacing All Signed Certificates with SHA-256

  1. Log in to the Engine machine as the root user.

  2. Check whether /etc/pki/ovirt-engine/openssl.conf includes the line default_md = sha256:

    # cat /etc/pki/ovirt-engine/openssl.conf

    If it still includes default_md = sha1, back up the existing configuration and change the default to sha256:

    # cp -p /etc/pki/ovirt-engine/openssl.conf /etc/pki/ovirt-engine/openssl.conf."$(date +"%Y%m%d%H%M%S")"
    # sed -i 's/^default_md = sha1/default_md = sha256/' /etc/pki/ovirt-engine/openssl.conf
  3. Re-sign the CA certificate by backing it up and creating a new certificate in ca.pem.new:

    # cp -p /etc/pki/ovirt-engine/private/ca.pem /etc/pki/ovirt-engine/private/ca.pem."$(date +"%Y%m%d%H%M%S")"
    # openssl x509 -signkey /etc/pki/ovirt-engine/private/ca.pem -in /etc/pki/ovirt-engine/ca.pem -out /etc/pki/ovirt-engine/ca.pem.new -days 3650 -sha256
  4. Replace the existing certificate with the new certificate:

    # mv /etc/pki/ovirt-engine/ca.pem.new /etc/pki/ovirt-engine/ca.pem
  5. Define the certificates that should be re-signed:

    # names="engine apache websocket-proxy jboss imageio-proxy"

    If you replaced the oVirt Engine SSL Certificate after the upgrade, run the following instead:

    # names="engine websocket-proxy jboss imageio-proxy"

    For more details see Replacing the oVirt Engine CA Certificate in the Administration Guide.

  6. On the Engine, re-sign the certificates:

    for name in $names; do
       subject="$(
            openssl \
                x509 \
                -in /etc/pki/ovirt-engine/certs/"${name}".cer \
                -noout \
                -subject \
            | sed \
                's;subject= \(.*\);\1;' \
            )"
         /usr/share/ovirt-engine/bin/pki-enroll-pkcs12.sh \
                --name="${name}" \
                --password=mypass \
                --subject="${subject}" \
                --san=DNS:"${ENGINE_FQDN}" \
                --keep-key
    done
  7. Restart the following services:

    # systemctl restart httpd
    # systemctl restart ovirt-engine
    # systemctl restart ovirt-websocket-proxy
    # systemctl restart ovirt-imageio-proxy
  8. Connect to the Administration Portal to confirm that the warning no longer appears.

  9. If you previously imported a CA or https certificate into the browser, find the certificate(s), remove them from the browser, and reimport the new CA certificate. Install the certificate authority according to the instructions provided by your browser. To get the certificate authority’s certificate, navigate to http://your-manager-fqdn/ovirt-engine/services/pki-resource?resource=ca-certificate&format=X509-PEM-CA, replacing your-manager-fqdn with the fully qualified domain name (FQDN).

  10. Enroll the certificates on the hosts. Repeat the following procedure for each host.

    1. In the Administration Portal, click Compute  Hosts.

    2. Select the host and click Management  Maintenance.

    3. Once the host is in maintenance mode, click Installation  Enroll Certificate.

    4. Click Management  Activate.

4.8. Updating OVN Providers Installed in oVirt 4.1

If you installed an Open Virtual Network (OVN) provider in oVirt 4.1, you must manually edit its configuration for oVirt 4.2.

Procedure
  1. Click Administration  Providers and select the OVN provider.

  2. Click Edit.

  3. Click the Networking Plugin text field and select oVirt Network Provider for OVN from the drop-down list.

  4. Click OK.

5. Fast-forwarding oVirt from 4.2 to 4.3

Upgrading your environment from 4.2 to 4.3 involves the following steps:

5.1. Prerequisites

  • Plan for any necessary virtual machine downtime. After you update the clusters' compatibility versions during the upgrade, a new hardware configuration is automatically applied to each virtual machine once it reboots. You must reboot any running or suspended virtual machines as soon as possible to apply the configuration changes.

  • Ensure the Engine has the correct repositories enabled.

5.2. Upgrading to oVirt Engine 4.3 with ovirt-fast-forward-upgrade

Procedure
  1. Log in to the Engine machine and install the ovirt-fast-forward-upgrade tool:

    # yum install ovirt-fast-forward-upgrade
  2. Run the following command to upgrade the Engine, while creating a backup of the current version:

    # ovirt-fast-forward-upgrade --backup --backup-dir=/backup

    oVirt recommends using the --backup and --backup-dir options to create a backup of the current Engine. If a backup directory is not specified, the backup is saved in /tmp.

    The --backup option is a wrapper for the engine-backup tool and is equivalent to running the following command:

    # engine-backup --scope=all --mode=backup --file=file_name --log=log_file_name

    To restore your backup, run engine-backup in restore mode:

    # engine-backup --mode=restore

    See Backing Up and Restoring the oVirt Engine in the Administration Guide for details.

    Alternatively, to upgrade without creating a backup, run the following command:

    # ovirt-fast-forward-upgrade
  3. If there are errors, check the log: /var/log/ovirt-engine/ovirt-fast-forward-upgrade.log.

You can now update the hosts.

5.3. Updating All Hosts in a Cluster

You can update all hosts in a cluster instead of updating hosts individually. This is particularly useful during upgrades to new versions of oVirt. See https://github.com/oVirt/ovirt-ansible-cluster-upgrade/blob/master/README.md for more information about the Ansible role used to automate the updates.

oVirt recommends updating one cluster at a time.

Limitations
  • On oVirt Node, the update only preserves modified content in the /etc and /var directories. Modified data in other paths is overwritten during an update.

  • If migration is enabled at the cluster level, virtual machines are automatically migrated to another host in the cluster.

  • In a self-hosted engine environment, the Engine virtual machine can only migrate between self-hosted engine nodes in the same cluster. It cannot migrate to standard hosts.

  • The cluster must have sufficient memory reserved for its hosts to perform maintenance. Otherwise, virtual machine migrations will hang and fail. You can reduce the memory usage of host updates by shutting down some or all virtual machines before updating hosts.

  • You cannot migrate a pinned virtual machine (such as a virtual machine using a vGPU) to another host. Pinned virtual machines are shut down during the update, unless you choose to skip that host instead.

Procedure
  1. In the Administration Portal, click Compute  Clusters and select the cluster.

  2. Click Upgrade.

  3. Select the hosts to update, then click Next.

  4. Configure the options:

    • Stop Pinned VMs shuts down any virtual machines that are pinned to hosts in the cluster, and is selected by default. You can clear this check box to skip updating those hosts so that the pinned virtual machines stay running, such as when a pinned virtual machine is running important services or processes and you do not want it to shut down at an unknown time during the update.

    • Upgrade Timeout (Minutes) sets the time to wait for an individual host to be updated before the cluster upgrade fails with a timeout. The default is 60. You can increase it for large clusters where 60 minutes might not be enough, or reduce it for small clusters where the hosts update quickly.

    • Check Upgrade checks each host for available updates before running the upgrade process. It is not selected by default, but you can select it if you need to ensure that recent updates are included, such as when you have configured the Engine to check for host updates less frequently than the default.

    • Reboot After Upgrade reboots each host after it is updated, and is selected by default. You can clear this check box to speed up the process if you are sure that there are no pending updates that require a host reboot.

    • Use Maintenance Policy sets the cluster’s scheduling policy to cluster_maintenance during the update. It is selected by default, so activity is limited and virtual machines cannot start unless they are highly available. You can clear this check box if you have a custom scheduling policy that you want to keep using during the update, but this could have unknown consequences. Ensure your custom policy is compatible with cluster upgrade activity before disabling this option.

  5. Click Next.

  6. Review the summary of the hosts and virtual machines that will be affected.

  7. Click Upgrade.

You can track the progress of host updates in the Compute  Hosts view, and in the Events section of the Notification Drawer (EventsIcon).

You can track the progress of individual virtual machine migrations in the Status column of the Compute  Virtual Machines view. In large environments, you may need to filter the results to show a particular group of virtual machines.

5.4. Changing the Cluster Compatibility Version

oVirt clusters have a compatibility version. The cluster compatibility version indicates the features of oVirt supported by all of the hosts in the cluster. The cluster compatibility is set according to the version of the least capable host operating system in the cluster.

To change the cluster compatibility version, you must have first updated all the hosts in your cluster to a level that supports your desired compatibility level. Check if there is an icon next to the host indicating an update is available.

Procedure
  1. In the Administration Portal, click Compute  Clusters.

  2. Select the cluster to change and click Edit.

  3. On the General tab, change the Compatibility Version to the desired value.

  4. Click OK. The Change Cluster Compatibility Version confirmation dialog opens.

  5. Click OK to confirm.

An error message might warn that some virtual machines and templates are incorrectly configured. To fix this error, edit each virtual machine manually. The Edit Virtual Machine window provides additional validations and warnings that show what to correct. Sometimes the issue is automatically corrected and the virtual machine’s configuration just needs to be saved again. After editing each virtual machine, you will be able to change the cluster compatibility version.

5.5. Changing Virtual Machine Cluster Compatibility

After updating a cluster’s compatibility version, you must update the cluster compatibility version of all running or suspended virtual machines by rebooting them from the Administration Portal, or using the REST API, instead of from within the guest operating system. Virtual machines that require a reboot are marked with the pending changes icon (pendingchanges).

Although you can wait to reboot the virtual machines at a convenient time, rebooting immediately is highly recommended so that the virtual machines use the latest configuration. Any virtual machine that has not been rebooted runs with the previous configuration, and subsequent configuration changes made to the virtual machine might overwrite its pending cluster compatibility changes.

Procedure
  1. In the Administration Portal, click Compute  Virtual Machines.

  2. Check which virtual machines require a reboot. In the Vms: search bar, enter the following query:

    next_run_config_exists=True

    The search results show all virtual machines with pending changes.

  3. Select each virtual machine and click Reboot.

When the virtual machine starts, the new compatibility version is automatically applied.

You cannot change the cluster compatibility version of a virtual machine snapshot that is in preview. You must first commit or undo the preview.

5.6. Changing the Data Center Compatibility Version

oVirt data centers have a compatibility version. The compatibility version indicates the version of oVirt with which the data center is intended to be compatible. All clusters in the data center must support the desired compatibility level.

To change the data center compatibility version, you must have first updated the compatibility version of all clusters and virtual machines in the data center.

Procedure
  1. In the Administration Portal, click Compute  Data Centers.

  2. Select the data center to change and click Edit.

  3. Change the Compatibility Version to the desired value.

  4. Click OK. The Change Data Center Compatibility Version confirmation dialog opens.

  5. Click OK to confirm.

If you previously upgraded to 4.2 without replacing SHA-1 certificates with SHA-256 certificates, you must do so now.

5.7. Replacing SHA-1 Certificates with SHA-256 Certificates

oVirt 4.3 uses SHA-256 signatures, which provide a more secure way to sign SSL certificates than SHA-1. Newly installed systems do not require any special steps to enable oVirt’s public key infrastructure (PKI) to use SHA-256 signatures. However, for systems upgraded from 4.1 or earlier, one of the following is required:

Preventing Warning Messages from Appearing in the Browser

  1. Log in to the Engine machine as the root user.

  2. Check whether /etc/pki/ovirt-engine/openssl.conf includes the line default_md = sha256:

    # cat /etc/pki/ovirt-engine/openssl.conf

    If it still includes default_md = sha1, back up the existing configuration and change the default to sha256:

    # cp -p /etc/pki/ovirt-engine/openssl.conf /etc/pki/ovirt-engine/openssl.conf."$(date +"%Y%m%d%H%M%S")"
    # sed -i 's/^default_md = sha1/default_md = sha256/' /etc/pki/ovirt-engine/openssl.conf
  3. Define the certificate that should be re-signed:

    # names="apache"
  4. On the Engine, re-sign the Apache certificate:

    for name in $names; do
        subject="$(
            openssl \
                x509 \
                -in /etc/pki/ovirt-engine/certs/"${name}".cer \
                -noout \
                -subject \
            | sed \
                's;subject= \(.*\);\1;' \
        )"
       /usr/share/ovirt-engine/bin/pki-enroll-pkcs12.sh \
            --name="${name}" \
            --password=mypass \
            --subject="${subject}" \
            --san=DNS:"${ENGINE_FQDN}" \
            --keep-key
    done
  5. Restart the httpd service:

    # systemctl restart httpd
  6. Connect to the Administration Portal to confirm that the warning no longer appears.

  7. If you previously imported a CA or https certificate into the browser, find the certificate(s), remove them from the browser, and reimport the new CA certificate. Install the certificate authority according to the instructions provided by your browser. To get the certificate authority’s certificate, navigate to http://your-manager-fqdn/ovirt-engine/services/pki-resource?resource=ca-certificate&format=X509-PEM-CA, replacing your-manager-fqdn with the fully qualified domain name (FQDN).

Replacing All Signed Certificates with SHA-256

  1. Log in to the Engine machine as the root user.

  2. Check whether /etc/pki/ovirt-engine/openssl.conf includes the line default_md = sha256:

    # cat /etc/pki/ovirt-engine/openssl.conf

    If it still includes default_md = sha1, back up the existing configuration and change the default to sha256:

    # cp -p /etc/pki/ovirt-engine/openssl.conf /etc/pki/ovirt-engine/openssl.conf."$(date +"%Y%m%d%H%M%S")"
    # sed -i 's/^default_md = sha1/default_md = sha256/' /etc/pki/ovirt-engine/openssl.conf
  3. Re-sign the CA certificate by backing it up and creating a new certificate in ca.pem.new:

    # cp -p /etc/pki/ovirt-engine/private/ca.pem /etc/pki/ovirt-engine/private/ca.pem."$(date +"%Y%m%d%H%M%S")"
    # openssl x509 -signkey /etc/pki/ovirt-engine/private/ca.pem -in /etc/pki/ovirt-engine/ca.pem -out /etc/pki/ovirt-engine/ca.pem.new -days 3650 -sha256
  4. Replace the existing certificate with the new certificate:

    # mv /etc/pki/ovirt-engine/ca.pem.new /etc/pki/ovirt-engine/ca.pem
  5. Define the certificates that should be re-signed:

    # names="engine apache websocket-proxy jboss imageio-proxy"

    If you replaced the oVirt Engine SSL Certificate after the upgrade, run the following instead:

    # names="engine websocket-proxy jboss imageio-proxy"

    For more details see Replacing the oVirt Engine CA Certificate in the Administration Guide.

  6. On the Engine, re-sign the certificates:

    for name in $names; do
       subject="$(
            openssl \
                x509 \
                -in /etc/pki/ovirt-engine/certs/"${name}".cer \
                -noout \
                -subject \
            | sed \
                's;subject= \(.*\);\1;' \
            )"
         /usr/share/ovirt-engine/bin/pki-enroll-pkcs12.sh \
                --name="${name}" \
                --password=mypass \
                --subject="${subject}" \
                --san=DNS:"${ENGINE_FQDN}" \
                --keep-key
    done
  7. Restart the following services:

    # systemctl restart httpd
    # systemctl restart ovirt-engine
    # systemctl restart ovirt-websocket-proxy
    # systemctl restart ovirt-imageio-proxy
  8. Connect to the Administration Portal to confirm that the warning no longer appears.

  9. If you previously imported a CA or https certificate into the browser, find the certificate(s), remove them from the browser, and reimport the new CA certificate. Install the certificate authority according to the instructions provided by your browser. To get the certificate authority’s certificate, navigate to http://your-manager-fqdn/ovirt-engine/services/pki-resource?resource=ca-certificate&format=X509-PEM-CA, replacing your-manager-fqdn with the fully qualified domain name (FQDN).

  10. Enroll the certificates on the hosts. Repeat the following procedure for each host.

    1. In the Administration Portal, click Compute  Hosts.

    2. Select the host and click Management  Maintenance.

    3. Once the host is in maintenance mode, click Installation  Enroll Certificate.

    4. Click Management  Activate.

Upgrading a Local Database Environment Manually

6. Upgrading from 4.0 to oVirt 4.3

The 4.0 compatibility version is not supported after oVirt 4.2. Therefore, when upgrading from oVirt 4.0 you must update the cluster and data center compatibility versions to at least 4.1 before upgrading the Engine from 4.2 to 4.3, then update the compatibility versions again after completing the Engine upgrades.

You must also update the hosts before updating the compatibility versions, but only need to do so once. The host repositories stay the same across oVirt versions, so the hosts will already be upgraded to the latest version after a single update.

Upgrading your environment from 4.0 to 4.3 involves the following steps:

6.1. Prerequisites

  • Plan for any necessary virtual machine downtime. After you update the clusters' compatibility versions during the upgrade, a new hardware configuration is automatically applied to each virtual machine once it reboots. You must reboot any running or suspended virtual machines as soon as possible to apply the configuration changes.

  • Ensure the Engine has the correct repositories enabled.

6.2. Updating the oVirt Engine

Be sure to enable the correct repositories as detailed in Prerequisites. Otherwise, the upgrade might fail.

Updates to the oVirt Engine are released through the Content Delivery Network.

Procedure
  1. Log in to the Engine machine.

  2. Check if updated packages are available:

    # engine-upgrade-check
  3. Update the setup packages:

    # yum update ovirt\*setup\*
  4. Update the oVirt Engine with the engine-setup script. The engine-setup script prompts you with some configuration questions, then stops the ovirt-engine service, downloads and installs the updated packages, backs up and updates the database, performs post-installation configuration, and starts the ovirt-engine service.

    # engine-setup

    When the script completes successfully, the following message appears:

    Execution of setup completed successfully

    The engine-setup script is also used during the oVirt Engine installation process, and it stores the configuration values supplied. During an update, the stored values are displayed when previewing the configuration, and might not be up to date if engine-config was used to update configuration after installation. For example, if engine-config was used to update SANWipeAfterDelete to true after installation, engine-setup will output "Default SAN wipe after delete: False" in the configuration preview. However, the updated values will not be overwritten by engine-setup.

    The update process might take some time. Do not stop the process before it completes.

  5. Update the base operating system and any optional packages installed on the Engine:

    # yum update

    If any kernel packages were updated, reboot the machine to complete the update.

6.3. Upgrading the Engine from 4.0 to 4.1

Upgrade the oVirt Engine from 4.0 to 4.1.

If the upgrade fails, the engine-setup command attempts to roll your oVirt Engine installation back to its previous state. For this reason, the previous version’s repositories must not be removed until after the upgrade is complete. If the upgrade fails, detailed instructions display that explain how to restore your installation.

Procedure
  1. Log in to the Engine machine.

  2. Enable the oVirt 4.1 repositories:

    All other repositories remain the same across oVirt releases.

  3. Update the setup packages:

    # yum update ovirt\*setup\*
  4. Run engine-setup and follow the prompts to upgrade the oVirt Engine:

    # engine-setup
  5. Update the base operating system:

    # yum update

    If any kernel packages were updated, reboot the machine to complete the upgrade.

6.4. Upgrading the Engine from 4.1 to 4.2

Upgrade the oVirt Engine from 4.1 to 4.2.

If the upgrade fails, the engine-setup command attempts to roll your oVirt Engine installation back to its previous state. For this reason, the previous version’s repositories must not be removed until after the upgrade is complete. If the upgrade fails, detailed instructions display that explain how to restore your installation.

Procedure
  1. Log in to the Engine machine.

  2. Enable the oVirt 4.2 repositories:

    # yum install https://resources.ovirt.org/pub/yum-repo/ovirt-release42.rpm

    All other repositories remain the same across oVirt releases.

  3. Update the setup packages:

    # yum update ovirt\*setup\*
  4. Run engine-setup and follow the prompts to upgrade the oVirt Engine:

    # engine-setup

    If you changed the default trust keystore password, you might get a keystore certificate error. If so, update the configuration to reflect the correct password. For more information, see RHV 4.2 - Failed to import provider certificate into the external provider keystore

  5. Update the base operating system:

    # yum update

    If any kernel packages were updated, reboot the machine to complete the upgrade.

You must now update the hosts before you can update the cluster and data center compatibility versions.

6.5. Updating Individual Hosts

Use the host upgrade manager to update individual hosts directly from the Administration Portal.

The upgrade manager only checks hosts with a status of Up or Non-operational, but not Maintenance.

Limitations
  • On oVirt Node, the update only preserves modified content in the /etc and /var directories. Modified data in other paths is overwritten during an update.

  • If migration is enabled at the cluster level, virtual machines are automatically migrated to another host in the cluster. Update a host when its usage is relatively low.

  • In a self-hosted engine environment, the Engine virtual machine can only migrate between self-hosted engine nodes in the same cluster. It cannot migrate to standard hosts.

  • The cluster must have sufficient memory reserved for its hosts to perform maintenance. Otherwise, virtual machine migrations will hang and fail. You can reduce the memory usage of host updates by shutting down some or all virtual machines before updating hosts.

  • Do not update all hosts at the same time, as one host must remain available to perform Storage Pool Manager (SPM) tasks.

  • You cannot migrate a pinned virtual machine (such as a virtual machine using a vGPU) to another host. Pinned virtual machines must be shut down before updating the host.

Procedure
  1. Ensure that the correct repositories are enabled. To view a list of currently enabled repositories, run yum repolist.

  2. In the Administration Portal, click Compute  Hosts and select the host to be updated.

  3. Click Installation  Check for Upgrade and click OK.

    Open the Notification Drawer (EventsIcon) and expand the Events section to see the result.

  4. If an update is available, click Installation  Upgrade.

  5. Click OK to update the host. Running virtual machines are migrated according to their migration policy. If migration is disabled for any virtual machines, you are prompted to shut them down.

    The details of the host are updated in Compute  Hosts and the status transitions through these stages:

    Maintenance > Installing > Reboot > Up

    If the update fails, the host’s status changes to Install Failed. From Install Failed you can click Installation  Upgrade again.

Repeat this procedure for each host in the oVirt environment.

You can now change the cluster compatibility version to 4.2.

6.6. Changing the Cluster Compatibility Version

oVirt clusters have a compatibility version. The cluster compatibility version indicates the features of oVirt supported by all of the hosts in the cluster. The cluster compatibility is set according to the version of the least capable host operating system in the cluster.

To change the cluster compatibility version, you must have first updated all the hosts in your cluster to a level that supports your desired compatibility level. Check if there is an icon next to the host indicating an update is available.

Procedure
  1. In the Administration Portal, click Compute  Clusters.

  2. Select the cluster to change and click Edit.

  3. On the General tab, change the Compatibility Version to the desired value.

  4. Click OK. The Change Cluster Compatibility Version confirmation dialog opens.

  5. Click OK to confirm.

An error message might warn that some virtual machines and templates are incorrectly configured. To fix this error, edit each virtual machine manually. The Edit Virtual Machine window provides additional validations and warnings that show what to correct. Sometimes the issue is automatically corrected and the virtual machine’s configuration just needs to be saved again. After editing each virtual machine, you will be able to change the cluster compatibility version.

6.7. Changing Virtual Machine Cluster Compatibility

After updating a cluster’s compatibility version, you must update the cluster compatibility version of all running or suspended virtual machines by rebooting them from the Administration Portal, or using the REST API, instead of from within the guest operating system. Virtual machines that require a reboot are marked with the pending changes icon (pendingchanges).

Although you can wait to reboot the virtual machines at a convenient time, rebooting immediately is highly recommended so that the virtual machines use the latest configuration. Any virtual machine that has not been rebooted runs with the previous configuration, and subsequent configuration changes made to the virtual machine might overwrite its pending cluster compatibility changes.

Procedure
  1. In the Administration Portal, click Compute  Virtual Machines.

  2. Check which virtual machines require a reboot. In the Vms: search bar, enter the following query:

    next_run_config_exists=True

    The search results show all virtual machines with pending changes.

  3. Select each virtual machine and click Reboot.

When the virtual machine starts, the new compatibility version is automatically applied.

You cannot change the cluster compatibility version of a virtual machine snapshot that is in preview. You must first commit or undo the preview.

You can now change the data center compatibility version to 4.2.

6.8. Changing the Data Center Compatibility Version

oVirt data centers have a compatibility version. The compatibility version indicates the version of oVirt with which the data center is intended to be compatible. All clusters in the data center must support the desired compatibility level.

To change the data center compatibility version, you must have first updated the compatibility version of all clusters and virtual machines in the data center.

Procedure
  1. In the Administration Portal, click Compute  Data Centers.

  2. Select the data center to change and click Edit.

  3. Change the Compatibility Version to the desired value.

  4. Click OK. The Change Data Center Compatibility Version confirmation dialog opens.

  5. Click OK to confirm.

6.9. Upgrading the Engine from 4.2 to 4.3

Upgrade the oVirt Engine from 4.2 to 4.3.

If the upgrade fails, the engine-setup command attempts to roll your oVirt Engine installation back to its previous state. For this reason, the previous version’s repositories must not be removed until after the upgrade is complete. If the upgrade fails, detailed instructions display that explain how to restore your installation.

Procedure
  1. Log in to the Engine machine.

  2. Enable the oVirt 4.3 repositories:

    # yum install https://resources.ovirt.org/pub/yum-repo/ovirt-release43.rpm

    All other repositories remain the same across oVirt releases.

  3. Update the setup packages:

    # yum update ovirt\*setup\*
  4. Run engine-setup and follow the prompts to upgrade the oVirt Engine:

    # engine-setup

    When the script completes successfully, the following message appears:

    Execution of setup completed successfully
  5. Update the base operating system:

    # yum update

    If any kernel packages were updated, reboot the machine to complete the upgrade.

You can now change the cluster compatibility version to 4.3.

6.10. Changing the Cluster Compatibility Version

oVirt clusters have a compatibility version. The cluster compatibility version indicates the features of oVirt supported by all of the hosts in the cluster. The cluster compatibility is set according to the version of the least capable host operating system in the cluster.

To change the cluster compatibility version, you must have first updated all the hosts in your cluster to a level that supports your desired compatibility level. Check if there is an icon next to the host indicating an update is available.

Procedure
  1. In the Administration Portal, click Compute  Clusters.

  2. Select the cluster to change and click Edit.

  3. On the General tab, change the Compatibility Version to the desired value.

  4. Click OK. The Change Cluster Compatibility Version confirmation dialog opens.

  5. Click OK to confirm.

An error message might warn that some virtual machines and templates are incorrectly configured. To fix this error, edit each virtual machine manually. The Edit Virtual Machine window provides additional validations and warnings that show what to correct. Sometimes the issue is automatically corrected and the virtual machine’s configuration just needs to be saved again. After editing each virtual machine, you will be able to change the cluster compatibility version.

6.11. Changing Virtual Machine Cluster Compatibility

After updating a cluster’s compatibility version, you must update the cluster compatibility version of all running or suspended virtual machines by rebooting them from the Administration Portal, or using the REST API, instead of from within the guest operating system. Virtual machines that require a reboot are marked with the pending changes icon (pendingchanges).

Although you can wait to reboot the virtual machines at a convenient time, rebooting immediately is highly recommended so that the virtual machines use the latest configuration. Any virtual machine that has not been rebooted runs with the previous configuration, and subsequent configuration changes made to the virtual machine might overwrite its pending cluster compatibility changes.

Procedure
  1. In the Administration Portal, click Compute  Virtual Machines.

  2. Check which virtual machines require a reboot. In the Vms: search bar, enter the following query:

    next_run_config_exists=True

    The search results show all virtual machines with pending changes.

  3. Select each virtual machine and click Reboot.

When the virtual machine starts, the new compatibility version is automatically applied.

You cannot change the cluster compatibility version of a virtual machine snapshot that is in preview. You must first commit or undo the preview.

You can now change the data center compatibility version to 4.3.

6.12. Changing the Data Center Compatibility Version

oVirt data centers have a compatibility version. The compatibility version indicates the version of oVirt with which the data center is intended to be compatible. All clusters in the data center must support the desired compatibility level.

To change the data center compatibility version, you must have first updated the compatibility version of all clusters and virtual machines in the data center.

Procedure
  1. In the Administration Portal, click Compute  Data Centers.

  2. Select the data center to change and click Edit.

  3. Change the Compatibility Version to the desired value.

  4. Click OK. The Change Data Center Compatibility Version confirmation dialog opens.

  5. Click OK to confirm.

6.13. Replacing SHA-1 Certificates with SHA-256 Certificates

oVirt 4.3 uses SHA-256 signatures, which provide a more secure way to sign SSL certificates than SHA-1. Newly installed systems do not require any special steps to enable oVirt’s public key infrastructure (PKI) to use SHA-256 signatures. However, for systems upgraded from 4.1 or earlier, one of the following is required:

Preventing Warning Messages from Appearing in the Browser

  1. Log in to the Engine machine as the root user.

  2. Check whether /etc/pki/ovirt-engine/openssl.conf includes the line default_md = sha256:

    # cat /etc/pki/ovirt-engine/openssl.conf

    If it still includes default_md = sha1, back up the existing configuration and change the default to sha256:

    # cp -p /etc/pki/ovirt-engine/openssl.conf /etc/pki/ovirt-engine/openssl.conf."$(date +"%Y%m%d%H%M%S")"
    # sed -i 's/^default_md = sha1/default_md = sha256/' /etc/pki/ovirt-engine/openssl.conf
  3. Define the certificate that should be re-signed:

    # names="apache"
  4. On the Engine, re-sign the Apache certificate:

    for name in $names; do
        subject="$(
            openssl \
                x509 \
                -in /etc/pki/ovirt-engine/certs/"${name}".cer \
                -noout \
                -subject \
            | sed \
                's;subject= \(.*\);\1;' \
        )"
       /usr/share/ovirt-engine/bin/pki-enroll-pkcs12.sh \
            --name="${name}" \
            --password=mypass \
            --subject="${subject}" \
            --san=DNS:"${ENGINE_FQDN}" \
            --keep-key
    done
  5. Restart the httpd service:

    # systemctl restart httpd
  6. Connect to the Administration Portal to confirm that the warning no longer appears.

  7. If you previously imported a CA or https certificate into the browser, find the certificate(s), remove them from the browser, and reimport the new CA certificate. Install the certificate authority according to the instructions provided by your browser. To get the certificate authority’s certificate, navigate to http://your-manager-fqdn/ovirt-engine/services/pki-resource?resource=ca-certificate&format=X509-PEM-CA, replacing your-manager-fqdn with the fully qualified domain name (FQDN).

Replacing All Signed Certificates with SHA-256

  1. Log in to the Engine machine as the root user.

  2. Check whether /etc/pki/ovirt-engine/openssl.conf includes the line default_md = sha256:

    # cat /etc/pki/ovirt-engine/openssl.conf

    If it still includes default_md = sha1, back up the existing configuration and change the default to sha256:

    # cp -p /etc/pki/ovirt-engine/openssl.conf /etc/pki/ovirt-engine/openssl.conf."$(date +"%Y%m%d%H%M%S")"
    # sed -i 's/^default_md = sha1/default_md = sha256/' /etc/pki/ovirt-engine/openssl.conf
  3. Re-sign the CA certificate by backing it up and creating a new certificate in ca.pem.new:

    # cp -p /etc/pki/ovirt-engine/private/ca.pem /etc/pki/ovirt-engine/private/ca.pem."$(date +"%Y%m%d%H%M%S")"
    # openssl x509 -signkey /etc/pki/ovirt-engine/private/ca.pem -in /etc/pki/ovirt-engine/ca.pem -out /etc/pki/ovirt-engine/ca.pem.new -days 3650 -sha256
  4. Replace the existing certificate with the new certificate:

    # mv /etc/pki/ovirt-engine/ca.pem.new /etc/pki/ovirt-engine/ca.pem
  5. Define the certificates that should be re-signed:

    # names="engine apache websocket-proxy jboss imageio-proxy"

    If you replaced the oVirt Engine SSL Certificate after the upgrade, run the following instead:

    # names="engine websocket-proxy jboss imageio-proxy"

    For more details see Replacing the oVirt Engine CA Certificate in the Administration Guide.

  6. On the Engine, re-sign the certificates:

    for name in $names; do
       subject="$(
            openssl \
                x509 \
                -in /etc/pki/ovirt-engine/certs/"${name}".cer \
                -noout \
                -subject \
            | sed \
                's;subject= \(.*\);\1;' \
            )"
         /usr/share/ovirt-engine/bin/pki-enroll-pkcs12.sh \
                --name="${name}" \
                --password=mypass \
                --subject="${subject}" \
                --san=DNS:"${ENGINE_FQDN}" \
                --keep-key
    done
  7. Restart the following services:

    # systemctl restart httpd
    # systemctl restart ovirt-engine
    # systemctl restart ovirt-websocket-proxy
    # systemctl restart ovirt-imageio-proxy
  8. Connect to the Administration Portal to confirm that the warning no longer appears.

  9. If you previously imported a CA or https certificate into the browser, find the certificate(s), remove them from the browser, and reimport the new CA certificate. Install the certificate authority according to the instructions provided by your browser. To get the certificate authority’s certificate, navigate to http://your-manager-fqdn/ovirt-engine/services/pki-resource?resource=ca-certificate&format=X509-PEM-CA, replacing your-manager-fqdn with the fully qualified domain name (FQDN).

  10. Enroll the certificates on the hosts. Repeat the following procedure for each host.

    1. In the Administration Portal, click Compute  Hosts.

    2. Select the host and click Management  Maintenance.

    3. Once the host is in maintenance mode, click Installation  Enroll Certificate.

    4. Click Management  Activate.

7. Upgrading from 4.1 to oVirt 4.3

Upgrading your environment from 4.1 to 4.3 involves the following steps:

7.1. Prerequisites

  • Plan for any necessary virtual machine downtime. After you update the clusters' compatibility versions during the upgrade, a new hardware configuration is automatically applied to each virtual machine once it reboots. You must reboot any running or suspended virtual machines as soon as possible to apply the configuration changes.

  • Ensure the Engine has the correct repositories enabled.

7.2. Updating the oVirt Engine

Be sure to enable the correct repositories as detailed in Prerequisites. Otherwise, the upgrade might fail.

Updates to the oVirt Engine are released through the Content Delivery Network.

Procedure
  1. Log in to the Engine machine.

  2. Check if updated packages are available:

    # engine-upgrade-check
  3. Update the setup packages:

    # yum update ovirt\*setup\*
  4. Update the oVirt Engine with the engine-setup script. The engine-setup script prompts you with some configuration questions, then stops the ovirt-engine service, downloads and installs the updated packages, backs up and updates the database, performs post-installation configuration, and starts the ovirt-engine service.

    # engine-setup

    When the script completes successfully, the following message appears:

    Execution of setup completed successfully

    The engine-setup script is also used during the oVirt Engine installation process, and it stores the configuration values supplied. During an update, the stored values are displayed when previewing the configuration, and might not be up to date if engine-config was used to update configuration after installation. For example, if engine-config was used to update SANWipeAfterDelete to true after installation, engine-setup will output "Default SAN wipe after delete: False" in the configuration preview. However, the updated values will not be overwritten by engine-setup.

    The update process might take some time. Do not stop the process before it completes.

  5. Update the base operating system and any optional packages installed on the Engine:

    # yum update

    If any kernel packages were updated, reboot the machine to complete the update.

7.3. Upgrading the Engine from 4.1 to 4.2

Upgrade the oVirt Engine from 4.1 to 4.2.

If the upgrade fails, the engine-setup command attempts to roll your oVirt Engine installation back to its previous state. For this reason, the previous version’s repositories must not be removed until after the upgrade is complete. If the upgrade fails, detailed instructions display that explain how to restore your installation.

Procedure
  1. Log in to the Engine machine.

  2. Enable the oVirt 4.2 repositories:

    # yum install https://resources.ovirt.org/pub/yum-repo/ovirt-release42.rpm

    All other repositories remain the same across oVirt releases.

  3. Update the setup packages:

    # yum update ovirt\*setup\*
  4. Run engine-setup and follow the prompts to upgrade the oVirt Engine:

    # engine-setup

    If you changed the default trust keystore password, you might get a keystore certificate error. If so, update the configuration to reflect the correct password. For more information, see RHV 4.2 - Failed to import provider certificate into the external provider keystore

  5. Update the base operating system:

    # yum update

    If any kernel packages were updated, reboot the machine to complete the upgrade.

7.4. Upgrading the Engine from 4.2 to 4.3

Upgrade the oVirt Engine from 4.2 to 4.3.

If the upgrade fails, the engine-setup command attempts to roll your oVirt Engine installation back to its previous state. For this reason, the previous version’s repositories must not be removed until after the upgrade is complete. If the upgrade fails, detailed instructions display that explain how to restore your installation.

Procedure
  1. Log in to the Engine machine.

  2. Enable the oVirt 4.3 repositories:

    # yum install https://resources.ovirt.org/pub/yum-repo/ovirt-release43.rpm

    All other repositories remain the same across oVirt releases.

  3. Update the setup packages:

    # yum update ovirt\*setup\*
  4. Run engine-setup and follow the prompts to upgrade the oVirt Engine:

    # engine-setup

    When the script completes successfully, the following message appears:

    Execution of setup completed successfully
  5. Update the base operating system:

    # yum update

    If any kernel packages were updated, reboot the machine to complete the upgrade.

You can now update the hosts.

7.5. Updating All Hosts in a Cluster

You can update all hosts in a cluster instead of updating hosts individually. This is particularly useful during upgrades to new versions of oVirt. See https://github.com/oVirt/ovirt-ansible-cluster-upgrade/blob/master/README.md for more information about the Ansible role used to automate the updates.

oVirt recommends updating one cluster at a time.

Limitations
  • On oVirt Node, the update only preserves modified content in the /etc and /var directories. Modified data in other paths is overwritten during an update.

  • If migration is enabled at the cluster level, virtual machines are automatically migrated to another host in the cluster.

  • In a self-hosted engine environment, the Engine virtual machine can only migrate between self-hosted engine nodes in the same cluster. It cannot migrate to standard hosts.

  • The cluster must have sufficient memory reserved for its hosts to perform maintenance. Otherwise, virtual machine migrations will hang and fail. You can reduce the memory usage of host updates by shutting down some or all virtual machines before updating hosts.

  • You cannot migrate a pinned virtual machine (such as a virtual machine using a vGPU) to another host. Pinned virtual machines are shut down during the update, unless you choose to skip that host instead.

Procedure
  1. In the Administration Portal, click Compute  Clusters and select the cluster.

  2. Click Upgrade.

  3. Select the hosts to update, then click Next.

  4. Configure the options:

    • Stop Pinned VMs shuts down any virtual machines that are pinned to hosts in the cluster, and is selected by default. You can clear this check box to skip updating those hosts so that the pinned virtual machines stay running, such as when a pinned virtual machine is running important services or processes and you do not want it to shut down at an unknown time during the update.

    • Upgrade Timeout (Minutes) sets the time to wait for an individual host to be updated before the cluster upgrade fails with a timeout. The default is 60. You can increase it for large clusters where 60 minutes might not be enough, or reduce it for small clusters where the hosts update quickly.

    • Check Upgrade checks each host for available updates before running the upgrade process. It is not selected by default, but you can select it if you need to ensure that recent updates are included, such as when you have configured the Engine to check for host updates less frequently than the default.

    • Reboot After Upgrade reboots each host after it is updated, and is selected by default. You can clear this check box to speed up the process if you are sure that there are no pending updates that require a host reboot.

    • Use Maintenance Policy sets the cluster’s scheduling policy to cluster_maintenance during the update. It is selected by default, so activity is limited and virtual machines cannot start unless they are highly available. You can clear this check box if you have a custom scheduling policy that you want to keep using during the update, but this could have unknown consequences. Ensure your custom policy is compatible with cluster upgrade activity before disabling this option.

  5. Click Next.

  6. Review the summary of the hosts and virtual machines that will be affected.

  7. Click Upgrade.

You can track the progress of host updates in the Compute  Hosts view, and in the Events section of the Notification Drawer (EventsIcon).

You can track the progress of individual virtual machine migrations in the Status column of the Compute  Virtual Machines view. In large environments, you may need to filter the results to show a particular group of virtual machines.

7.6. Changing the Cluster Compatibility Version

oVirt clusters have a compatibility version. The cluster compatibility version indicates the features of oVirt supported by all of the hosts in the cluster. The cluster compatibility is set according to the version of the least capable host operating system in the cluster.

To change the cluster compatibility version, you must have first updated all the hosts in your cluster to a level that supports your desired compatibility level. Check if there is an icon next to the host indicating an update is available.

Procedure
  1. In the Administration Portal, click Compute  Clusters.

  2. Select the cluster to change and click Edit.

  3. On the General tab, change the Compatibility Version to the desired value.

  4. Click OK. The Change Cluster Compatibility Version confirmation dialog opens.

  5. Click OK to confirm.

An error message might warn that some virtual machines and templates are incorrectly configured. To fix this error, edit each virtual machine manually. The Edit Virtual Machine window provides additional validations and warnings that show what to correct. Sometimes the issue is automatically corrected and the virtual machine’s configuration just needs to be saved again. After editing each virtual machine, you will be able to change the cluster compatibility version.

7.7. Changing Virtual Machine Cluster Compatibility

After updating a cluster’s compatibility version, you must update the cluster compatibility version of all running or suspended virtual machines by rebooting them from the Administration Portal, or using the REST API, instead of from within the guest operating system. Virtual machines that require a reboot are marked with the pending changes icon (pendingchanges).

Although you can wait to reboot the virtual machines at a convenient time, rebooting immediately is highly recommended so that the virtual machines use the latest configuration. Any virtual machine that has not been rebooted runs with the previous configuration, and subsequent configuration changes made to the virtual machine might overwrite its pending cluster compatibility changes.

Procedure
  1. In the Administration Portal, click Compute  Virtual Machines.

  2. Check which virtual machines require a reboot. In the Vms: search bar, enter the following query:

    next_run_config_exists=True

    The search results show all virtual machines with pending changes.

  3. Select each virtual machine and click Reboot.

When the virtual machine starts, the new compatibility version is automatically applied.

You cannot change the cluster compatibility version of a virtual machine snapshot that is in preview. You must first commit or undo the preview.

7.8. Changing the Data Center Compatibility Version

oVirt data centers have a compatibility version. The compatibility version indicates the version of oVirt with which the data center is intended to be compatible. All clusters in the data center must support the desired compatibility level.

To change the data center compatibility version, you must have first updated the compatibility version of all clusters and virtual machines in the data center.

Procedure
  1. In the Administration Portal, click Compute  Data Centers.

  2. Select the data center to change and click Edit.

  3. Change the Compatibility Version to the desired value.

  4. Click OK. The Change Data Center Compatibility Version confirmation dialog opens.

  5. Click OK to confirm.

7.9. Replacing SHA-1 Certificates with SHA-256 Certificates

oVirt 4.3 uses SHA-256 signatures, which provide a more secure way to sign SSL certificates than SHA-1. Newly installed systems do not require any special steps to enable oVirt’s public key infrastructure (PKI) to use SHA-256 signatures. However, for systems upgraded from 4.1 or earlier, one of the following is required:

Preventing Warning Messages from Appearing in the Browser

  1. Log in to the Engine machine as the root user.

  2. Check whether /etc/pki/ovirt-engine/openssl.conf includes the line default_md = sha256:

    # cat /etc/pki/ovirt-engine/openssl.conf

    If it still includes default_md = sha1, back up the existing configuration and change the default to sha256:

    # cp -p /etc/pki/ovirt-engine/openssl.conf /etc/pki/ovirt-engine/openssl.conf."$(date +"%Y%m%d%H%M%S")"
    # sed -i 's/^default_md = sha1/default_md = sha256/' /etc/pki/ovirt-engine/openssl.conf
  3. Define the certificate that should be re-signed:

    # names="apache"
  4. On the Engine, re-sign the Apache certificate:

    for name in $names; do
        subject="$(
            openssl \
                x509 \
                -in /etc/pki/ovirt-engine/certs/"${name}".cer \
                -noout \
                -subject \
            | sed \
                's;subject= \(.*\);\1;' \
        )"
       /usr/share/ovirt-engine/bin/pki-enroll-pkcs12.sh \
            --name="${name}" \
            --password=mypass \
            --subject="${subject}" \
            --san=DNS:"${ENGINE_FQDN}" \
            --keep-key
    done
  5. Restart the httpd service:

    # systemctl restart httpd
  6. Connect to the Administration Portal to confirm that the warning no longer appears.

  7. If you previously imported a CA or https certificate into the browser, find the certificate(s), remove them from the browser, and reimport the new CA certificate. Install the certificate authority according to the instructions provided by your browser. To get the certificate authority’s certificate, navigate to http://your-manager-fqdn/ovirt-engine/services/pki-resource?resource=ca-certificate&format=X509-PEM-CA, replacing your-manager-fqdn with the fully qualified domain name (FQDN).

Replacing All Signed Certificates with SHA-256

  1. Log in to the Engine machine as the root user.

  2. Check whether /etc/pki/ovirt-engine/openssl.conf includes the line default_md = sha256:

    # cat /etc/pki/ovirt-engine/openssl.conf

    If it still includes default_md = sha1, back up the existing configuration and change the default to sha256:

    # cp -p /etc/pki/ovirt-engine/openssl.conf /etc/pki/ovirt-engine/openssl.conf."$(date +"%Y%m%d%H%M%S")"
    # sed -i 's/^default_md = sha1/default_md = sha256/' /etc/pki/ovirt-engine/openssl.conf
  3. Re-sign the CA certificate by backing it up and creating a new certificate in ca.pem.new:

    # cp -p /etc/pki/ovirt-engine/private/ca.pem /etc/pki/ovirt-engine/private/ca.pem."$(date +"%Y%m%d%H%M%S")"
    # openssl x509 -signkey /etc/pki/ovirt-engine/private/ca.pem -in /etc/pki/ovirt-engine/ca.pem -out /etc/pki/ovirt-engine/ca.pem.new -days 3650 -sha256
  4. Replace the existing certificate with the new certificate:

    # mv /etc/pki/ovirt-engine/ca.pem.new /etc/pki/ovirt-engine/ca.pem
  5. Define the certificates that should be re-signed:

    # names="engine apache websocket-proxy jboss imageio-proxy"

    If you replaced the oVirt Engine SSL Certificate after the upgrade, run the following instead:

    # names="engine websocket-proxy jboss imageio-proxy"

    For more details see Replacing the oVirt Engine CA Certificate in the Administration Guide.

  6. On the Engine, re-sign the certificates:

    for name in $names; do
       subject="$(
            openssl \
                x509 \
                -in /etc/pki/ovirt-engine/certs/"${name}".cer \
                -noout \
                -subject \
            | sed \
                's;subject= \(.*\);\1;' \
            )"
         /usr/share/ovirt-engine/bin/pki-enroll-pkcs12.sh \
                --name="${name}" \
                --password=mypass \
                --subject="${subject}" \
                --san=DNS:"${ENGINE_FQDN}" \
                --keep-key
    done
  7. Restart the following services:

    # systemctl restart httpd
    # systemctl restart ovirt-engine
    # systemctl restart ovirt-websocket-proxy
    # systemctl restart ovirt-imageio-proxy
  8. Connect to the Administration Portal to confirm that the warning no longer appears.

  9. If you previously imported a CA or https certificate into the browser, find the certificate(s), remove them from the browser, and reimport the new CA certificate. Install the certificate authority according to the instructions provided by your browser. To get the certificate authority’s certificate, navigate to http://your-manager-fqdn/ovirt-engine/services/pki-resource?resource=ca-certificate&format=X509-PEM-CA, replacing your-manager-fqdn with the fully qualified domain name (FQDN).

  10. Enroll the certificates on the hosts. Repeat the following procedure for each host.

    1. In the Administration Portal, click Compute  Hosts.

    2. Select the host and click Management  Maintenance.

    3. Once the host is in maintenance mode, click Installation  Enroll Certificate.

    4. Click Management  Activate.

7.10. Updating OVN Providers Installed in oVirt 4.1

If you installed an Open Virtual Network (OVN) provider in oVirt 4.1, you must manually edit its configuration for oVirt 4.2.

Procedure
  1. Click Administration  Providers and select the OVN provider.

  2. Click Edit.

  3. Click the Networking Plugin text field and select oVirt Network Provider for OVN from the drop-down list.

  4. Click OK.

8. Upgrading from 4.2 to oVirt 4.3

Upgrading your environment from 4.2 to 4.3 involves the following steps:

8.1. Prerequisites

  • Plan for any necessary virtual machine downtime. After you update the clusters' compatibility versions during the upgrade, a new hardware configuration is automatically applied to each virtual machine once it reboots. You must reboot any running or suspended virtual machines as soon as possible to apply the configuration changes.

  • Ensure the Engine has the correct repositories enabled.

8.2. Updating the oVirt Engine

Be sure to enable the correct repositories as detailed in Prerequisites. Otherwise, the upgrade might fail.

Updates to the oVirt Engine are released through the Content Delivery Network.

Procedure
  1. Log in to the Engine machine.

  2. Check if updated packages are available:

    # engine-upgrade-check
  3. Update the setup packages:

    # yum update ovirt\*setup\*
  4. Update the oVirt Engine with the engine-setup script. The engine-setup script prompts you with some configuration questions, then stops the ovirt-engine service, downloads and installs the updated packages, backs up and updates the database, performs post-installation configuration, and starts the ovirt-engine service.

    # engine-setup

    When the script completes successfully, the following message appears:

    Execution of setup completed successfully

    The engine-setup script is also used during the oVirt Engine installation process, and it stores the configuration values supplied. During an update, the stored values are displayed when previewing the configuration, and might not be up to date if engine-config was used to update configuration after installation. For example, if engine-config was used to update SANWipeAfterDelete to true after installation, engine-setup will output "Default SAN wipe after delete: False" in the configuration preview. However, the updated values will not be overwritten by engine-setup.

    The update process might take some time. Do not stop the process before it completes.

  5. Update the base operating system and any optional packages installed on the Engine:

    # yum update

    If any kernel packages were updated, reboot the machine to complete the update.

8.3. Upgrading the Engine from 4.2 to 4.3

Upgrade the oVirt Engine from 4.2 to 4.3.

If the upgrade fails, the engine-setup command attempts to roll your oVirt Engine installation back to its previous state. For this reason, the previous version’s repositories must not be removed until after the upgrade is complete. If the upgrade fails, detailed instructions display that explain how to restore your installation.

Procedure
  1. Log in to the Engine machine.

  2. Enable the oVirt 4.3 repositories:

    # yum install https://resources.ovirt.org/pub/yum-repo/ovirt-release43.rpm

    All other repositories remain the same across oVirt releases.

  3. Update the setup packages:

    # yum update ovirt\*setup\*
  4. Run engine-setup and follow the prompts to upgrade the oVirt Engine:

    # engine-setup

    When the script completes successfully, the following message appears:

    Execution of setup completed successfully
  5. Update the base operating system:

    # yum update

    If any kernel packages were updated, reboot the machine to complete the upgrade.

You can now update the hosts.

8.4. Updating All Hosts in a Cluster

You can update all hosts in a cluster instead of updating hosts individually. This is particularly useful during upgrades to new versions of oVirt. See https://github.com/oVirt/ovirt-ansible-cluster-upgrade/blob/master/README.md for more information about the Ansible role used to automate the updates.

oVirt recommends updating one cluster at a time.

Limitations
  • On oVirt Node, the update only preserves modified content in the /etc and /var directories. Modified data in other paths is overwritten during an update.

  • If migration is enabled at the cluster level, virtual machines are automatically migrated to another host in the cluster.

  • In a self-hosted engine environment, the Engine virtual machine can only migrate between self-hosted engine nodes in the same cluster. It cannot migrate to standard hosts.

  • The cluster must have sufficient memory reserved for its hosts to perform maintenance. Otherwise, virtual machine migrations will hang and fail. You can reduce the memory usage of host updates by shutting down some or all virtual machines before updating hosts.

  • You cannot migrate a pinned virtual machine (such as a virtual machine using a vGPU) to another host. Pinned virtual machines are shut down during the update, unless you choose to skip that host instead.

Procedure
  1. In the Administration Portal, click Compute  Clusters and select the cluster.

  2. Click Upgrade.

  3. Select the hosts to update, then click Next.

  4. Configure the options:

    • Stop Pinned VMs shuts down any virtual machines that are pinned to hosts in the cluster, and is selected by default. You can clear this check box to skip updating those hosts so that the pinned virtual machines stay running, such as when a pinned virtual machine is running important services or processes and you do not want it to shut down at an unknown time during the update.

    • Upgrade Timeout (Minutes) sets the time to wait for an individual host to be updated before the cluster upgrade fails with a timeout. The default is 60. You can increase it for large clusters where 60 minutes might not be enough, or reduce it for small clusters where the hosts update quickly.

    • Check Upgrade checks each host for available updates before running the upgrade process. It is not selected by default, but you can select it if you need to ensure that recent updates are included, such as when you have configured the Engine to check for host updates less frequently than the default.

    • Reboot After Upgrade reboots each host after it is updated, and is selected by default. You can clear this check box to speed up the process if you are sure that there are no pending updates that require a host reboot.

    • Use Maintenance Policy sets the cluster’s scheduling policy to cluster_maintenance during the update. It is selected by default, so activity is limited and virtual machines cannot start unless they are highly available. You can clear this check box if you have a custom scheduling policy that you want to keep using during the update, but this could have unknown consequences. Ensure your custom policy is compatible with cluster upgrade activity before disabling this option.

  5. Click Next.

  6. Review the summary of the hosts and virtual machines that will be affected.

  7. Click Upgrade.

You can track the progress of host updates in the Compute  Hosts view, and in the Events section of the Notification Drawer (EventsIcon).

You can track the progress of individual virtual machine migrations in the Status column of the Compute  Virtual Machines view. In large environments, you may need to filter the results to show a particular group of virtual machines.

8.5. Changing the Cluster Compatibility Version

oVirt clusters have a compatibility version. The cluster compatibility version indicates the features of oVirt supported by all of the hosts in the cluster. The cluster compatibility is set according to the version of the least capable host operating system in the cluster.

To change the cluster compatibility version, you must have first updated all the hosts in your cluster to a level that supports your desired compatibility level. Check if there is an icon next to the host indicating an update is available.

Procedure
  1. In the Administration Portal, click Compute  Clusters.

  2. Select the cluster to change and click Edit.

  3. On the General tab, change the Compatibility Version to the desired value.

  4. Click OK. The Change Cluster Compatibility Version confirmation dialog opens.

  5. Click OK to confirm.

An error message might warn that some virtual machines and templates are incorrectly configured. To fix this error, edit each virtual machine manually. The Edit Virtual Machine window provides additional validations and warnings that show what to correct. Sometimes the issue is automatically corrected and the virtual machine’s configuration just needs to be saved again. After editing each virtual machine, you will be able to change the cluster compatibility version.

8.6. Changing Virtual Machine Cluster Compatibility

After updating a cluster’s compatibility version, you must update the cluster compatibility version of all running or suspended virtual machines by rebooting them from the Administration Portal, or using the REST API, instead of from within the guest operating system. Virtual machines that require a reboot are marked with the pending changes icon (pendingchanges).

Although you can wait to reboot the virtual machines at a convenient time, rebooting immediately is highly recommended so that the virtual machines use the latest configuration. Any virtual machine that has not been rebooted runs with the previous configuration, and subsequent configuration changes made to the virtual machine might overwrite its pending cluster compatibility changes.

Procedure
  1. In the Administration Portal, click Compute  Virtual Machines.

  2. Check which virtual machines require a reboot. In the Vms: search bar, enter the following query:

    next_run_config_exists=True

    The search results show all virtual machines with pending changes.

  3. Select each virtual machine and click Reboot.

When the virtual machine starts, the new compatibility version is automatically applied.

You cannot change the cluster compatibility version of a virtual machine snapshot that is in preview. You must first commit or undo the preview.

8.7. Changing the Data Center Compatibility Version

oVirt data centers have a compatibility version. The compatibility version indicates the version of oVirt with which the data center is intended to be compatible. All clusters in the data center must support the desired compatibility level.

To change the data center compatibility version, you must have first updated the compatibility version of all clusters and virtual machines in the data center.

Procedure
  1. In the Administration Portal, click Compute  Data Centers.

  2. Select the data center to change and click Edit.

  3. Change the Compatibility Version to the desired value.

  4. Click OK. The Change Data Center Compatibility Version confirmation dialog opens.

  5. Click OK to confirm.

If you previously upgraded to 4.2 without replacing SHA-1 certificates with SHA-256 certificates, you must do so now.

8.8. Replacing SHA-1 Certificates with SHA-256 Certificates

oVirt 4.3 uses SHA-256 signatures, which provide a more secure way to sign SSL certificates than SHA-1. Newly installed systems do not require any special steps to enable oVirt’s public key infrastructure (PKI) to use SHA-256 signatures. However, for systems upgraded from 4.1 or earlier, one of the following is required:

Preventing Warning Messages from Appearing in the Browser

  1. Log in to the Engine machine as the root user.

  2. Check whether /etc/pki/ovirt-engine/openssl.conf includes the line default_md = sha256:

    # cat /etc/pki/ovirt-engine/openssl.conf

    If it still includes default_md = sha1, back up the existing configuration and change the default to sha256:

    # cp -p /etc/pki/ovirt-engine/openssl.conf /etc/pki/ovirt-engine/openssl.conf."$(date +"%Y%m%d%H%M%S")"
    # sed -i 's/^default_md = sha1/default_md = sha256/' /etc/pki/ovirt-engine/openssl.conf
  3. Define the certificate that should be re-signed:

    # names="apache"
  4. On the Engine, re-sign the Apache certificate:

    for name in $names; do
        subject="$(
            openssl \
                x509 \
                -in /etc/pki/ovirt-engine/certs/"${name}".cer \
                -noout \
                -subject \
            | sed \
                's;subject= \(.*\);\1;' \
        )"
       /usr/share/ovirt-engine/bin/pki-enroll-pkcs12.sh \
            --name="${name}" \
            --password=mypass \
            --subject="${subject}" \
            --san=DNS:"${ENGINE_FQDN}" \
            --keep-key
    done
  5. Restart the httpd service:

    # systemctl restart httpd
  6. Connect to the Administration Portal to confirm that the warning no longer appears.

  7. If you previously imported a CA or https certificate into the browser, find the certificate(s), remove them from the browser, and reimport the new CA certificate. Install the certificate authority according to the instructions provided by your browser. To get the certificate authority’s certificate, navigate to http://your-manager-fqdn/ovirt-engine/services/pki-resource?resource=ca-certificate&format=X509-PEM-CA, replacing your-manager-fqdn with the fully qualified domain name (FQDN).

Replacing All Signed Certificates with SHA-256

  1. Log in to the Engine machine as the root user.

  2. Check whether /etc/pki/ovirt-engine/openssl.conf includes the line default_md = sha256:

    # cat /etc/pki/ovirt-engine/openssl.conf

    If it still includes default_md = sha1, back up the existing configuration and change the default to sha256:

    # cp -p /etc/pki/ovirt-engine/openssl.conf /etc/pki/ovirt-engine/openssl.conf."$(date +"%Y%m%d%H%M%S")"
    # sed -i 's/^default_md = sha1/default_md = sha256/' /etc/pki/ovirt-engine/openssl.conf
  3. Re-sign the CA certificate by backing it up and creating a new certificate in ca.pem.new:

    # cp -p /etc/pki/ovirt-engine/private/ca.pem /etc/pki/ovirt-engine/private/ca.pem."$(date +"%Y%m%d%H%M%S")"
    # openssl x509 -signkey /etc/pki/ovirt-engine/private/ca.pem -in /etc/pki/ovirt-engine/ca.pem -out /etc/pki/ovirt-engine/ca.pem.new -days 3650 -sha256
  4. Replace the existing certificate with the new certificate:

    # mv /etc/pki/ovirt-engine/ca.pem.new /etc/pki/ovirt-engine/ca.pem
  5. Define the certificates that should be re-signed:

    # names="engine apache websocket-proxy jboss imageio-proxy"

    If you replaced the oVirt Engine SSL Certificate after the upgrade, run the following instead:

    # names="engine websocket-proxy jboss imageio-proxy"

    For more details see Replacing the oVirt Engine CA Certificate in the Administration Guide.

  6. On the Engine, re-sign the certificates:

    for name in $names; do
       subject="$(
            openssl \
                x509 \
                -in /etc/pki/ovirt-engine/certs/"${name}".cer \
                -noout \
                -subject \
            | sed \
                's;subject= \(.*\);\1;' \
            )"
         /usr/share/ovirt-engine/bin/pki-enroll-pkcs12.sh \
                --name="${name}" \
                --password=mypass \
                --subject="${subject}" \
                --san=DNS:"${ENGINE_FQDN}" \
                --keep-key
    done
  7. Restart the following services:

    # systemctl restart httpd
    # systemctl restart ovirt-engine
    # systemctl restart ovirt-websocket-proxy
    # systemctl restart ovirt-imageio-proxy
  8. Connect to the Administration Portal to confirm that the warning no longer appears.

  9. If you previously imported a CA or https certificate into the browser, find the certificate(s), remove them from the browser, and reimport the new CA certificate. Install the certificate authority according to the instructions provided by your browser. To get the certificate authority’s certificate, navigate to http://your-manager-fqdn/ovirt-engine/services/pki-resource?resource=ca-certificate&format=X509-PEM-CA, replacing your-manager-fqdn with the fully qualified domain name (FQDN).

  10. Enroll the certificates on the hosts. Repeat the following procedure for each host.

    1. In the Administration Portal, click Compute  Hosts.

    2. Select the host and click Management  Maintenance.

    3. Once the host is in maintenance mode, click Installation  Enroll Certificate.

    4. Click Management  Activate.

9. Upgrading from 4.3 to oVirt 4.4

9.1. Upgrading the Engine from 4.3 to 4.4

Upgrade the oVirt Engine from 4.3 to 4.4.

Upgrading from 4.3 to 4.4 requires host reinstallation with Enterprise Linux 8. Procedure is then a migration more than an upgrade.

Procedure

For a standalone engine this means basically:

  1. backup engine data on 4.3.9 with: engine-backup --scope=all --mode=backup --file=backup.bck --log=backuplog.log

  2. copy the backup to a safe location

  3. reinstall engine host with EL 8

  4. enable repos with: dnf install https://resources.ovirt.org/pub/yum-repo/ovirt-release44.rpm

  5. dnf update (reboot if needed)

  6. enable modules with: dnf module enable -y javapackages-tools pki-deps postgresql:12 389-ds

  7. install engine rpms with: dnf install ovirt-engine

  8. restore the engine data with: engine-backup --mode=restore --file=backup.bck --log=restore.log --provision-db --provision-dwh-db --restore-permissions --provision-dwh-db

  9. run engine-setup.

You can now update the hosts.

Upgrading a Remote Database Environment Manually

10. Upgrading a Remote Database Environment from 4.0 to oVirt 4.3

The 4.0 compatibility version is not supported after oVirt 4.2. Therefore, when upgrading from oVirt 4.0 you must update the cluster and data center compatibility versions to at least 4.1 before upgrading the Engine from 4.2 to 4.3, then update the compatibility versions again after completing the Engine upgrades.

You must also update the hosts before updating the compatibility versions, but only need to do so once. The host repositories stay the same across oVirt versions, so the hosts will already be upgraded to the latest version after a single update.

Upgrading your environment from 4.0 to 4.3 involves the following steps:

10.1. Prerequisites

  • Plan for any necessary virtual machine downtime. After you update the clusters' compatibility versions during the upgrade, a new hardware configuration is automatically applied to each virtual machine once it reboots. You must reboot any running or suspended virtual machines as soon as possible to apply the configuration changes.

  • Ensure the Engine has the correct repositories enabled.

10.2. Updating the oVirt Engine

Be sure to enable the correct repositories as detailed in Prerequisites. Otherwise, the upgrade might fail.

Updates to the oVirt Engine are released through the Content Delivery Network.

Procedure
  1. Log in to the Engine machine.

  2. Check if updated packages are available:

    # engine-upgrade-check
  3. Update the setup packages:

    # yum update ovirt\*setup\*
  4. Update the oVirt Engine with the engine-setup script. The engine-setup script prompts you with some configuration questions, then stops the ovirt-engine service, downloads and installs the updated packages, backs up and updates the database, performs post-installation configuration, and starts the ovirt-engine service.

    # engine-setup

    When the script completes successfully, the following message appears:

    Execution of setup completed successfully

    The engine-setup script is also used during the oVirt Engine installation process, and it stores the configuration values supplied. During an update, the stored values are displayed when previewing the configuration, and might not be up to date if engine-config was used to update configuration after installation. For example, if engine-config was used to update SANWipeAfterDelete to true after installation, engine-setup will output "Default SAN wipe after delete: False" in the configuration preview. However, the updated values will not be overwritten by engine-setup.

    The update process might take some time. Do not stop the process before it completes.

  5. Update the base operating system and any optional packages installed on the Engine:

    # yum update

    If any kernel packages were updated, reboot the machine to complete the update.

10.3. Upgrading the Engine from 4.0 to 4.1

Upgrade the oVirt Engine from 4.0 to 4.1.

If the upgrade fails, the engine-setup command attempts to roll your oVirt Engine installation back to its previous state. For this reason, the previous version’s repositories must not be removed until after the upgrade is complete. If the upgrade fails, detailed instructions display that explain how to restore your installation.

Procedure
  1. Log in to the Engine machine.

  2. Enable the oVirt 4.1 repositories:

    All other repositories remain the same across oVirt releases.

  3. Update the setup packages:

    # yum update ovirt\*setup\*
  4. Run engine-setup and follow the prompts to upgrade the oVirt Engine:

    # engine-setup
  5. Update the base operating system:

    # yum update

    If any kernel packages were updated, reboot the machine to complete the upgrade.

10.4. Upgrading Remote Databases from PostgreSQL 9.2 to 9.5

oVirt 4.2 uses PostgreSQL 9.5 instead of PostgreSQL 9.2. If your databases are installed locally, the upgrade script will automatically upgrade them from version 9.2 to 9.5. However, if either of your databases (Engine or Data Warehouse) is installed on a separate machine, you must perform the following procedure on each remote database before upgrading the Engine.

  1. Stop the service running on the machine:

    • Stop the ovirt-engine service on the Engine machine:

      # systemctl stop ovirt-engine
    • Stop the ovirt-engine-dwh service on the Data Warehouse machine:

      # systemctl stop ovirt-engine-dwhd
  2. Enable the required repository to receive the PostgreSQL 9.5 package:

  3. Install the PostgreSQL 9.5 packages:

    # yum install rh-postgresql95 rh-postgresql95-postgresql-contrib
  4. Stop and disable the PostgreSQL 9.2 service:

    # systemctl stop postgresql
    # systemctl disable postgresql
  5. Upgrade the PostgreSQL 9.2 database to PostgreSQL 9.5:

    # scl enable rh-postgresql95 -- postgresql-setup upgrade
  6. Start and enable the rh-postgresql95-postgresql.service and check that it is running:

    # systemctl start rh-postgresql95-postgresql.service
    # systemctl enable rh-postgresql95-postgresql.service
    # systemctl status rh-postgresql95-postgresql.service

    Ensure that you see an output similar to the following:

    rh-postgresql95-postgresql.service - PostgreSQL database server
       Loaded: loaded (/usr/lib/systemd/system/rh-postgresql95-postgresql.service;
    enabled; vendor preset: disabled)
       Active: active (running) since Mon 2018-05-07 08:48:27 CEST; 1h 59min ago
  7. Log in to the database and enable the uuid-ossp extension:

    # su - postgres -c "scl enable rh-postgresql95 -- psql -d database-name"
  8. Execute the following SQL commands:

    # database-name=# DROP FUNCTION IF EXISTS uuid_generate_v1();
    # database-name=# CREATE EXTENSION "uuid-ossp";
  9. Copy the pg_hba.conf client configuration file from the 9.2 environment to your 9.5 environment:

    # cp -p /var/lib/pgsql/data/pg_hba.conf  /var/opt/rh/rh-postgresql95/lib/pgsql/data/pg_hba.conf
  10. Update the following parameters in /var/opt/rh/rh-postgresql95/lib/pgsql/data/postgresql.conf:

    listen_addresses='*'
    autovacuum_vacuum_scale_factor = 0.01
    autovacuum_analyze_scale_factor = 0.075
    autovacuum_max_workers = 6
    maintenance_work_mem = 65536
    max_connections = 150
    work_mem = 8192
  11. Restart the PostgreSQL 9.5 service to apply the configuration changes:

    # systemctl restart rh-postgresql95-postgresql.service
  12. Start the ovirt-engine-dwhd service:

    # systemctl start ovirt-engine-dwhd

You can now upgrade the Engine to 4.2.

10.5. Upgrading the Engine from 4.1 to 4.2

Upgrade the oVirt Engine from 4.1 to 4.2.

If the upgrade fails, the engine-setup command attempts to roll your oVirt Engine installation back to its previous state. For this reason, the previous version’s repositories must not be removed until after the upgrade is complete. If the upgrade fails, detailed instructions display that explain how to restore your installation.

Procedure
  1. Log in to the Engine machine.

  2. Enable the oVirt 4.2 repositories:

    # yum install https://resources.ovirt.org/pub/yum-repo/ovirt-release42.rpm

    All other repositories remain the same across oVirt releases.

  3. Update the setup packages:

    # yum update ovirt\*setup\*
  4. Run engine-setup and follow the prompts to upgrade the oVirt Engine:

    # engine-setup

    If you changed the default trust keystore password, you might get a keystore certificate error. If so, update the configuration to reflect the correct password. For more information, see RHV 4.2 - Failed to import provider certificate into the external provider keystore

  5. Update the base operating system:

    # yum update

    If any kernel packages were updated, reboot the machine to complete the upgrade.

You must now update the hosts before you can update the cluster and data center compatibility versions.

10.6. Updating Individual Hosts

Use the host upgrade manager to update individual hosts directly from the Administration Portal.

The upgrade manager only checks hosts with a status of Up or Non-operational, but not Maintenance.

Limitations
  • On oVirt Node, the update only preserves modified content in the /etc and /var directories. Modified data in other paths is overwritten during an update.

  • If migration is enabled at the cluster level, virtual machines are automatically migrated to another host in the cluster. Update a host when its usage is relatively low.

  • In a self-hosted engine environment, the Engine virtual machine can only migrate between self-hosted engine nodes in the same cluster. It cannot migrate to standard hosts.

  • The cluster must have sufficient memory reserved for its hosts to perform maintenance. Otherwise, virtual machine migrations will hang and fail. You can reduce the memory usage of host updates by shutting down some or all virtual machines before updating hosts.

  • Do not update all hosts at the same time, as one host must remain available to perform Storage Pool Manager (SPM) tasks.

  • You cannot migrate a pinned virtual machine (such as a virtual machine using a vGPU) to another host. Pinned virtual machines must be shut down before updating the host.

Procedure
  1. Ensure that the correct repositories are enabled. To view a list of currently enabled repositories, run yum repolist.

  2. In the Administration Portal, click Compute  Hosts and select the host to be updated.

  3. Click Installation  Check for Upgrade and click OK.

    Open the Notification Drawer (EventsIcon) and expand the Events section to see the result.

  4. If an update is available, click Installation  Upgrade.

  5. Click OK to update the host. Running virtual machines are migrated according to their migration policy. If migration is disabled for any virtual machines, you are prompted to shut them down.

    The details of the host are updated in Compute  Hosts and the status transitions through these stages:

    Maintenance > Installing > Reboot > Up

    If the update fails, the host’s status changes to Install Failed. From Install Failed you can click Installation  Upgrade again.

Repeat this procedure for each host in the oVirt environment.

You can now change the cluster compatibility version to 4.2.

10.7. Changing the Cluster Compatibility Version

oVirt clusters have a compatibility version. The cluster compatibility version indicates the features of oVirt supported by all of the hosts in the cluster. The cluster compatibility is set according to the version of the least capable host operating system in the cluster.

To change the cluster compatibility version, you must have first updated all the hosts in your cluster to a level that supports your desired compatibility level. Check if there is an icon next to the host indicating an update is available.

Procedure
  1. In the Administration Portal, click Compute  Clusters.

  2. Select the cluster to change and click Edit.

  3. On the General tab, change the Compatibility Version to the desired value.

  4. Click OK. The Change Cluster Compatibility Version confirmation dialog opens.

  5. Click OK to confirm.

An error message might warn that some virtual machines and templates are incorrectly configured. To fix this error, edit each virtual machine manually. The Edit Virtual Machine window provides additional validations and warnings that show what to correct. Sometimes the issue is automatically corrected and the virtual machine’s configuration just needs to be saved again. After editing each virtual machine, you will be able to change the cluster compatibility version.

10.8. Changing Virtual Machine Cluster Compatibility

After updating a cluster’s compatibility version, you must update the cluster compatibility version of all running or suspended virtual machines by rebooting them from the Administration Portal, or using the REST API, instead of from within the guest operating system. Virtual machines that require a reboot are marked with the pending changes icon (pendingchanges).

Although you can wait to reboot the virtual machines at a convenient time, rebooting immediately is highly recommended so that the virtual machines use the latest configuration. Any virtual machine that has not been rebooted runs with the previous configuration, and subsequent configuration changes made to the virtual machine might overwrite its pending cluster compatibility changes.

Procedure
  1. In the Administration Portal, click Compute  Virtual Machines.

  2. Check which virtual machines require a reboot. In the Vms: search bar, enter the following query:

    next_run_config_exists=True

    The search results show all virtual machines with pending changes.

  3. Select each virtual machine and click Reboot.

When the virtual machine starts, the new compatibility version is automatically applied.

You cannot change the cluster compatibility version of a virtual machine snapshot that is in preview. You must first commit or undo the preview.

You can now change the data center compatibility version to 4.2.

10.9. Changing the Data Center Compatibility Version

oVirt data centers have a compatibility version. The compatibility version indicates the version of oVirt with which the data center is intended to be compatible. All clusters in the data center must support the desired compatibility level.

To change the data center compatibility version, you must have first updated the compatibility version of all clusters and virtual machines in the data center.

Procedure
  1. In the Administration Portal, click Compute  Data Centers.

  2. Select the data center to change and click Edit.

  3. Change the Compatibility Version to the desired value.

  4. Click OK. The Change Data Center Compatibility Version confirmation dialog opens.

  5. Click OK to confirm.

10.10. Upgrading Remote Databases from PostgreSQL 9.5 to 10

oVirt 4.3 uses PostgreSQL 10 instead of PostgreSQL 9.5. If your databases are installed locally, the upgrade script automatically upgrades them from version 9.5 to 10. However, if either of your databases (Engine or Data Warehouse) is installed on a separate machine, you must perform the following procedure on each remote database before upgrading the Engine.

  1. Stop the service running on the machine:

    • Stop the ovirt-engine service on the Engine machine:

      # systemctl stop ovirt-engine
    • Stop the ovirt-engine-dwh service on the Data Warehouse machine:

      # systemctl stop ovirt-engine-dwhd
  2. Enable the required repository to receive the PostgreSQL 10 package:

  3. Install the PostgreSQL 10 packages:

    # yum install rh-postgresql10 rh-postgresql10-postgresql-contrib
  4. Stop and disable the PostgreSQL 9.5 service:

    # systemctl stop rh-postgresql95-postgresql
    # systemctl disable rh-postgresql95-postgresql
  5. Upgrade the PostgreSQL 9.5 database to PostgreSQL 10:

    # scl enable rh-postgresql10 -- postgresql-setup --upgrade-from=rh-postgresql95-postgresql --upgrade
  6. Start and enable the rh-postgresql10-postgresql.service and check that it is running:

    # systemctl start rh-postgresql10-postgresql.service
    # systemctl enable rh-postgresql10-postgresql.service
    # systemctl status rh-postgresql10-postgresql.service

    Ensure that you see output similar to the following:

    rh-postgresql10-postgresql.service - PostgreSQL database server
       Loaded: loaded (/usr/lib/systemd/system/rh-postgresql10-postgresql.service;
    enabled; vendor preset: disabled)
       Active: active (running) since ...
  7. Copy the pg_hba.conf client configuration file from the PostgreSQL 9.5 environment to the PostgreSQL 10 environment:

    # cp -p /var/opt/rh/rh-postgresql95/lib/pgsql/data/pg_hba.conf  /var/opt/rh/rh-postgresql10/lib/pgsql/data/pg_hba.conf
  8. Update the following parameters in /var/opt/rh/rh-postgresql10/lib/pgsql/data/postgresql.conf:

    listen_addresses='*'
    autovacuum_vacuum_scale_factor=0.01
    autovacuum_analyze_scale_factor=0.075
    autovacuum_max_workers=6
    maintenance_work_mem=65536
    max_connections=150
    work_mem = 8192
  9. Restart the PostgreSQL 10 service to apply the configuration changes:

    # systemctl restart rh-postgresql10-postgresql.service
  10. Start the ovirt-engine-dwhd service:

    # systemctl start ovirt-engine-dwhd

You can now upgrade the Engine to 4.3.

10.11. Upgrading the Engine from 4.2 to 4.3

Upgrade the oVirt Engine from 4.2 to 4.3.

If the upgrade fails, the engine-setup command attempts to roll your oVirt Engine installation back to its previous state. For this reason, the previous version’s repositories must not be removed until after the upgrade is complete. If the upgrade fails, detailed instructions display that explain how to restore your installation.

Procedure
  1. Log in to the Engine machine.

  2. Enable the oVirt 4.3 repositories:

    # yum install https://resources.ovirt.org/pub/yum-repo/ovirt-release43.rpm

    All other repositories remain the same across oVirt releases.

  3. Update the setup packages:

    # yum update ovirt\*setup\*
  4. Run engine-setup and follow the prompts to upgrade the oVirt Engine:

    # engine-setup

    When the script completes successfully, the following message appears:

    Execution of setup completed successfully
  5. Update the base operating system:

    # yum update

    If any kernel packages were updated, reboot the machine to complete the upgrade.

You can now change the cluster compatibility version to 4.3.

10.12. Changing the Cluster Compatibility Version

oVirt clusters have a compatibility version. The cluster compatibility version indicates the features of oVirt supported by all of the hosts in the cluster. The cluster compatibility is set according to the version of the least capable host operating system in the cluster.

To change the cluster compatibility version, you must have first updated all the hosts in your cluster to a level that supports your desired compatibility level. Check if there is an icon next to the host indicating an update is available.

Procedure
  1. In the Administration Portal, click Compute  Clusters.

  2. Select the cluster to change and click Edit.

  3. On the General tab, change the Compatibility Version to the desired value.

  4. Click OK. The Change Cluster Compatibility Version confirmation dialog opens.

  5. Click OK to confirm.

An error message might warn that some virtual machines and templates are incorrectly configured. To fix this error, edit each virtual machine manually. The Edit Virtual Machine window provides additional validations and warnings that show what to correct. Sometimes the issue is automatically corrected and the virtual machine’s configuration just needs to be saved again. After editing each virtual machine, you will be able to change the cluster compatibility version.

10.13. Changing Virtual Machine Cluster Compatibility

After updating a cluster’s compatibility version, you must update the cluster compatibility version of all running or suspended virtual machines by rebooting them from the Administration Portal, or using the REST API, instead of from within the guest operating system. Virtual machines that require a reboot are marked with the pending changes icon (pendingchanges).

Although you can wait to reboot the virtual machines at a convenient time, rebooting immediately is highly recommended so that the virtual machines use the latest configuration. Any virtual machine that has not been rebooted runs with the previous configuration, and subsequent configuration changes made to the virtual machine might overwrite its pending cluster compatibility changes.

Procedure
  1. In the Administration Portal, click Compute  Virtual Machines.

  2. Check which virtual machines require a reboot. In the Vms: search bar, enter the following query:

    next_run_config_exists=True

    The search results show all virtual machines with pending changes.

  3. Select each virtual machine and click Reboot.

When the virtual machine starts, the new compatibility version is automatically applied.

You cannot change the cluster compatibility version of a virtual machine snapshot that is in preview. You must first commit or undo the preview.

You can now change the data center compatibility version to 4.3.

10.14. Changing the Data Center Compatibility Version

oVirt data centers have a compatibility version. The compatibility version indicates the version of oVirt with which the data center is intended to be compatible. All clusters in the data center must support the desired compatibility level.

To change the data center compatibility version, you must have first updated the compatibility version of all clusters and virtual machines in the data center.

Procedure
  1. In the Administration Portal, click Compute  Data Centers.

  2. Select the data center to change and click Edit.

  3. Change the Compatibility Version to the desired value.

  4. Click OK. The Change Data Center Compatibility Version confirmation dialog opens.

  5. Click OK to confirm.

10.15. Replacing SHA-1 Certificates with SHA-256 Certificates

oVirt 4.3 uses SHA-256 signatures, which provide a more secure way to sign SSL certificates than SHA-1. Newly installed systems do not require any special steps to enable oVirt’s public key infrastructure (PKI) to use SHA-256 signatures. However, for systems upgraded from 4.1 or earlier, one of the following is required:

Preventing Warning Messages from Appearing in the Browser

  1. Log in to the Engine machine as the root user.

  2. Check whether /etc/pki/ovirt-engine/openssl.conf includes the line default_md = sha256:

    # cat /etc/pki/ovirt-engine/openssl.conf

    If it still includes default_md = sha1, back up the existing configuration and change the default to sha256:

    # cp -p /etc/pki/ovirt-engine/openssl.conf /etc/pki/ovirt-engine/openssl.conf."$(date +"%Y%m%d%H%M%S")"
    # sed -i 's/^default_md = sha1/default_md = sha256/' /etc/pki/ovirt-engine/openssl.conf
  3. Define the certificate that should be re-signed:

    # names="apache"
  4. On the Engine, re-sign the Apache certificate:

    for name in $names; do
        subject="$(
            openssl \
                x509 \
                -in /etc/pki/ovirt-engine/certs/"${name}".cer \
                -noout \
                -subject \
            | sed \
                's;subject= \(.*\);\1;' \
        )"
       /usr/share/ovirt-engine/bin/pki-enroll-pkcs12.sh \
            --name="${name}" \
            --password=mypass \
            --subject="${subject}" \
            --san=DNS:"${ENGINE_FQDN}" \
            --keep-key
    done
  5. Restart the httpd service:

    # systemctl restart httpd
  6. Connect to the Administration Portal to confirm that the warning no longer appears.

  7. If you previously imported a CA or https certificate into the browser, find the certificate(s), remove them from the browser, and reimport the new CA certificate. Install the certificate authority according to the instructions provided by your browser. To get the certificate authority’s certificate, navigate to http://your-manager-fqdn/ovirt-engine/services/pki-resource?resource=ca-certificate&format=X509-PEM-CA, replacing your-manager-fqdn with the fully qualified domain name (FQDN).

Replacing All Signed Certificates with SHA-256

  1. Log in to the Engine machine as the root user.

  2. Check whether /etc/pki/ovirt-engine/openssl.conf includes the line default_md = sha256:

    # cat /etc/pki/ovirt-engine/openssl.conf

    If it still includes default_md = sha1, back up the existing configuration and change the default to sha256:

    # cp -p /etc/pki/ovirt-engine/openssl.conf /etc/pki/ovirt-engine/openssl.conf."$(date +"%Y%m%d%H%M%S")"
    # sed -i 's/^default_md = sha1/default_md = sha256/' /etc/pki/ovirt-engine/openssl.conf
  3. Re-sign the CA certificate by backing it up and creating a new certificate in ca.pem.new:

    # cp -p /etc/pki/ovirt-engine/private/ca.pem /etc/pki/ovirt-engine/private/ca.pem."$(date +"%Y%m%d%H%M%S")"
    # openssl x509 -signkey /etc/pki/ovirt-engine/private/ca.pem -in /etc/pki/ovirt-engine/ca.pem -out /etc/pki/ovirt-engine/ca.pem.new -days 3650 -sha256
  4. Replace the existing certificate with the new certificate:

    # mv /etc/pki/ovirt-engine/ca.pem.new /etc/pki/ovirt-engine/ca.pem
  5. Define the certificates that should be re-signed:

    # names="engine apache websocket-proxy jboss imageio-proxy"

    If you replaced the oVirt Engine SSL Certificate after the upgrade, run the following instead:

    # names="engine websocket-proxy jboss imageio-proxy"

    For more details see Replacing the oVirt Engine CA Certificate in the Administration Guide.

  6. On the Engine, re-sign the certificates:

    for name in $names; do
       subject="$(
            openssl \
                x509 \
                -in /etc/pki/ovirt-engine/certs/"${name}".cer \
                -noout \
                -subject \
            | sed \
                's;subject= \(.*\);\1;' \
            )"
         /usr/share/ovirt-engine/bin/pki-enroll-pkcs12.sh \
                --name="${name}" \
                --password=mypass \
                --subject="${subject}" \
                --san=DNS:"${ENGINE_FQDN}" \
                --keep-key
    done
  7. Restart the following services:

    # systemctl restart httpd
    # systemctl restart ovirt-engine
    # systemctl restart ovirt-websocket-proxy
    # systemctl restart ovirt-imageio-proxy
  8. Connect to the Administration Portal to confirm that the warning no longer appears.

  9. If you previously imported a CA or https certificate into the browser, find the certificate(s), remove them from the browser, and reimport the new CA certificate. Install the certificate authority according to the instructions provided by your browser. To get the certificate authority’s certificate, navigate to http://your-manager-fqdn/ovirt-engine/services/pki-resource?resource=ca-certificate&format=X509-PEM-CA, replacing your-manager-fqdn with the fully qualified domain name (FQDN).

  10. Enroll the certificates on the hosts. Repeat the following procedure for each host.

    1. In the Administration Portal, click Compute  Hosts.

    2. Select the host and click Management  Maintenance.

    3. Once the host is in maintenance mode, click Installation  Enroll Certificate.

    4. Click Management  Activate.

11. Upgrading a Remote Database Environment from 4.1 to oVirt 4.3

Upgrading your environment from 4.1 to 4.3 involves the following steps:

11.1. Prerequisites

  • Plan for any necessary virtual machine downtime. After you update the clusters' compatibility versions during the upgrade, a new hardware configuration is automatically applied to each virtual machine once it reboots. You must reboot any running or suspended virtual machines as soon as possible to apply the configuration changes.

  • Ensure the Engine has the correct repositories enabled.

11.2. Upgrading Remote Databases from PostgreSQL 9.2 to 9.5

oVirt 4.2 uses PostgreSQL 9.5 instead of PostgreSQL 9.2. If your databases are installed locally, the upgrade script will automatically upgrade them from version 9.2 to 9.5. However, if either of your databases (Engine or Data Warehouse) is installed on a separate machine, you must perform the following procedure on each remote database before upgrading the Engine.

  1. Stop the service running on the machine:

    • Stop the ovirt-engine service on the Engine machine:

      # systemctl stop ovirt-engine
    • Stop the ovirt-engine-dwh service on the Data Warehouse machine:

      # systemctl stop ovirt-engine-dwhd
  2. Enable the required repository to receive the PostgreSQL 9.5 package:

  3. Install the PostgreSQL 9.5 packages:

    # yum install rh-postgresql95 rh-postgresql95-postgresql-contrib
  4. Stop and disable the PostgreSQL 9.2 service:

    # systemctl stop postgresql
    # systemctl disable postgresql
  5. Upgrade the PostgreSQL 9.2 database to PostgreSQL 9.5:

    # scl enable rh-postgresql95 -- postgresql-setup upgrade
  6. Start and enable the rh-postgresql95-postgresql.service and check that it is running:

    # systemctl start rh-postgresql95-postgresql.service
    # systemctl enable rh-postgresql95-postgresql.service
    # systemctl status rh-postgresql95-postgresql.service

    Ensure that you see an output similar to the following:

    rh-postgresql95-postgresql.service - PostgreSQL database server
       Loaded: loaded (/usr/lib/systemd/system/rh-postgresql95-postgresql.service;
    enabled; vendor preset: disabled)
       Active: active (running) since Mon 2018-05-07 08:48:27 CEST; 1h 59min ago
  7. Log in to the database and enable the uuid-ossp extension:

    # su - postgres -c "scl enable rh-postgresql95 -- psql -d database-name"
  8. Execute the following SQL commands:

    # database-name=# DROP FUNCTION IF EXISTS uuid_generate_v1();
    # database-name=# CREATE EXTENSION "uuid-ossp";
  9. Copy the pg_hba.conf client configuration file from the 9.2 environment to your 9.5 environment:

    # cp -p /var/lib/pgsql/data/pg_hba.conf  /var/opt/rh/rh-postgresql95/lib/pgsql/data/pg_hba.conf
  10. Update the following parameters in /var/opt/rh/rh-postgresql95/lib/pgsql/data/postgresql.conf:

    listen_addresses='*'
    autovacuum_vacuum_scale_factor = 0.01
    autovacuum_analyze_scale_factor = 0.075
    autovacuum_max_workers = 6
    maintenance_work_mem = 65536
    max_connections = 150
    work_mem = 8192
  11. Restart the PostgreSQL 9.5 service to apply the configuration changes:

    # systemctl restart rh-postgresql95-postgresql.service
  12. Start the ovirt-engine-dwhd service:

    # systemctl start ovirt-engine-dwhd

You can now update the Engine to the latest version of 4.1.

11.3. Updating the oVirt Engine

Be sure to enable the correct repositories as detailed in Prerequisites. Otherwise, the upgrade might fail.

Updates to the oVirt Engine are released through the Content Delivery Network.

Procedure
  1. Log in to the Engine machine.

  2. Check if updated packages are available:

    # engine-upgrade-check
  3. Update the setup packages:

    # yum update ovirt\*setup\*
  4. Update the oVirt Engine with the engine-setup script. The engine-setup script prompts you with some configuration questions, then stops the ovirt-engine service, downloads and installs the updated packages, backs up and updates the database, performs post-installation configuration, and starts the ovirt-engine service.

    # engine-setup

    When the script completes successfully, the following message appears:

    Execution of setup completed successfully

    The engine-setup script is also used during the oVirt Engine installation process, and it stores the configuration values supplied. During an update, the stored values are displayed when previewing the configuration, and might not be up to date if engine-config was used to update configuration after installation. For example, if engine-config was used to update SANWipeAfterDelete to true after installation, engine-setup will output "Default SAN wipe after delete: False" in the configuration preview. However, the updated values will not be overwritten by engine-setup.

    The update process might take some time. Do not stop the process before it completes.

  5. Update the base operating system and any optional packages installed on the Engine:

    # yum update

    If any kernel packages were updated, reboot the machine to complete the update.

You can now upgrade the Engine to 4.2.

11.4. Upgrading the Engine from 4.1 to 4.2

Upgrade the oVirt Engine from 4.1 to 4.2.

If the upgrade fails, the engine-setup command attempts to roll your oVirt Engine installation back to its previous state. For this reason, the previous version’s repositories must not be removed until after the upgrade is complete. If the upgrade fails, detailed instructions display that explain how to restore your installation.

Procedure
  1. Log in to the Engine machine.

  2. Enable the oVirt 4.2 repositories:

    # yum install https://resources.ovirt.org/pub/yum-repo/ovirt-release42.rpm

    All other repositories remain the same across oVirt releases.

  3. Update the setup packages:

    # yum update ovirt\*setup\*
  4. Run engine-setup and follow the prompts to upgrade the oVirt Engine:

    # engine-setup

    If you changed the default trust keystore password, you might get a keystore certificate error. If so, update the configuration to reflect the correct password. For more information, see RHV 4.2 - Failed to import provider certificate into the external provider keystore

  5. Update the base operating system:

    # yum update

    If any kernel packages were updated, reboot the machine to complete the upgrade.

11.5. Upgrading Remote Databases from PostgreSQL 9.5 to 10

oVirt 4.3 uses PostgreSQL 10 instead of PostgreSQL 9.5. If your databases are installed locally, the upgrade script automatically upgrades them from version 9.5 to 10. However, if either of your databases (Engine or Data Warehouse) is installed on a separate machine, you must perform the following procedure on each remote database before upgrading the Engine.

  1. Stop the service running on the machine:

    • Stop the ovirt-engine service on the Engine machine:

      # systemctl stop ovirt-engine
    • Stop the ovirt-engine-dwh service on the Data Warehouse machine:

      # systemctl stop ovirt-engine-dwhd
  2. Enable the required repository to receive the PostgreSQL 10 package:

  3. Install the PostgreSQL 10 packages:

    # yum install rh-postgresql10 rh-postgresql10-postgresql-contrib
  4. Stop and disable the PostgreSQL 9.5 service:

    # systemctl stop rh-postgresql95-postgresql
    # systemctl disable rh-postgresql95-postgresql
  5. Upgrade the PostgreSQL 9.5 database to PostgreSQL 10:

    # scl enable rh-postgresql10 -- postgresql-setup --upgrade-from=rh-postgresql95-postgresql --upgrade
  6. Start and enable the rh-postgresql10-postgresql.service and check that it is running:

    # systemctl start rh-postgresql10-postgresql.service
    # systemctl enable rh-postgresql10-postgresql.service
    # systemctl status rh-postgresql10-postgresql.service

    Ensure that you see output similar to the following:

    rh-postgresql10-postgresql.service - PostgreSQL database server
       Loaded: loaded (/usr/lib/systemd/system/rh-postgresql10-postgresql.service;
    enabled; vendor preset: disabled)
       Active: active (running) since ...
  7. Copy the pg_hba.conf client configuration file from the PostgreSQL 9.5 environment to the PostgreSQL 10 environment:

    # cp -p /var/opt/rh/rh-postgresql95/lib/pgsql/data/pg_hba.conf  /var/opt/rh/rh-postgresql10/lib/pgsql/data/pg_hba.conf
  8. Update the following parameters in /var/opt/rh/rh-postgresql10/lib/pgsql/data/postgresql.conf:

    listen_addresses='*'
    autovacuum_vacuum_scale_factor=0.01
    autovacuum_analyze_scale_factor=0.075
    autovacuum_max_workers=6
    maintenance_work_mem=65536
    max_connections=150
    work_mem = 8192
  9. Restart the PostgreSQL 10 service to apply the configuration changes:

    # systemctl restart rh-postgresql10-postgresql.service
  10. Start the ovirt-engine-dwhd service:

    # systemctl start ovirt-engine-dwhd

You can now upgrade the Engine to 4.3.

11.6. Upgrading the Engine from 4.2 to 4.3

Upgrade the oVirt Engine from 4.2 to 4.3.

If the upgrade fails, the engine-setup command attempts to roll your oVirt Engine installation back to its previous state. For this reason, the previous version’s repositories must not be removed until after the upgrade is complete. If the upgrade fails, detailed instructions display that explain how to restore your installation.

Procedure
  1. Log in to the Engine machine.

  2. Enable the oVirt 4.3 repositories:

    # yum install https://resources.ovirt.org/pub/yum-repo/ovirt-release43.rpm

    All other repositories remain the same across oVirt releases.

  3. Update the setup packages:

    # yum update ovirt\*setup\*
  4. Run engine-setup and follow the prompts to upgrade the oVirt Engine:

    # engine-setup

    When the script completes successfully, the following message appears:

    Execution of setup completed successfully
  5. Update the base operating system:

    # yum update

    If any kernel packages were updated, reboot the machine to complete the upgrade.

You can now update the hosts.

11.7. Updating All Hosts in a Cluster

You can update all hosts in a cluster instead of updating hosts individually. This is particularly useful during upgrades to new versions of oVirt. See https://github.com/oVirt/ovirt-ansible-cluster-upgrade/blob/master/README.md for more information about the Ansible role used to automate the updates.

oVirt recommends updating one cluster at a time.

Limitations
  • On oVirt Node, the update only preserves modified content in the /etc and /var directories. Modified data in other paths is overwritten during an update.

  • If migration is enabled at the cluster level, virtual machines are automatically migrated to another host in the cluster.

  • In a self-hosted engine environment, the Engine virtual machine can only migrate between self-hosted engine nodes in the same cluster. It cannot migrate to standard hosts.

  • The cluster must have sufficient memory reserved for its hosts to perform maintenance. Otherwise, virtual machine migrations will hang and fail. You can reduce the memory usage of host updates by shutting down some or all virtual machines before updating hosts.

  • You cannot migrate a pinned virtual machine (such as a virtual machine using a vGPU) to another host. Pinned virtual machines are shut down during the update, unless you choose to skip that host instead.

Procedure
  1. In the Administration Portal, click Compute  Clusters and select the cluster.

  2. Click Upgrade.

  3. Select the hosts to update, then click Next.

  4. Configure the options:

    • Stop Pinned VMs shuts down any virtual machines that are pinned to hosts in the cluster, and is selected by default. You can clear this check box to skip updating those hosts so that the pinned virtual machines stay running, such as when a pinned virtual machine is running important services or processes and you do not want it to shut down at an unknown time during the update.

    • Upgrade Timeout (Minutes) sets the time to wait for an individual host to be updated before the cluster upgrade fails with a timeout. The default is 60. You can increase it for large clusters where 60 minutes might not be enough, or reduce it for small clusters where the hosts update quickly.

    • Check Upgrade checks each host for available updates before running the upgrade process. It is not selected by default, but you can select it if you need to ensure that recent updates are included, such as when you have configured the Engine to check for host updates less frequently than the default.

    • Reboot After Upgrade reboots each host after it is updated, and is selected by default. You can clear this check box to speed up the process if you are sure that there are no pending updates that require a host reboot.

    • Use Maintenance Policy sets the cluster’s scheduling policy to cluster_maintenance during the update. It is selected by default, so activity is limited and virtual machines cannot start unless they are highly available. You can clear this check box if you have a custom scheduling policy that you want to keep using during the update, but this could have unknown consequences. Ensure your custom policy is compatible with cluster upgrade activity before disabling this option.

  5. Click Next.

  6. Review the summary of the hosts and virtual machines that will be affected.

  7. Click Upgrade.

You can track the progress of host updates in the Compute  Hosts view, and in the Events section of the Notification Drawer (EventsIcon).

You can track the progress of individual virtual machine migrations in the Status column of the Compute  Virtual Machines view. In large environments, you may need to filter the results to show a particular group of virtual machines.

11.8. Changing the Cluster Compatibility Version

oVirt clusters have a compatibility version. The cluster compatibility version indicates the features of oVirt supported by all of the hosts in the cluster. The cluster compatibility is set according to the version of the least capable host operating system in the cluster.

To change the cluster compatibility version, you must have first updated all the hosts in your cluster to a level that supports your desired compatibility level. Check if there is an icon next to the host indicating an update is available.

Procedure
  1. In the Administration Portal, click Compute  Clusters.

  2. Select the cluster to change and click Edit.

  3. On the General tab, change the Compatibility Version to the desired value.

  4. Click OK. The Change Cluster Compatibility Version confirmation dialog opens.

  5. Click OK to confirm.

An error message might warn that some virtual machines and templates are incorrectly configured. To fix this error, edit each virtual machine manually. The Edit Virtual Machine window provides additional validations and warnings that show what to correct. Sometimes the issue is automatically corrected and the virtual machine’s configuration just needs to be saved again. After editing each virtual machine, you will be able to change the cluster compatibility version.

11.9. Changing Virtual Machine Cluster Compatibility

After updating a cluster’s compatibility version, you must update the cluster compatibility version of all running or suspended virtual machines by rebooting them from the Administration Portal, or using the REST API, instead of from within the guest operating system. Virtual machines that require a reboot are marked with the pending changes icon (pendingchanges).

Although you can wait to reboot the virtual machines at a convenient time, rebooting immediately is highly recommended so that the virtual machines use the latest configuration. Any virtual machine that has not been rebooted runs with the previous configuration, and subsequent configuration changes made to the virtual machine might overwrite its pending cluster compatibility changes.

Procedure
  1. In the Administration Portal, click Compute  Virtual Machines.

  2. Check which virtual machines require a reboot. In the Vms: search bar, enter the following query:

    next_run_config_exists=True

    The search results show all virtual machines with pending changes.

  3. Select each virtual machine and click Reboot.

When the virtual machine starts, the new compatibility version is automatically applied.

You cannot change the cluster compatibility version of a virtual machine snapshot that is in preview. You must first commit or undo the preview.

11.10. Changing the Data Center Compatibility Version

oVirt data centers have a compatibility version. The compatibility version indicates the version of oVirt with which the data center is intended to be compatible. All clusters in the data center must support the desired compatibility level.

To change the data center compatibility version, you must have first updated the compatibility version of all clusters and virtual machines in the data center.

Procedure
  1. In the Administration Portal, click Compute  Data Centers.

  2. Select the data center to change and click Edit.

  3. Change the Compatibility Version to the desired value.

  4. Click OK. The Change Data Center Compatibility Version confirmation dialog opens.

  5. Click OK to confirm.

11.11. Replacing SHA-1 Certificates with SHA-256 Certificates

oVirt 4.3 uses SHA-256 signatures, which provide a more secure way to sign SSL certificates than SHA-1. Newly installed systems do not require any special steps to enable oVirt’s public key infrastructure (PKI) to use SHA-256 signatures. However, for systems upgraded from 4.1 or earlier, one of the following is required:

Preventing Warning Messages from Appearing in the Browser

  1. Log in to the Engine machine as the root user.

  2. Check whether /etc/pki/ovirt-engine/openssl.conf includes the line default_md = sha256:

    # cat /etc/pki/ovirt-engine/openssl.conf

    If it still includes default_md = sha1, back up the existing configuration and change the default to sha256:

    # cp -p /etc/pki/ovirt-engine/openssl.conf /etc/pki/ovirt-engine/openssl.conf."$(date +"%Y%m%d%H%M%S")"
    # sed -i 's/^default_md = sha1/default_md = sha256/' /etc/pki/ovirt-engine/openssl.conf
  3. Define the certificate that should be re-signed:

    # names="apache"
  4. On the Engine, re-sign the Apache certificate:

    for name in $names; do
        subject="$(
            openssl \
                x509 \
                -in /etc/pki/ovirt-engine/certs/"${name}".cer \
                -noout \
                -subject \
            | sed \
                's;subject= \(.*\);\1;' \
        )"
       /usr/share/ovirt-engine/bin/pki-enroll-pkcs12.sh \
            --name="${name}" \
            --password=mypass \
            --subject="${subject}" \
            --san=DNS:"${ENGINE_FQDN}" \
            --keep-key
    done
  5. Restart the httpd service:

    # systemctl restart httpd
  6. Connect to the Administration Portal to confirm that the warning no longer appears.

  7. If you previously imported a CA or https certificate into the browser, find the certificate(s), remove them from the browser, and reimport the new CA certificate. Install the certificate authority according to the instructions provided by your browser. To get the certificate authority’s certificate, navigate to http://your-manager-fqdn/ovirt-engine/services/pki-resource?resource=ca-certificate&format=X509-PEM-CA, replacing your-manager-fqdn with the fully qualified domain name (FQDN).

Replacing All Signed Certificates with SHA-256

  1. Log in to the Engine machine as the root user.

  2. Check whether /etc/pki/ovirt-engine/openssl.conf includes the line default_md = sha256:

    # cat /etc/pki/ovirt-engine/openssl.conf

    If it still includes default_md = sha1, back up the existing configuration and change the default to sha256:

    # cp -p /etc/pki/ovirt-engine/openssl.conf /etc/pki/ovirt-engine/openssl.conf."$(date +"%Y%m%d%H%M%S")"
    # sed -i 's/^default_md = sha1/default_md = sha256/' /etc/pki/ovirt-engine/openssl.conf
  3. Re-sign the CA certificate by backing it up and creating a new certificate in ca.pem.new:

    # cp -p /etc/pki/ovirt-engine/private/ca.pem /etc/pki/ovirt-engine/private/ca.pem."$(date +"%Y%m%d%H%M%S")"
    # openssl x509 -signkey /etc/pki/ovirt-engine/private/ca.pem -in /etc/pki/ovirt-engine/ca.pem -out /etc/pki/ovirt-engine/ca.pem.new -days 3650 -sha256
  4. Replace the existing certificate with the new certificate:

    # mv /etc/pki/ovirt-engine/ca.pem.new /etc/pki/ovirt-engine/ca.pem
  5. Define the certificates that should be re-signed:

    # names="engine apache websocket-proxy jboss imageio-proxy"

    If you replaced the oVirt Engine SSL Certificate after the upgrade, run the following instead:

    # names="engine websocket-proxy jboss imageio-proxy"

    For more details see Replacing the oVirt Engine CA Certificate in the Administration Guide.

  6. On the Engine, re-sign the certificates:

    for name in $names; do
       subject="$(
            openssl \
                x509 \
                -in /etc/pki/ovirt-engine/certs/"${name}".cer \
                -noout \
                -subject \
            | sed \
                's;subject= \(.*\);\1;' \
            )"
         /usr/share/ovirt-engine/bin/pki-enroll-pkcs12.sh \
                --name="${name}" \
                --password=mypass \
                --subject="${subject}" \
                --san=DNS:"${ENGINE_FQDN}" \
                --keep-key
    done
  7. Restart the following services:

    # systemctl restart httpd
    # systemctl restart ovirt-engine
    # systemctl restart ovirt-websocket-proxy
    # systemctl restart ovirt-imageio-proxy
  8. Connect to the Administration Portal to confirm that the warning no longer appears.

  9. If you previously imported a CA or https certificate into the browser, find the certificate(s), remove them from the browser, and reimport the new CA certificate. Install the certificate authority according to the instructions provided by your browser. To get the certificate authority’s certificate, navigate to http://your-manager-fqdn/ovirt-engine/services/pki-resource?resource=ca-certificate&format=X509-PEM-CA, replacing your-manager-fqdn with the fully qualified domain name (FQDN).

  10. Enroll the certificates on the hosts. Repeat the following procedure for each host.

    1. In the Administration Portal, click Compute  Hosts.

    2. Select the host and click Management  Maintenance.

    3. Once the host is in maintenance mode, click Installation  Enroll Certificate.

    4. Click Management  Activate.

11.12. Updating OVN Providers Installed in oVirt 4.1

If you installed an Open Virtual Network (OVN) provider in oVirt 4.1, you must manually edit its configuration for oVirt 4.2.

Procedure
  1. Click Administration  Providers and select the OVN provider.

  2. Click Edit.

  3. Click the Networking Plugin text field and select oVirt Network Provider for OVN from the drop-down list.

  4. Click OK.

12. Upgrading a Remote Database Environment from 4.2 to oVirt 4.3

Upgrading your environment from 4.2 to 4.3 involves the following steps:

12.1. Prerequisites

  • Plan for any necessary virtual machine downtime. After you update the clusters' compatibility versions during the upgrade, a new hardware configuration is automatically applied to each virtual machine once it reboots. You must reboot any running or suspended virtual machines as soon as possible to apply the configuration changes.

  • Ensure the Engine has the correct repositories enabled.

12.2. Upgrading Remote Databases from PostgreSQL 9.5 to 10

oVirt 4.3 uses PostgreSQL 10 instead of PostgreSQL 9.5. If your databases are installed locally, the upgrade script automatically upgrades them from version 9.5 to 10. However, if either of your databases (Engine or Data Warehouse) is installed on a separate machine, you must perform the following procedure on each remote database before upgrading the Engine.

  1. Stop the service running on the machine:

    • Stop the ovirt-engine service on the Engine machine:

      # systemctl stop ovirt-engine
    • Stop the ovirt-engine-dwh service on the Data Warehouse machine:

      # systemctl stop ovirt-engine-dwhd
  2. Enable the required repository to receive the PostgreSQL 10 package:

  3. Install the PostgreSQL 10 packages:

    # yum install rh-postgresql10 rh-postgresql10-postgresql-contrib
  4. Stop and disable the PostgreSQL 9.5 service:

    # systemctl stop rh-postgresql95-postgresql
    # systemctl disable rh-postgresql95-postgresql
  5. Upgrade the PostgreSQL 9.5 database to PostgreSQL 10:

    # scl enable rh-postgresql10 -- postgresql-setup --upgrade-from=rh-postgresql95-postgresql --upgrade
  6. Start and enable the rh-postgresql10-postgresql.service and check that it is running:

    # systemctl start rh-postgresql10-postgresql.service
    # systemctl enable rh-postgresql10-postgresql.service
    # systemctl status rh-postgresql10-postgresql.service

    Ensure that you see output similar to the following:

    rh-postgresql10-postgresql.service - PostgreSQL database server
       Loaded: loaded (/usr/lib/systemd/system/rh-postgresql10-postgresql.service;
    enabled; vendor preset: disabled)
       Active: active (running) since ...
  7. Copy the pg_hba.conf client configuration file from the PostgreSQL 9.5 environment to the PostgreSQL 10 environment:

    # cp -p /var/opt/rh/rh-postgresql95/lib/pgsql/data/pg_hba.conf  /var/opt/rh/rh-postgresql10/lib/pgsql/data/pg_hba.conf
  8. Update the following parameters in /var/opt/rh/rh-postgresql10/lib/pgsql/data/postgresql.conf:

    listen_addresses='*'
    autovacuum_vacuum_scale_factor=0.01
    autovacuum_analyze_scale_factor=0.075
    autovacuum_max_workers=6
    maintenance_work_mem=65536
    max_connections=150
    work_mem = 8192
  9. Restart the PostgreSQL 10 service to apply the configuration changes:

    # systemctl restart rh-postgresql10-postgresql.service
  10. Start the ovirt-engine-dwhd service:

    # systemctl start ovirt-engine-dwhd

You can now update the Engine to the latest version of 4.2.

12.3. Updating the oVirt Engine

Be sure to enable the correct repositories as detailed in Prerequisites. Otherwise, the upgrade might fail.

Updates to the oVirt Engine are released through the Content Delivery Network.

Procedure
  1. Log in to the Engine machine.

  2. Check if updated packages are available:

    # engine-upgrade-check
  3. Update the setup packages:

    # yum update ovirt\*setup\*
  4. Update the oVirt Engine with the engine-setup script. The engine-setup script prompts you with some configuration questions, then stops the ovirt-engine service, downloads and installs the updated packages, backs up and updates the database, performs post-installation configuration, and starts the ovirt-engine service.

    # engine-setup

    When the script completes successfully, the following message appears:

    Execution of setup completed successfully

    The engine-setup script is also used during the oVirt Engine installation process, and it stores the configuration values supplied. During an update, the stored values are displayed when previewing the configuration, and might not be up to date if engine-config was used to update configuration after installation. For example, if engine-config was used to update SANWipeAfterDelete to true after installation, engine-setup will output "Default SAN wipe after delete: False" in the configuration preview. However, the updated values will not be overwritten by engine-setup.

    The update process might take some time. Do not stop the process before it completes.

  5. Update the base operating system and any optional packages installed on the Engine:

    # yum update

    If any kernel packages were updated, reboot the machine to complete the update.

You can now upgrade the Engine to 4.3.

12.4. Upgrading the Engine from 4.2 to 4.3

Upgrade the oVirt Engine from 4.2 to 4.3.

If the upgrade fails, the engine-setup command attempts to roll your oVirt Engine installation back to its previous state. For this reason, the previous version’s repositories must not be removed until after the upgrade is complete. If the upgrade fails, detailed instructions display that explain how to restore your installation.

Procedure
  1. Log in to the Engine machine.

  2. Enable the oVirt 4.3 repositories:

    # yum install https://resources.ovirt.org/pub/yum-repo/ovirt-release43.rpm

    All other repositories remain the same across oVirt releases.

  3. Update the setup packages:

    # yum update ovirt\*setup\*
  4. Run engine-setup and follow the prompts to upgrade the oVirt Engine:

    # engine-setup

    When the script completes successfully, the following message appears:

    Execution of setup completed successfully
  5. Update the base operating system:

    # yum update

    If any kernel packages were updated, reboot the machine to complete the upgrade.

You can now update the hosts.

12.5. Updating All Hosts in a Cluster

You can update all hosts in a cluster instead of updating hosts individually. This is particularly useful during upgrades to new versions of oVirt. See https://github.com/oVirt/ovirt-ansible-cluster-upgrade/blob/master/README.md for more information about the Ansible role used to automate the updates.

oVirt recommends updating one cluster at a time.

Limitations
  • On oVirt Node, the update only preserves modified content in the /etc and /var directories. Modified data in other paths is overwritten during an update.

  • If migration is enabled at the cluster level, virtual machines are automatically migrated to another host in the cluster.

  • In a self-hosted engine environment, the Engine virtual machine can only migrate between self-hosted engine nodes in the same cluster. It cannot migrate to standard hosts.

  • The cluster must have sufficient memory reserved for its hosts to perform maintenance. Otherwise, virtual machine migrations will hang and fail. You can reduce the memory usage of host updates by shutting down some or all virtual machines before updating hosts.

  • You cannot migrate a pinned virtual machine (such as a virtual machine using a vGPU) to another host. Pinned virtual machines are shut down during the update, unless you choose to skip that host instead.

Procedure
  1. In the Administration Portal, click Compute  Clusters and select the cluster.

  2. Click Upgrade.

  3. Select the hosts to update, then click Next.

  4. Configure the options:

    • Stop Pinned VMs shuts down any virtual machines that are pinned to hosts in the cluster, and is selected by default. You can clear this check box to skip updating those hosts so that the pinned virtual machines stay running, such as when a pinned virtual machine is running important services or processes and you do not want it to shut down at an unknown time during the update.

    • Upgrade Timeout (Minutes) sets the time to wait for an individual host to be updated before the cluster upgrade fails with a timeout. The default is 60. You can increase it for large clusters where 60 minutes might not be enough, or reduce it for small clusters where the hosts update quickly.

    • Check Upgrade checks each host for available updates before running the upgrade process. It is not selected by default, but you can select it if you need to ensure that recent updates are included, such as when you have configured the Engine to check for host updates less frequently than the default.

    • Reboot After Upgrade reboots each host after it is updated, and is selected by default. You can clear this check box to speed up the process if you are sure that there are no pending updates that require a host reboot.

    • Use Maintenance Policy sets the cluster’s scheduling policy to cluster_maintenance during the update. It is selected by default, so activity is limited and virtual machines cannot start unless they are highly available. You can clear this check box if you have a custom scheduling policy that you want to keep using during the update, but this could have unknown consequences. Ensure your custom policy is compatible with cluster upgrade activity before disabling this option.

  5. Click Next.

  6. Review the summary of the hosts and virtual machines that will be affected.

  7. Click Upgrade.

You can track the progress of host updates in the Compute  Hosts view, and in the Events section of the Notification Drawer (EventsIcon).

You can track the progress of individual virtual machine migrations in the Status column of the Compute  Virtual Machines view. In large environments, you may need to filter the results to show a particular group of virtual machines.

12.6. Changing the Cluster Compatibility Version

oVirt clusters have a compatibility version. The cluster compatibility version indicates the features of oVirt supported by all of the hosts in the cluster. The cluster compatibility is set according to the version of the least capable host operating system in the cluster.

To change the cluster compatibility version, you must have first updated all the hosts in your cluster to a level that supports your desired compatibility level. Check if there is an icon next to the host indicating an update is available.

Procedure
  1. In the Administration Portal, click Compute  Clusters.

  2. Select the cluster to change and click Edit.

  3. On the General tab, change the Compatibility Version to the desired value.

  4. Click OK. The Change Cluster Compatibility Version confirmation dialog opens.

  5. Click OK to confirm.

An error message might warn that some virtual machines and templates are incorrectly configured. To fix this error, edit each virtual machine manually. The Edit Virtual Machine window provides additional validations and warnings that show what to correct. Sometimes the issue is automatically corrected and the virtual machine’s configuration just needs to be saved again. After editing each virtual machine, you will be able to change the cluster compatibility version.

12.7. Changing Virtual Machine Cluster Compatibility

After updating a cluster’s compatibility version, you must update the cluster compatibility version of all running or suspended virtual machines by rebooting them from the Administration Portal, or using the REST API, instead of from within the guest operating system. Virtual machines that require a reboot are marked with the pending changes icon (pendingchanges).

Although you can wait to reboot the virtual machines at a convenient time, rebooting immediately is highly recommended so that the virtual machines use the latest configuration. Any virtual machine that has not been rebooted runs with the previous configuration, and subsequent configuration changes made to the virtual machine might overwrite its pending cluster compatibility changes.

Procedure
  1. In the Administration Portal, click Compute  Virtual Machines.

  2. Check which virtual machines require a reboot. In the Vms: search bar, enter the following query:

    next_run_config_exists=True

    The search results show all virtual machines with pending changes.

  3. Select each virtual machine and click Reboot.

When the virtual machine starts, the new compatibility version is automatically applied.

You cannot change the cluster compatibility version of a virtual machine snapshot that is in preview. You must first commit or undo the preview.

12.8. Changing the Data Center Compatibility Version

oVirt data centers have a compatibility version. The compatibility version indicates the version of oVirt with which the data center is intended to be compatible. All clusters in the data center must support the desired compatibility level.

To change the data center compatibility version, you must have first updated the compatibility version of all clusters and virtual machines in the data center.

Procedure
  1. In the Administration Portal, click Compute  Data Centers.

  2. Select the data center to change and click Edit.

  3. Change the Compatibility Version to the desired value.

  4. Click OK. The Change Data Center Compatibility Version confirmation dialog opens.

  5. Click OK to confirm.

If you previously upgraded to 4.2 without replacing SHA-1 certificates with SHA-256 certificates, you must do so now.

12.9. Replacing SHA-1 Certificates with SHA-256 Certificates

oVirt 4.3 uses SHA-256 signatures, which provide a more secure way to sign SSL certificates than SHA-1. Newly installed systems do not require any special steps to enable oVirt’s public key infrastructure (PKI) to use SHA-256 signatures. However, for systems upgraded from 4.1 or earlier, one of the following is required:

Preventing Warning Messages from Appearing in the Browser

  1. Log in to the Engine machine as the root user.

  2. Check whether /etc/pki/ovirt-engine/openssl.conf includes the line default_md = sha256:

    # cat /etc/pki/ovirt-engine/openssl.conf

    If it still includes default_md = sha1, back up the existing configuration and change the default to sha256:

    # cp -p /etc/pki/ovirt-engine/openssl.conf /etc/pki/ovirt-engine/openssl.conf."$(date +"%Y%m%d%H%M%S")"
    # sed -i 's/^default_md = sha1/default_md = sha256/' /etc/pki/ovirt-engine/openssl.conf
  3. Define the certificate that should be re-signed:

    # names="apache"
  4. On the Engine, re-sign the Apache certificate:

    for name in $names; do
        subject="$(
            openssl \
                x509 \
                -in /etc/pki/ovirt-engine/certs/"${name}".cer \
                -noout \
                -subject \
            | sed \
                's;subject= \(.*\);\1;' \
        )"
       /usr/share/ovirt-engine/bin/pki-enroll-pkcs12.sh \
            --name="${name}" \
            --password=mypass \
            --subject="${subject}" \
            --san=DNS:"${ENGINE_FQDN}" \
            --keep-key
    done
  5. Restart the httpd service:

    # systemctl restart httpd
  6. Connect to the Administration Portal to confirm that the warning no longer appears.

  7. If you previously imported a CA or https certificate into the browser, find the certificate(s), remove them from the browser, and reimport the new CA certificate. Install the certificate authority according to the instructions provided by your browser. To get the certificate authority’s certificate, navigate to http://your-manager-fqdn/ovirt-engine/services/pki-resource?resource=ca-certificate&format=X509-PEM-CA, replacing your-manager-fqdn with the fully qualified domain name (FQDN).

Replacing All Signed Certificates with SHA-256

  1. Log in to the Engine machine as the root user.

  2. Check whether /etc/pki/ovirt-engine/openssl.conf includes the line default_md = sha256:

    # cat /etc/pki/ovirt-engine/openssl.conf

    If it still includes default_md = sha1, back up the existing configuration and change the default to sha256:

    # cp -p /etc/pki/ovirt-engine/openssl.conf /etc/pki/ovirt-engine/openssl.conf."$(date +"%Y%m%d%H%M%S")"
    # sed -i 's/^default_md = sha1/default_md = sha256/' /etc/pki/ovirt-engine/openssl.conf
  3. Re-sign the CA certificate by backing it up and creating a new certificate in ca.pem.new:

    # cp -p /etc/pki/ovirt-engine/private/ca.pem /etc/pki/ovirt-engine/private/ca.pem."$(date +"%Y%m%d%H%M%S")"
    # openssl x509 -signkey /etc/pki/ovirt-engine/private/ca.pem -in /etc/pki/ovirt-engine/ca.pem -out /etc/pki/ovirt-engine/ca.pem.new -days 3650 -sha256
  4. Replace the existing certificate with the new certificate:

    # mv /etc/pki/ovirt-engine/ca.pem.new /etc/pki/ovirt-engine/ca.pem
  5. Define the certificates that should be re-signed:

    # names="engine apache websocket-proxy jboss imageio-proxy"

    If you replaced the oVirt Engine SSL Certificate after the upgrade, run the following instead:

    # names="engine websocket-proxy jboss imageio-proxy"

    For more details see Replacing the oVirt Engine CA Certificate in the Administration Guide.

  6. On the Engine, re-sign the certificates:

    for name in $names; do
       subject="$(
            openssl \
                x509 \
                -in /etc/pki/ovirt-engine/certs/"${name}".cer \
                -noout \
                -subject \
            | sed \
                's;subject= \(.*\);\1;' \
            )"
         /usr/share/ovirt-engine/bin/pki-enroll-pkcs12.sh \
                --name="${name}" \
                --password=mypass \
                --subject="${subject}" \
                --san=DNS:"${ENGINE_FQDN}" \
                --keep-key
    done
  7. Restart the following services:

    # systemctl restart httpd
    # systemctl restart ovirt-engine
    # systemctl restart ovirt-websocket-proxy
    # systemctl restart ovirt-imageio-proxy
  8. Connect to the Administration Portal to confirm that the warning no longer appears.

  9. If you previously imported a CA or https certificate into the browser, find the certificate(s), remove them from the browser, and reimport the new CA certificate. Install the certificate authority according to the instructions provided by your browser. To get the certificate authority’s certificate, navigate to http://your-manager-fqdn/ovirt-engine/services/pki-resource?resource=ca-certificate&format=X509-PEM-CA, replacing your-manager-fqdn with the fully qualified domain name (FQDN).

  10. Enroll the certificates on the hosts. Repeat the following procedure for each host.

    1. In the Administration Portal, click Compute  Hosts.

    2. Select the host and click Management  Maintenance.

    3. Once the host is in maintenance mode, click Installation  Enroll Certificate.

    4. Click Management  Activate.

Upgrading a Self-hosted Engine Environment Manually

13. Upgrading a Self-Hosted Engine from 4.0 to oVirt 4.3

The 4.0 compatibility version is not supported after oVirt 4.2. Therefore, when upgrading from oVirt 4.0 you must update the cluster and data center compatibility versions to at least 4.1 before upgrading the Engine from 4.2 to 4.3, then update the compatibility versions again after completing the Engine upgrades.

You must also update the hosts before updating the compatibility versions, but only need to do so once. The host repositories stay the same across oVirt versions, so the hosts will already be upgraded to the latest version after a single update.

Upgrading your environment from 4.0 to 4.3 involves the following steps:

13.1. Prerequisites

  • Plan for any necessary virtual machine downtime. After you update the clusters' compatibility versions during the upgrade, a new hardware configuration is automatically applied to each virtual machine once it reboots. You must reboot any running or suspended virtual machines as soon as possible to apply the configuration changes.

  • Ensure the Engine has the correct repositories enabled.

13.2. Enabling Global Maintenance Mode

You must place the self-hosted engine environment in global maintenance mode before performing any setup or upgrade tasks on the Engine virtual machine.

Procedure
  1. Log in to one of the self-hosted engine nodes and enable global maintenance mode:

    # hosted-engine --set-maintenance --mode=global
  2. Confirm that the environment is in maintenance mode before proceeding:

    # hosted-engine --vm-status

    You should see a message indicating that the cluster is in maintenance mode.

13.3. Updating the oVirt Engine

Be sure to enable the correct repositories as detailed in Prerequisites. Otherwise, the upgrade might fail.

Updates to the oVirt Engine are released through the Content Delivery Network.

Procedure
  1. Log in to the Engine virtual machine.

  2. Check if updated packages are available:

    # engine-upgrade-check
  3. Update the setup packages:

    # yum update ovirt\*setup\*
  4. Update the oVirt Engine with the engine-setup script. The engine-setup script prompts you with some configuration questions, then stops the ovirt-engine service, downloads and installs the updated packages, backs up and updates the database, performs post-installation configuration, and starts the ovirt-engine service.

    # engine-setup

    When the script completes successfully, the following message appears:

    Execution of setup completed successfully

    The engine-setup script is also used during the oVirt Engine installation process, and it stores the configuration values supplied. During an update, the stored values are displayed when previewing the configuration, and might not be up to date if engine-config was used to update configuration after installation. For example, if engine-config was used to update SANWipeAfterDelete to true after installation, engine-setup will output "Default SAN wipe after delete: False" in the configuration preview. However, the updated values will not be overwritten by engine-setup.

    The update process might take some time. Do not stop the process before it completes.

  5. Update the base operating system and any optional packages installed on the Engine:

    # yum update

    If any kernel packages were updated, reboot the machine to complete the update.

13.4. Upgrading the Engine from 4.0 to 4.1

Upgrade the oVirt Engine from 4.0 to 4.1.

If the upgrade fails, the engine-setup command attempts to roll your oVirt Engine installation back to its previous state. For this reason, the previous version’s repositories must not be removed until after the upgrade is complete. If the upgrade fails, detailed instructions display that explain how to restore your installation.

Procedure
  1. Log in to the Engine virtual machine.

  2. Enable the oVirt 4.1 repositories:

    All other repositories remain the same across oVirt releases.

  3. Update the setup packages:

    # yum update ovirt\*setup\*
  4. Run engine-setup and follow the prompts to upgrade the oVirt Engine:

    # engine-setup
  5. Update the base operating system:

    # yum update

    If any kernel packages were updated, reboot the machine to complete the upgrade.

13.5. Upgrading the Engine from 4.1 to 4.2

Upgrade the oVirt Engine from 4.1 to 4.2.

If the upgrade fails, the engine-setup command attempts to roll your oVirt Engine installation back to its previous state. For this reason, the previous version’s repositories must not be removed until after the upgrade is complete. If the upgrade fails, detailed instructions display that explain how to restore your installation.

Procedure
  1. Log in to the Engine virtual machine.

  2. Enable the oVirt 4.2 repositories:

    # yum install https://resources.ovirt.org/pub/yum-repo/ovirt-release42.rpm

    All other repositories remain the same across oVirt releases.

  3. Update the setup packages:

    # yum update ovirt\*setup\*
  4. Run engine-setup and follow the prompts to upgrade the oVirt Engine:

    # engine-setup

    If you changed the default trust keystore password, you might get a keystore certificate error. If so, update the configuration to reflect the correct password. For more information, see RHV 4.2 - Failed to import provider certificate into the external provider keystore

  5. Update the base operating system:

    # yum update

    If any kernel packages were updated, reboot the machine to complete the upgrade.

13.6. Disabling Global Maintenance Mode

Procedure
  1. Log in to the Engine virtual machine and shut it down.

  2. Log in to one of the self-hosted engine nodes and disable global maintenance mode:

    # hosted-engine --set-maintenance --mode=none

    When you exit global maintenance mode, ovirt-ha-agent starts the Engine virtual machine, and then the Engine automatically starts. It can take up to ten minutes for the Engine to start.

  3. Confirm that the environment is running:

    # hosted-engine --vm-status

    The listed information includes Engine Status. The value for Engine status should be:

    {"health": "good", "vm": "up", "detail": "Up"}

    When the virtual machine is still booting and the Engine hasn’t started yet, the Engine status is:

    {"reason": "bad vm status", "health": "bad", "vm": "up", "detail": "Powering up"}

    If this happens, wait a few minutes and try again.

You must now update the hosts before you can update the cluster and data center compatibility versions. Update the self-hosted engine nodes first, and then any standard hosts. The procedure is the same for both host types.

13.7. Updating Individual Hosts

Use the host upgrade manager to update individual hosts directly from the Administration Portal.

The upgrade manager only checks hosts with a status of Up or Non-operational, but not Maintenance.

Limitations
  • On oVirt Node, the update only preserves modified content in the /etc and /var directories. Modified data in other paths is overwritten during an update.

  • If migration is enabled at the cluster level, virtual machines are automatically migrated to another host in the cluster. Update a host when its usage is relatively low.

  • In a self-hosted engine environment, the Engine virtual machine can only migrate between self-hosted engine nodes in the same cluster. It cannot migrate to standard hosts.

  • The cluster must have sufficient memory reserved for its hosts to perform maintenance. Otherwise, virtual machine migrations will hang and fail. You can reduce the memory usage of host updates by shutting down some or all virtual machines before updating hosts.

  • Do not update all hosts at the same time, as one host must remain available to perform Storage Pool Manager (SPM) tasks.

  • You cannot migrate a pinned virtual machine (such as a virtual machine using a vGPU) to another host. Pinned virtual machines must be shut down before updating the host.

Procedure
  1. Ensure that the correct repositories are enabled. To view a list of currently enabled repositories, run yum repolist.

  2. In the Administration Portal, click Compute  Hosts and select the host to be updated.

  3. Click Installation  Check for Upgrade and click OK.

    Open the Notification Drawer (EventsIcon) and expand the Events section to see the result.

  4. If an update is available, click Installation  Upgrade.

  5. Click OK to update the host. Running virtual machines are migrated according to their migration policy. If migration is disabled for any virtual machines, you are prompted to shut them down.

    The details of the host are updated in Compute  Hosts and the status transitions through these stages:

    Maintenance > Installing > Reboot > Up

    If the update fails, the host’s status changes to Install Failed. From Install Failed you can click Installation  Upgrade again.

Repeat this procedure for each host in the oVirt environment.

You can now change the cluster compatibility version to 4.2.

13.8. Changing the Cluster Compatibility Version

oVirt clusters have a compatibility version. The cluster compatibility version indicates the features of oVirt supported by all of the hosts in the cluster. The cluster compatibility is set according to the version of the least capable host operating system in the cluster.

To change the cluster compatibility version, you must have first updated all the hosts in your cluster to a level that supports your desired compatibility level. Check if there is an icon next to the host indicating an update is available.

Procedure
  1. In the Administration Portal, click Compute  Clusters.

  2. Select the cluster to change and click Edit.

  3. On the General tab, change the Compatibility Version to the desired value.

  4. Click OK. The Change Cluster Compatibility Version confirmation dialog opens.

  5. Click OK to confirm.

An error message might warn that some virtual machines and templates are incorrectly configured. To fix this error, edit each virtual machine manually. The Edit Virtual Machine window provides additional validations and warnings that show what to correct. Sometimes the issue is automatically corrected and the virtual machine’s configuration just needs to be saved again. After editing each virtual machine, you will be able to change the cluster compatibility version.

13.9. Changing Virtual Machine Cluster Compatibility

After updating a cluster’s compatibility version, you must update the cluster compatibility version of all running or suspended virtual machines by rebooting them from the Administration Portal, or using the REST API, instead of from within the guest operating system. Virtual machines that require a reboot are marked with the pending changes icon (pendingchanges).

The Engine virtual machine does not need to be rebooted.

Although you can wait to reboot the virtual machines at a convenient time, rebooting immediately is highly recommended so that the virtual machines use the latest configuration. Any virtual machine that has not been rebooted runs with the previous configuration, and subsequent configuration changes made to the virtual machine might overwrite its pending cluster compatibility changes.

Procedure
  1. In the Administration Portal, click Compute  Virtual Machines.

  2. Check which virtual machines require a reboot. In the Vms: search bar, enter the following query:

    next_run_config_exists=True

    The search results show all virtual machines with pending changes.

  3. Select each virtual machine and click Reboot.

When the virtual machine starts, the new compatibility version is automatically applied.

You cannot change the cluster compatibility version of a virtual machine snapshot that is in preview. You must first commit or undo the preview.

You can now change the data center compatibility version to 4.2.

13.10. Changing the Data Center Compatibility Version

oVirt data centers have a compatibility version. The compatibility version indicates the version of oVirt with which the data center is intended to be compatible. All clusters in the data center must support the desired compatibility level.

To change the data center compatibility version, you must have first updated the compatibility version of all clusters and virtual machines in the data center.

Procedure
  1. In the Administration Portal, click Compute  Data Centers.

  2. Select the data center to change and click Edit.

  3. Change the Compatibility Version to the desired value.

  4. Click OK. The Change Data Center Compatibility Version confirmation dialog opens.

  5. Click OK to confirm.

13.11. Enabling Global Maintenance Mode

You must place the self-hosted engine environment in global maintenance mode before performing any setup or upgrade tasks on the Engine virtual machine.

Procedure
  1. Log in to one of the self-hosted engine nodes and enable global maintenance mode:

    # hosted-engine --set-maintenance --mode=global
  2. Confirm that the environment is in maintenance mode before proceeding:

    # hosted-engine --vm-status

    You should see a message indicating that the cluster is in maintenance mode.

13.12. Upgrading the Engine from 4.2 to 4.3

Upgrade the oVirt Engine from 4.2 to 4.3.

If the upgrade fails, the engine-setup command attempts to roll your oVirt Engine installation back to its previous state. For this reason, the previous version’s repositories must not be removed until after the upgrade is complete. If the upgrade fails, detailed instructions display that explain how to restore your installation.

Procedure
  1. Log in to the Engine virtual machine.

  2. Enable the oVirt 4.3 repositories:

    # yum install https://resources.ovirt.org/pub/yum-repo/ovirt-release43.rpm

    All other repositories remain the same across oVirt releases.

  3. Update the setup packages:

    # yum update ovirt\*setup\*
  4. Run engine-setup and follow the prompts to upgrade the oVirt Engine:

    # engine-setup

    When the script completes successfully, the following message appears:

    Execution of setup completed successfully
  5. Update the base operating system:

    # yum update

    If any kernel packages were updated, reboot the machine to complete the upgrade.

13.13. Disabling Global Maintenance Mode

Procedure
  1. Log in to the Engine virtual machine and shut it down.

  2. Log in to one of the self-hosted engine nodes and disable global maintenance mode:

    # hosted-engine --set-maintenance --mode=none

    When you exit global maintenance mode, ovirt-ha-agent starts the Engine virtual machine, and then the Engine automatically starts. It can take up to ten minutes for the Engine to start.

  3. Confirm that the environment is running:

    # hosted-engine --vm-status

    The listed information includes Engine Status. The value for Engine status should be:

    {"health": "good", "vm": "up", "detail": "Up"}

    When the virtual machine is still booting and the Engine hasn’t started yet, the Engine status is:

    {"reason": "bad vm status", "health": "bad", "vm": "up", "detail": "Powering up"}

    If this happens, wait a few minutes and try again.

13.14. Changing the Cluster Compatibility Version

oVirt clusters have a compatibility version. The cluster compatibility version indicates the features of oVirt supported by all of the hosts in the cluster. The cluster compatibility is set according to the version of the least capable host operating system in the cluster.

To change the cluster compatibility version, you must have first updated all the hosts in your cluster to a level that supports your desired compatibility level. Check if there is an icon next to the host indicating an update is available.

Procedure
  1. In the Administration Portal, click Compute  Clusters.

  2. Select the cluster to change and click Edit.

  3. On the General tab, change the Compatibility Version to the desired value.

  4. Click OK. The Change Cluster Compatibility Version confirmation dialog opens.

  5. Click OK to confirm.

An error message might warn that some virtual machines and templates are incorrectly configured. To fix this error, edit each virtual machine manually. The Edit Virtual Machine window provides additional validations and warnings that show what to correct. Sometimes the issue is automatically corrected and the virtual machine’s configuration just needs to be saved again. After editing each virtual machine, you will be able to change the cluster compatibility version.

13.15. Changing Virtual Machine Cluster Compatibility

After updating a cluster’s compatibility version, you must update the cluster compatibility version of all running or suspended virtual machines by rebooting them from the Administration Portal, or using the REST API, instead of from within the guest operating system. Virtual machines that require a reboot are marked with the pending changes icon (pendingchanges).

The Engine virtual machine does not need to be rebooted.

Although you can wait to reboot the virtual machines at a convenient time, rebooting immediately is highly recommended so that the virtual machines use the latest configuration. Any virtual machine that has not been rebooted runs with the previous configuration, and subsequent configuration changes made to the virtual machine might overwrite its pending cluster compatibility changes.

Procedure
  1. In the Administration Portal, click Compute  Virtual Machines.

  2. Check which virtual machines require a reboot. In the Vms: search bar, enter the following query:

    next_run_config_exists=True

    The search results show all virtual machines with pending changes.

  3. Select each virtual machine and click Reboot.

When the virtual machine starts, the new compatibility version is automatically applied.

You cannot change the cluster compatibility version of a virtual machine snapshot that is in preview. You must first commit or undo the preview.

13.16. Changing the Data Center Compatibility Version

oVirt data centers have a compatibility version. The compatibility version indicates the version of oVirt with which the data center is intended to be compatible. All clusters in the data center must support the desired compatibility level.

To change the data center compatibility version, you must have first updated the compatibility version of all clusters and virtual machines in the data center.

Procedure
  1. In the Administration Portal, click Compute  Data Centers.

  2. Select the data center to change and click Edit.

  3. Change the Compatibility Version to the desired value.

  4. Click OK. The Change Data Center Compatibility Version confirmation dialog opens.

  5. Click OK to confirm.

13.17. Replacing SHA-1 Certificates with SHA-256 Certificates

oVirt 4.3 uses SHA-256 signatures, which provide a more secure way to sign SSL certificates than SHA-1. Newly installed systems do not require any special steps to enable oVirt’s public key infrastructure (PKI) to use SHA-256 signatures. However, for systems upgraded from 4.1 or earlier, one of the following is required:

Preventing Warning Messages from Appearing in the Browser

  1. Log in to the Engine machine as the root user.

  2. Check whether /etc/pki/ovirt-engine/openssl.conf includes the line default_md = sha256:

    # cat /etc/pki/ovirt-engine/openssl.conf

    If it still includes default_md = sha1, back up the existing configuration and change the default to sha256:

    # cp -p /etc/pki/ovirt-engine/openssl.conf /etc/pki/ovirt-engine/openssl.conf."$(date +"%Y%m%d%H%M%S")"
    # sed -i 's/^default_md = sha1/default_md = sha256/' /etc/pki/ovirt-engine/openssl.conf
  3. Define the certificate that should be re-signed:

    # names="apache"
  4. Log in to one of the self-hosted engine nodes and enable global maintenance:

    # hosted-engine --set-maintenance --mode=global
  5. On the Engine, re-sign the Apache certificate:

    for name in $names; do
        subject="$(
            openssl \
                x509 \
                -in /etc/pki/ovirt-engine/certs/"${name}".cer \
                -noout \
                -subject \
            | sed \
                's;subject= \(.*\);\1;' \
        )"
       /usr/share/ovirt-engine/bin/pki-enroll-pkcs12.sh \
            --name="${name}" \
            --password=mypass \
            --subject="${subject}" \
            --san=DNS:"${ENGINE_FQDN}" \
            --keep-key
    done
  6. Restart the httpd service:

    # systemctl restart httpd
  7. Log in to one of the self-hosted engine nodes and disable global maintenance:

    # hosted-engine --set-maintenance --mode=none
  8. Connect to the Administration Portal to confirm that the warning no longer appears.

  9. If you previously imported a CA or https certificate into the browser, find the certificate(s), remove them from the browser, and reimport the new CA certificate. Install the certificate authority according to the instructions provided by your browser. To get the certificate authority’s certificate, navigate to http://your-manager-fqdn/ovirt-engine/services/pki-resource?resource=ca-certificate&format=X509-PEM-CA, replacing your-manager-fqdn with the fully qualified domain name (FQDN).

Replacing All Signed Certificates with SHA-256

  1. Log in to the Engine machine as the root user.

  2. Check whether /etc/pki/ovirt-engine/openssl.conf includes the line default_md = sha256:

    # cat /etc/pki/ovirt-engine/openssl.conf

    If it still includes default_md = sha1, back up the existing configuration and change the default to sha256:

    # cp -p /etc/pki/ovirt-engine/openssl.conf /etc/pki/ovirt-engine/openssl.conf."$(date +"%Y%m%d%H%M%S")"
    # sed -i 's/^default_md = sha1/default_md = sha256/' /etc/pki/ovirt-engine/openssl.conf
  3. Re-sign the CA certificate by backing it up and creating a new certificate in ca.pem.new:

    # cp -p /etc/pki/ovirt-engine/private/ca.pem /etc/pki/ovirt-engine/private/ca.pem."$(date +"%Y%m%d%H%M%S")"
    # openssl x509 -signkey /etc/pki/ovirt-engine/private/ca.pem -in /etc/pki/ovirt-engine/ca.pem -out /etc/pki/ovirt-engine/ca.pem.new -days 3650 -sha256
  4. Replace the existing certificate with the new certificate:

    # mv /etc/pki/ovirt-engine/ca.pem.new /etc/pki/ovirt-engine/ca.pem
  5. Define the certificates that should be re-signed:

    # names="engine apache websocket-proxy jboss imageio-proxy"

    If you replaced the oVirt Engine SSL Certificate after the upgrade, run the following instead:

    # names="engine websocket-proxy jboss imageio-proxy"

    For more details see Replacing the oVirt Engine CA Certificate in the Administration Guide.

  6. Log in to one of the self-hosted engine nodes and enable global maintenance:

    # hosted-engine --set-maintenance --mode=global
  7. On the Engine, re-sign the certificates:

    for name in $names; do
       subject="$(
            openssl \
                x509 \
                -in /etc/pki/ovirt-engine/certs/"${name}".cer \
                -noout \
                -subject \
            | sed \
                's;subject= \(.*\);\1;' \
            )"
         /usr/share/ovirt-engine/bin/pki-enroll-pkcs12.sh \
                --name="${name}" \
                --password=mypass \
                --subject="${subject}" \
                --san=DNS:"${ENGINE_FQDN}" \
                --keep-key
    done
  8. Restart the following services:

    # systemctl restart httpd
    # systemctl restart ovirt-engine
    # systemctl restart ovirt-websocket-proxy
    # systemctl restart ovirt-imageio-proxy
  9. Log in to one of the self-hosted engine nodes and disable global maintenance:

    # hosted-engine --set-maintenance --mode=none
  10. Connect to the Administration Portal to confirm that the warning no longer appears.

  11. If you previously imported a CA or https certificate into the browser, find the certificate(s), remove them from the browser, and reimport the new CA certificate. Install the certificate authority according to the instructions provided by your browser. To get the certificate authority’s certificate, navigate to http://your-manager-fqdn/ovirt-engine/services/pki-resource?resource=ca-certificate&format=X509-PEM-CA, replacing your-manager-fqdn with the fully qualified domain name (FQDN).

  12. Enroll the certificates on the hosts. Repeat the following procedure for each host.

    1. In the Administration Portal, click Compute  Hosts.

    2. Select the host and click Management  Maintenance.

    3. Once the host is in maintenance mode, click Installation  Enroll Certificate.

    4. Click Management  Activate.

14. Upgrading a Self-Hosted Engine from 4.1 to oVirt 4.3

Upgrading a self-hosted engine environment from version 4.1 to 4.2 involves the following steps:

14.1. Prerequisites

  • Plan for any necessary virtual machine downtime. After you update the clusters' compatibility versions during the upgrade, a new hardware configuration is automatically applied to each virtual machine once it reboots. You must reboot any running or suspended virtual machines as soon as possible to apply the configuration changes.

  • Ensure the Engine has the correct repositories enabled.

14.2. Enabling Global Maintenance Mode

You must place the self-hosted engine environment in global maintenance mode before performing any setup or upgrade tasks on the Engine virtual machine.

Procedure
  1. Log in to one of the self-hosted engine nodes and enable global maintenance mode:

    # hosted-engine --set-maintenance --mode=global
  2. Confirm that the environment is in maintenance mode before proceeding:

    # hosted-engine --vm-status

    You should see a message indicating that the cluster is in maintenance mode.

14.3. Updating the oVirt Engine

Be sure to enable the correct repositories as detailed in Prerequisites. Otherwise, the upgrade might fail.

Updates to the oVirt Engine are released through the Content Delivery Network.

Procedure
  1. Log in to the Engine virtual machine.

  2. Check if updated packages are available:

    # engine-upgrade-check
  3. Update the setup packages:

    # yum update ovirt\*setup\*
  4. Update the oVirt Engine with the engine-setup script. The engine-setup script prompts you with some configuration questions, then stops the ovirt-engine service, downloads and installs the updated packages, backs up and updates the database, performs post-installation configuration, and starts the ovirt-engine service.

    # engine-setup

    When the script completes successfully, the following message appears:

    Execution of setup completed successfully

    The engine-setup script is also used during the oVirt Engine installation process, and it stores the configuration values supplied. During an update, the stored values are displayed when previewing the configuration, and might not be up to date if engine-config was used to update configuration after installation. For example, if engine-config was used to update SANWipeAfterDelete to true after installation, engine-setup will output "Default SAN wipe after delete: False" in the configuration preview. However, the updated values will not be overwritten by engine-setup.

    The update process might take some time. Do not stop the process before it completes.

  5. Update the base operating system and any optional packages installed on the Engine:

    # yum update

    If any kernel packages were updated, reboot the machine to complete the update.

14.4. Upgrading the Engine from 4.1 to 4.2

Upgrade the oVirt Engine from 4.1 to 4.2.

If the upgrade fails, the engine-setup command attempts to roll your oVirt Engine installation back to its previous state. For this reason, the previous version’s repositories must not be removed until after the upgrade is complete. If the upgrade fails, detailed instructions display that explain how to restore your installation.

Procedure
  1. Log in to the Engine virtual machine.

  2. Enable the oVirt 4.2 repositories:

    # yum install https://resources.ovirt.org/pub/yum-repo/ovirt-release42.rpm

    All other repositories remain the same across oVirt releases.

  3. Update the setup packages:

    # yum update ovirt\*setup\*
  4. Run engine-setup and follow the prompts to upgrade the oVirt Engine:

    # engine-setup

    If you changed the default trust keystore password, you might get a keystore certificate error. If so, update the configuration to reflect the correct password. For more information, see RHV 4.2 - Failed to import provider certificate into the external provider keystore

  5. Update the base operating system:

    # yum update

    If any kernel packages were updated, reboot the machine to complete the upgrade.

14.5. Upgrading the Engine from 4.2 to 4.3

Upgrade the oVirt Engine from 4.2 to 4.3.

If the upgrade fails, the engine-setup command attempts to roll your oVirt Engine installation back to its previous state. For this reason, the previous version’s repositories must not be removed until after the upgrade is complete. If the upgrade fails, detailed instructions display that explain how to restore your installation.

Procedure
  1. Log in to the Engine virtual machine.

  2. Enable the oVirt 4.3 repositories:

    # yum install https://resources.ovirt.org/pub/yum-repo/ovirt-release43.rpm

    All other repositories remain the same across oVirt releases.

  3. Update the setup packages:

    # yum update ovirt\*setup\*
  4. Run engine-setup and follow the prompts to upgrade the oVirt Engine:

    # engine-setup

    When the script completes successfully, the following message appears:

    Execution of setup completed successfully
  5. Update the base operating system:

    # yum update

    If any kernel packages were updated, reboot the machine to complete the upgrade.

14.6. Disabling Global Maintenance Mode

Procedure
  1. Log in to the Engine virtual machine and shut it down.

  2. Log in to one of the self-hosted engine nodes and disable global maintenance mode:

    # hosted-engine --set-maintenance --mode=none

    When you exit global maintenance mode, ovirt-ha-agent starts the Engine virtual machine, and then the Engine automatically starts. It can take up to ten minutes for the Engine to start.

  3. Confirm that the environment is running:

    # hosted-engine --vm-status

    The listed information includes Engine Status. The value for Engine status should be:

    {"health": "good", "vm": "up", "detail": "Up"}

    When the virtual machine is still booting and the Engine hasn’t started yet, the Engine status is:

    {"reason": "bad vm status", "health": "bad", "vm": "up", "detail": "Powering up"}

    If this happens, wait a few minutes and try again.

You can now update the self-hosted engine nodes, and then any standard hosts. The procedure is the same for both host types.

14.7. Updating All Hosts in a Cluster

You can update all hosts in a cluster instead of updating hosts individually. This is particularly useful during upgrades to new versions of oVirt. See https://github.com/oVirt/ovirt-ansible-cluster-upgrade/blob/master/README.md for more information about the Ansible role used to automate the updates.

oVirt recommends updating one cluster at a time.

Limitations
  • On oVirt Node, the update only preserves modified content in the /etc and /var directories. Modified data in other paths is overwritten during an update.

  • If migration is enabled at the cluster level, virtual machines are automatically migrated to another host in the cluster.

  • In a self-hosted engine environment, the Engine virtual machine can only migrate between self-hosted engine nodes in the same cluster. It cannot migrate to standard hosts.

  • The cluster must have sufficient memory reserved for its hosts to perform maintenance. Otherwise, virtual machine migrations will hang and fail. You can reduce the memory usage of host updates by shutting down some or all virtual machines before updating hosts.

  • You cannot migrate a pinned virtual machine (such as a virtual machine using a vGPU) to another host. Pinned virtual machines are shut down during the update, unless you choose to skip that host instead.

Procedure
  1. In the Administration Portal, click Compute  Clusters and select the cluster.

  2. Click Upgrade.

  3. Select the hosts to update, then click Next.

  4. Configure the options:

    • Stop Pinned VMs shuts down any virtual machines that are pinned to hosts in the cluster, and is selected by default. You can clear this check box to skip updating those hosts so that the pinned virtual machines stay running, such as when a pinned virtual machine is running important services or processes and you do not want it to shut down at an unknown time during the update.

    • Upgrade Timeout (Minutes) sets the time to wait for an individual host to be updated before the cluster upgrade fails with a timeout. The default is 60. You can increase it for large clusters where 60 minutes might not be enough, or reduce it for small clusters where the hosts update quickly.

    • Check Upgrade checks each host for available updates before running the upgrade process. It is not selected by default, but you can select it if you need to ensure that recent updates are included, such as when you have configured the Engine to check for host updates less frequently than the default.

    • Reboot After Upgrade reboots each host after it is updated, and is selected by default. You can clear this check box to speed up the process if you are sure that there are no pending updates that require a host reboot.

    • Use Maintenance Policy sets the cluster’s scheduling policy to cluster_maintenance during the update. It is selected by default, so activity is limited and virtual machines cannot start unless they are highly available. You can clear this check box if you have a custom scheduling policy that you want to keep using during the update, but this could have unknown consequences. Ensure your custom policy is compatible with cluster upgrade activity before disabling this option.

  5. Click Next.

  6. Review the summary of the hosts and virtual machines that will be affected.

  7. Click Upgrade.

You can track the progress of host updates in the Compute  Hosts view, and in the Events section of the Notification Drawer (EventsIcon).

You can track the progress of individual virtual machine migrations in the Status column of the Compute  Virtual Machines view. In large environments, you may need to filter the results to show a particular group of virtual machines.

14.8. Changing the Cluster Compatibility Version

oVirt clusters have a compatibility version. The cluster compatibility version indicates the features of oVirt supported by all of the hosts in the cluster. The cluster compatibility is set according to the version of the least capable host operating system in the cluster.

To change the cluster compatibility version, you must have first updated all the hosts in your cluster to a level that supports your desired compatibility level. Check if there is an icon next to the host indicating an update is available.

Procedure
  1. In the Administration Portal, click Compute  Clusters.

  2. Select the cluster to change and click Edit.

  3. On the General tab, change the Compatibility Version to the desired value.

  4. Click OK. The Change Cluster Compatibility Version confirmation dialog opens.

  5. Click OK to confirm.

An error message might warn that some virtual machines and templates are incorrectly configured. To fix this error, edit each virtual machine manually. The Edit Virtual Machine window provides additional validations and warnings that show what to correct. Sometimes the issue is automatically corrected and the virtual machine’s configuration just needs to be saved again. After editing each virtual machine, you will be able to change the cluster compatibility version.

14.9. Changing Virtual Machine Cluster Compatibility

After updating a cluster’s compatibility version, you must update the cluster compatibility version of all running or suspended virtual machines by rebooting them from the Administration Portal, or using the REST API, instead of from within the guest operating system. Virtual machines that require a reboot are marked with the pending changes icon (pendingchanges).

The Engine virtual machine does not need to be rebooted.

Although you can wait to reboot the virtual machines at a convenient time, rebooting immediately is highly recommended so that the virtual machines use the latest configuration. Any virtual machine that has not been rebooted runs with the previous configuration, and subsequent configuration changes made to the virtual machine might overwrite its pending cluster compatibility changes.

Procedure
  1. In the Administration Portal, click Compute  Virtual Machines.

  2. Check which virtual machines require a reboot. In the Vms: search bar, enter the following query:

    next_run_config_exists=True

    The search results show all virtual machines with pending changes.

  3. Select each virtual machine and click Reboot.

When the virtual machine starts, the new compatibility version is automatically applied.

You cannot change the cluster compatibility version of a virtual machine snapshot that is in preview. You must first commit or undo the preview.

14.10. Changing the Data Center Compatibility Version

oVirt data centers have a compatibility version. The compatibility version indicates the version of oVirt with which the data center is intended to be compatible. All clusters in the data center must support the desired compatibility level.

To change the data center compatibility version, you must have first updated the compatibility version of all clusters and virtual machines in the data center.

Procedure
  1. In the Administration Portal, click Compute  Data Centers.

  2. Select the data center to change and click Edit.

  3. Change the Compatibility Version to the desired value.

  4. Click OK. The Change Data Center Compatibility Version confirmation dialog opens.

  5. Click OK to confirm.

14.11. Updating OVN Providers Installed in oVirt 4.1

If you installed an Open Virtual Network (OVN) provider in oVirt 4.1, you must manually edit its configuration for oVirt 4.2.

Procedure
  1. Click Administration  Providers and select the OVN provider.

  2. Click Edit.

  3. Click the Networking Plugin text field and select oVirt Network Provider for OVN from the drop-down list.

  4. Click OK.

14.12. Replacing SHA-1 Certificates with SHA-256 Certificates

oVirt 4.3 uses SHA-256 signatures, which provide a more secure way to sign SSL certificates than SHA-1. Newly installed systems do not require any special steps to enable oVirt’s public key infrastructure (PKI) to use SHA-256 signatures. However, for systems upgraded from 4.1 or earlier, one of the following is required:

Preventing Warning Messages from Appearing in the Browser

  1. Log in to the Engine machine as the root user.

  2. Check whether /etc/pki/ovirt-engine/openssl.conf includes the line default_md = sha256:

    # cat /etc/pki/ovirt-engine/openssl.conf

    If it still includes default_md = sha1, back up the existing configuration and change the default to sha256:

    # cp -p /etc/pki/ovirt-engine/openssl.conf /etc/pki/ovirt-engine/openssl.conf."$(date +"%Y%m%d%H%M%S")"
    # sed -i 's/^default_md = sha1/default_md = sha256/' /etc/pki/ovirt-engine/openssl.conf
  3. Define the certificate that should be re-signed:

    # names="apache"
  4. Log in to one of the self-hosted engine nodes and enable global maintenance:

    # hosted-engine --set-maintenance --mode=global
  5. On the Engine, re-sign the Apache certificate:

    for name in $names; do
        subject="$(
            openssl \
                x509 \
                -in /etc/pki/ovirt-engine/certs/"${name}".cer \
                -noout \
                -subject \
            | sed \
                's;subject= \(.*\);\1;' \
        )"
       /usr/share/ovirt-engine/bin/pki-enroll-pkcs12.sh \
            --name="${name}" \
            --password=mypass \
            --subject="${subject}" \
            --san=DNS:"${ENGINE_FQDN}" \
            --keep-key
    done
  6. Restart the httpd service:

    # systemctl restart httpd
  7. Log in to one of the self-hosted engine nodes and disable global maintenance:

    # hosted-engine --set-maintenance --mode=none
  8. Connect to the Administration Portal to confirm that the warning no longer appears.

  9. If you previously imported a CA or https certificate into the browser, find the certificate(s), remove them from the browser, and reimport the new CA certificate. Install the certificate authority according to the instructions provided by your browser. To get the certificate authority’s certificate, navigate to http://your-manager-fqdn/ovirt-engine/services/pki-resource?resource=ca-certificate&format=X509-PEM-CA, replacing your-manager-fqdn with the fully qualified domain name (FQDN).

Replacing All Signed Certificates with SHA-256

  1. Log in to the Engine machine as the root user.

  2. Check whether /etc/pki/ovirt-engine/openssl.conf includes the line default_md = sha256:

    # cat /etc/pki/ovirt-engine/openssl.conf

    If it still includes default_md = sha1, back up the existing configuration and change the default to sha256:

    # cp -p /etc/pki/ovirt-engine/openssl.conf /etc/pki/ovirt-engine/openssl.conf."$(date +"%Y%m%d%H%M%S")"
    # sed -i 's/^default_md = sha1/default_md = sha256/' /etc/pki/ovirt-engine/openssl.conf
  3. Re-sign the CA certificate by backing it up and creating a new certificate in ca.pem.new:

    # cp -p /etc/pki/ovirt-engine/private/ca.pem /etc/pki/ovirt-engine/private/ca.pem."$(date +"%Y%m%d%H%M%S")"
    # openssl x509 -signkey /etc/pki/ovirt-engine/private/ca.pem -in /etc/pki/ovirt-engine/ca.pem -out /etc/pki/ovirt-engine/ca.pem.new -days 3650 -sha256
  4. Replace the existing certificate with the new certificate:

    # mv /etc/pki/ovirt-engine/ca.pem.new /etc/pki/ovirt-engine/ca.pem
  5. Define the certificates that should be re-signed:

    # names="engine apache websocket-proxy jboss imageio-proxy"

    If you replaced the oVirt Engine SSL Certificate after the upgrade, run the following instead:

    # names="engine websocket-proxy jboss imageio-proxy"

    For more details see Replacing the oVirt Engine CA Certificate in the Administration Guide.

  6. Log in to one of the self-hosted engine nodes and enable global maintenance:

    # hosted-engine --set-maintenance --mode=global
  7. On the Engine, re-sign the certificates:

    for name in $names; do
       subject="$(
            openssl \
                x509 \
                -in /etc/pki/ovirt-engine/certs/"${name}".cer \
                -noout \
                -subject \
            | sed \
                's;subject= \(.*\);\1;' \
            )"
         /usr/share/ovirt-engine/bin/pki-enroll-pkcs12.sh \
                --name="${name}" \
                --password=mypass \
                --subject="${subject}" \
                --san=DNS:"${ENGINE_FQDN}" \
                --keep-key
    done
  8. Restart the following services:

    # systemctl restart httpd
    # systemctl restart ovirt-engine
    # systemctl restart ovirt-websocket-proxy
    # systemctl restart ovirt-imageio-proxy
  9. Log in to one of the self-hosted engine nodes and disable global maintenance:

    # hosted-engine --set-maintenance --mode=none
  10. Connect to the Administration Portal to confirm that the warning no longer appears.

  11. If you previously imported a CA or https certificate into the browser, find the certificate(s), remove them from the browser, and reimport the new CA certificate. Install the certificate authority according to the instructions provided by your browser. To get the certificate authority’s certificate, navigate to http://your-manager-fqdn/ovirt-engine/services/pki-resource?resource=ca-certificate&format=X509-PEM-CA, replacing your-manager-fqdn with the fully qualified domain name (FQDN).

  12. Enroll the certificates on the hosts. Repeat the following procedure for each host.

    1. In the Administration Portal, click Compute  Hosts.

    2. Select the host and click Management  Maintenance.

    3. Once the host is in maintenance mode, click Installation  Enroll Certificate.

    4. Click Management  Activate.

15. Upgrading a Self-Hosted Engine from 4.2 to oVirt 4.3

Upgrading a self-hosted engine environment from version 4.2 to 4.3 involves the following steps:

15.1. Prerequisites

  • Plan for any necessary virtual machine downtime. After you update the clusters' compatibility versions during the upgrade, a new hardware configuration is automatically applied to each virtual machine once it reboots. You must reboot any running or suspended virtual machines as soon as possible to apply the configuration changes.

  • Ensure the Engine has the correct repositories enabled.

15.2. Enabling Global Maintenance Mode

You must place the self-hosted engine environment in global maintenance mode before performing any setup or upgrade tasks on the Engine virtual machine.

Procedure
  1. Log in to one of the self-hosted engine nodes and enable global maintenance mode:

    # hosted-engine --set-maintenance --mode=global
  2. Confirm that the environment is in maintenance mode before proceeding:

    # hosted-engine --vm-status

    You should see a message indicating that the cluster is in maintenance mode.

15.3. Updating the oVirt Engine

Be sure to enable the correct repositories as detailed in Prerequisites. Otherwise, the upgrade might fail.

Updates to the oVirt Engine are released through the Content Delivery Network.

Procedure
  1. Log in to the Engine virtual machine.

  2. Check if updated packages are available:

    # engine-upgrade-check
  3. Update the setup packages:

    # yum update ovirt\*setup\*
  4. Update the oVirt Engine with the engine-setup script. The engine-setup script prompts you with some configuration questions, then stops the ovirt-engine service, downloads and installs the updated packages, backs up and updates the database, performs post-installation configuration, and starts the ovirt-engine service.

    # engine-setup

    When the script completes successfully, the following message appears:

    Execution of setup completed successfully

    The engine-setup script is also used during the oVirt Engine installation process, and it stores the configuration values supplied. During an update, the stored values are displayed when previewing the configuration, and might not be up to date if engine-config was used to update configuration after installation. For example, if engine-config was used to update SANWipeAfterDelete to true after installation, engine-setup will output "Default SAN wipe after delete: False" in the configuration preview. However, the updated values will not be overwritten by engine-setup.

    The update process might take some time. Do not stop the process before it completes.

  5. Update the base operating system and any optional packages installed on the Engine:

    # yum update

    If any kernel packages were updated, reboot the machine to complete the update.

15.4. Upgrading the Engine from 4.2 to 4.3

Upgrade the oVirt Engine from 4.2 to 4.3.

If the upgrade fails, the engine-setup command attempts to roll your oVirt Engine installation back to its previous state. For this reason, the previous version’s repositories must not be removed until after the upgrade is complete. If the upgrade fails, detailed instructions display that explain how to restore your installation.

Procedure
  1. Log in to the Engine virtual machine.

  2. Enable the oVirt 4.3 repositories:

    # yum install https://resources.ovirt.org/pub/yum-repo/ovirt-release43.rpm

    All other repositories remain the same across oVirt releases.

  3. Update the setup packages:

    # yum update ovirt\*setup\*
  4. Run engine-setup and follow the prompts to upgrade the oVirt Engine:

    # engine-setup

    When the script completes successfully, the following message appears:

    Execution of setup completed successfully
  5. Update the base operating system:

    # yum update

    If any kernel packages were updated, reboot the machine to complete the upgrade.

15.5. Disabling Global Maintenance Mode

Procedure
  1. Log in to the Engine virtual machine and shut it down.

  2. Log in to one of the self-hosted engine nodes and disable global maintenance mode:

    # hosted-engine --set-maintenance --mode=none

    When you exit global maintenance mode, ovirt-ha-agent starts the Engine virtual machine, and then the Engine automatically starts. It can take up to ten minutes for the Engine to start.

  3. Confirm that the environment is running:

    # hosted-engine --vm-status

    The listed information includes Engine Status. The value for Engine status should be:

    {"health": "good", "vm": "up", "detail": "Up"}

    When the virtual machine is still booting and the Engine hasn’t started yet, the Engine status is:

    {"reason": "bad vm status", "health": "bad", "vm": "up", "detail": "Powering up"}

    If this happens, wait a few minutes and try again.

You can now update the self-hosted engine nodes, and then any standard hosts. The procedure is the same for both host types.

15.6. Updating All Hosts in a Cluster

You can update all hosts in a cluster instead of updating hosts individually. This is particularly useful during upgrades to new versions of oVirt. See https://github.com/oVirt/ovirt-ansible-cluster-upgrade/blob/master/README.md for more information about the Ansible role used to automate the updates.

oVirt recommends updating one cluster at a time.

Limitations
  • On oVirt Node, the update only preserves modified content in the /etc and /var directories. Modified data in other paths is overwritten during an update.

  • If migration is enabled at the cluster level, virtual machines are automatically migrated to another host in the cluster.

  • In a self-hosted engine environment, the Engine virtual machine can only migrate between self-hosted engine nodes in the same cluster. It cannot migrate to standard hosts.

  • The cluster must have sufficient memory reserved for its hosts to perform maintenance. Otherwise, virtual machine migrations will hang and fail. You can reduce the memory usage of host updates by shutting down some or all virtual machines before updating hosts.

  • You cannot migrate a pinned virtual machine (such as a virtual machine using a vGPU) to another host. Pinned virtual machines are shut down during the update, unless you choose to skip that host instead.

Procedure
  1. In the Administration Portal, click Compute  Clusters and select the cluster.

  2. Click Upgrade.

  3. Select the hosts to update, then click Next.

  4. Configure the options:

    • Stop Pinned VMs shuts down any virtual machines that are pinned to hosts in the cluster, and is selected by default. You can clear this check box to skip updating those hosts so that the pinned virtual machines stay running, such as when a pinned virtual machine is running important services or processes and you do not want it to shut down at an unknown time during the update.

    • Upgrade Timeout (Minutes) sets the time to wait for an individual host to be updated before the cluster upgrade fails with a timeout. The default is 60. You can increase it for large clusters where 60 minutes might not be enough, or reduce it for small clusters where the hosts update quickly.

    • Check Upgrade checks each host for available updates before running the upgrade process. It is not selected by default, but you can select it if you need to ensure that recent updates are included, such as when you have configured the Engine to check for host updates less frequently than the default.

    • Reboot After Upgrade reboots each host after it is updated, and is selected by default. You can clear this check box to speed up the process if you are sure that there are no pending updates that require a host reboot.

    • Use Maintenance Policy sets the cluster’s scheduling policy to cluster_maintenance during the update. It is selected by default, so activity is limited and virtual machines cannot start unless they are highly available. You can clear this check box if you have a custom scheduling policy that you want to keep using during the update, but this could have unknown consequences. Ensure your custom policy is compatible with cluster upgrade activity before disabling this option.

  5. Click Next.

  6. Review the summary of the hosts and virtual machines that will be affected.

  7. Click Upgrade.

You can track the progress of host updates in the Compute  Hosts view, and in the Events section of the Notification Drawer (EventsIcon).

You can track the progress of individual virtual machine migrations in the Status column of the Compute  Virtual Machines view. In large environments, you may need to filter the results to show a particular group of virtual machines.

15.7. Changing the Cluster Compatibility Version

oVirt clusters have a compatibility version. The cluster compatibility version indicates the features of oVirt supported by all of the hosts in the cluster. The cluster compatibility is set according to the version of the least capable host operating system in the cluster.

To change the cluster compatibility version, you must have first updated all the hosts in your cluster to a level that supports your desired compatibility level. Check if there is an icon next to the host indicating an update is available.

Procedure
  1. In the Administration Portal, click Compute  Clusters.

  2. Select the cluster to change and click Edit.

  3. On the General tab, change the Compatibility Version to the desired value.

  4. Click OK. The Change Cluster Compatibility Version confirmation dialog opens.

  5. Click OK to confirm.

An error message might warn that some virtual machines and templates are incorrectly configured. To fix this error, edit each virtual machine manually. The Edit Virtual Machine window provides additional validations and warnings that show what to correct. Sometimes the issue is automatically corrected and the virtual machine’s configuration just needs to be saved again. After editing each virtual machine, you will be able to change the cluster compatibility version.

15.8. Changing Virtual Machine Cluster Compatibility

After updating a cluster’s compatibility version, you must update the cluster compatibility version of all running or suspended virtual machines by rebooting them from the Administration Portal, or using the REST API, instead of from within the guest operating system. Virtual machines that require a reboot are marked with the pending changes icon (pendingchanges).

The Engine virtual machine does not need to be rebooted.

Although you can wait to reboot the virtual machines at a convenient time, rebooting immediately is highly recommended so that the virtual machines use the latest configuration. Any virtual machine that has not been rebooted runs with the previous configuration, and subsequent configuration changes made to the virtual machine might overwrite its pending cluster compatibility changes.

Procedure
  1. In the Administration Portal, click Compute  Virtual Machines.

  2. Check which virtual machines require a reboot. In the Vms: search bar, enter the following query:

    next_run_config_exists=True

    The search results show all virtual machines with pending changes.

  3. Select each virtual machine and click Reboot.

When the virtual machine starts, the new compatibility version is automatically applied.

You cannot change the cluster compatibility version of a virtual machine snapshot that is in preview. You must first commit or undo the preview.

15.9. Changing the Data Center Compatibility Version

oVirt data centers have a compatibility version. The compatibility version indicates the version of oVirt with which the data center is intended to be compatible. All clusters in the data center must support the desired compatibility level.

To change the data center compatibility version, you must have first updated the compatibility version of all clusters and virtual machines in the data center.

Procedure
  1. In the Administration Portal, click Compute  Data Centers.

  2. Select the data center to change and click Edit.

  3. Change the Compatibility Version to the desired value.

  4. Click OK. The Change Data Center Compatibility Version confirmation dialog opens.

  5. Click OK to confirm.

If you previously upgraded to 4.2 without replacing SHA-1 certificates with SHA-256 certificates, you must do so now.

15.10. Replacing SHA-1 Certificates with SHA-256 Certificates

oVirt 4.3 uses SHA-256 signatures, which provide a more secure way to sign SSL certificates than SHA-1. Newly installed systems do not require any special steps to enable oVirt’s public key infrastructure (PKI) to use SHA-256 signatures. However, for systems upgraded from 4.1 or earlier, one of the following is required:

Preventing Warning Messages from Appearing in the Browser

  1. Log in to the Engine machine as the root user.

  2. Check whether /etc/pki/ovirt-engine/openssl.conf includes the line default_md = sha256:

    # cat /etc/pki/ovirt-engine/openssl.conf

    If it still includes default_md = sha1, back up the existing configuration and change the default to sha256:

    # cp -p /etc/pki/ovirt-engine/openssl.conf /etc/pki/ovirt-engine/openssl.conf."$(date +"%Y%m%d%H%M%S")"
    # sed -i 's/^default_md = sha1/default_md = sha256/' /etc/pki/ovirt-engine/openssl.conf
  3. Define the certificate that should be re-signed:

    # names="apache"
  4. Log in to one of the self-hosted engine nodes and enable global maintenance:

    # hosted-engine --set-maintenance --mode=global
  5. On the Engine, re-sign the Apache certificate:

    for name in $names; do
        subject="$(
            openssl \
                x509 \
                -in /etc/pki/ovirt-engine/certs/"${name}".cer \
                -noout \
                -subject \
            | sed \
                's;subject= \(.*\);\1;' \
        )"
       /usr/share/ovirt-engine/bin/pki-enroll-pkcs12.sh \
            --name="${name}" \
            --password=mypass \
            --subject="${subject}" \
            --san=DNS:"${ENGINE_FQDN}" \
            --keep-key
    done
  6. Restart the httpd service:

    # systemctl restart httpd
  7. Log in to one of the self-hosted engine nodes and disable global maintenance:

    # hosted-engine --set-maintenance --mode=none
  8. Connect to the Administration Portal to confirm that the warning no longer appears.

  9. If you previously imported a CA or https certificate into the browser, find the certificate(s), remove them from the browser, and reimport the new CA certificate. Install the certificate authority according to the instructions provided by your browser. To get the certificate authority’s certificate, navigate to http://your-manager-fqdn/ovirt-engine/services/pki-resource?resource=ca-certificate&format=X509-PEM-CA, replacing your-manager-fqdn with the fully qualified domain name (FQDN).

Replacing All Signed Certificates with SHA-256

  1. Log in to the Engine machine as the root user.

  2. Check whether /etc/pki/ovirt-engine/openssl.conf includes the line default_md = sha256:

    # cat /etc/pki/ovirt-engine/openssl.conf

    If it still includes default_md = sha1, back up the existing configuration and change the default to sha256:

    # cp -p /etc/pki/ovirt-engine/openssl.conf /etc/pki/ovirt-engine/openssl.conf."$(date +"%Y%m%d%H%M%S")"
    # sed -i 's/^default_md = sha1/default_md = sha256/' /etc/pki/ovirt-engine/openssl.conf
  3. Re-sign the CA certificate by backing it up and creating a new certificate in ca.pem.new:

    # cp -p /etc/pki/ovirt-engine/private/ca.pem /etc/pki/ovirt-engine/private/ca.pem."$(date +"%Y%m%d%H%M%S")"
    # openssl x509 -signkey /etc/pki/ovirt-engine/private/ca.pem -in /etc/pki/ovirt-engine/ca.pem -out /etc/pki/ovirt-engine/ca.pem.new -days 3650 -sha256
  4. Replace the existing certificate with the new certificate:

    # mv /etc/pki/ovirt-engine/ca.pem.new /etc/pki/ovirt-engine/ca.pem
  5. Define the certificates that should be re-signed:

    # names="engine apache websocket-proxy jboss imageio-proxy"

    If you replaced the oVirt Engine SSL Certificate after the upgrade, run the following instead:

    # names="engine websocket-proxy jboss imageio-proxy"

    For more details see Replacing the oVirt Engine CA Certificate in the Administration Guide.

  6. Log in to one of the self-hosted engine nodes and enable global maintenance:

    # hosted-engine --set-maintenance --mode=global
  7. On the Engine, re-sign the certificates:

    for name in $names; do
       subject="$(
            openssl \
                x509 \
                -in /etc/pki/ovirt-engine/certs/"${name}".cer \
                -noout \
                -subject \
            | sed \
                's;subject= \(.*\);\1;' \
            )"
         /usr/share/ovirt-engine/bin/pki-enroll-pkcs12.sh \
                --name="${name}" \
                --password=mypass \
                --subject="${subject}" \
                --san=DNS:"${ENGINE_FQDN}" \
                --keep-key
    done
  8. Restart the following services:

    # systemctl restart httpd
    # systemctl restart ovirt-engine
    # systemctl restart ovirt-websocket-proxy
    # systemctl restart ovirt-imageio-proxy
  9. Log in to one of the self-hosted engine nodes and disable global maintenance:

    # hosted-engine --set-maintenance --mode=none
  10. Connect to the Administration Portal to confirm that the warning no longer appears.

  11. If you previously imported a CA or https certificate into the browser, find the certificate(s), remove them from the browser, and reimport the new CA certificate. Install the certificate authority according to the instructions provided by your browser. To get the certificate authority’s certificate, navigate to http://your-manager-fqdn/ovirt-engine/services/pki-resource?resource=ca-certificate&format=X509-PEM-CA, replacing your-manager-fqdn with the fully qualified domain name (FQDN).

  12. Enroll the certificates on the hosts. Repeat the following procedure for each host.

    1. In the Administration Portal, click Compute  Hosts.

    2. Select the host and click Management  Maintenance.

    3. Once the host is in maintenance mode, click Installation  Enroll Certificate.

    4. Click Management  Activate.

Appendices

Appendix A: Updates between Minor Releases

To update from your current version of 4.3 to the latest version of 4.3, update the Engine and then update the hosts.

To update a standalone Engine, follow the standard procedure for minor updates:

Updating the oVirt Engine

Be sure to enable the correct repositories as detailed in Prerequisites. Otherwise, the upgrade might fail.

Updates to the oVirt Engine are released through the Content Delivery Network.

Procedure
  1. Log in to the Engine machine.

  2. Check if updated packages are available:

    # engine-upgrade-check
  3. Update the setup packages:

    # yum update ovirt\*setup\*
  4. Update the oVirt Engine with the engine-setup script. The engine-setup script prompts you with some configuration questions, then stops the ovirt-engine service, downloads and installs the updated packages, backs up and updates the database, performs post-installation configuration, and starts the ovirt-engine service.

    # engine-setup

    When the script completes successfully, the following message appears:

    Execution of setup completed successfully

    The engine-setup script is also used during the oVirt Engine installation process, and it stores the configuration values supplied. During an update, the stored values are displayed when previewing the configuration, and might not be up to date if engine-config was used to update configuration after installation. For example, if engine-config was used to update SANWipeAfterDelete to true after installation, engine-setup will output "Default SAN wipe after delete: False" in the configuration preview. However, the updated values will not be overwritten by engine-setup.

    The update process might take some time. Do not stop the process before it completes.

  5. Update the base operating system and any optional packages installed on the Engine:

    # yum update

    If any kernel packages were updated, reboot the machine to complete the update.

If the update from oVirt 4.2.7 to oVirt 4.2.8 fails with a message indicating a dependency error with an eap7-jboss-server-migration-wildfly package:

  1. Check if all the required repositories are enabled.

  2. Update the eap7-jboss-server-migration-wildfly packages installed on the Manager:

    # yum update eap7-jboss-server-migration-wildfly*
  3. Run engine-setup again.

Updating a Self-Hosted Engine

To update a self-hosted engine from your current version of 4.3 to the latest version of 4.3, you must place the environment in global maintenance mode and then follow the standard procedure for updating between minor versions.

Enabling Global Maintenance Mode

You must place the self-hosted engine environment in global maintenance mode before performing any setup or upgrade tasks on the Engine virtual machine.

Procedure
  1. Log in to one of the self-hosted engine nodes and enable global maintenance mode:

    # hosted-engine --set-maintenance --mode=global
  2. Confirm that the environment is in maintenance mode before proceeding:

    # hosted-engine --vm-status

    You should see a message indicating that the cluster is in maintenance mode.

Updating the oVirt Engine

Be sure to enable the correct repositories as detailed in Prerequisites. Otherwise, the upgrade might fail.

Updates to the oVirt Engine are released through the Content Delivery Network.

Procedure
  1. Log in to the Engine virtual machine.

  2. Log in to the Engine machine.

  3. Check if updated packages are available:

    # engine-upgrade-check
  4. Update the setup packages:

    # yum update ovirt\*setup\*
  5. Update the oVirt Engine with the engine-setup script. The engine-setup script prompts you with some configuration questions, then stops the ovirt-engine service, downloads and installs the updated packages, backs up and updates the database, performs post-installation configuration, and starts the ovirt-engine service.

    # engine-setup

    When the script completes successfully, the following message appears:

    Execution of setup completed successfully

    The engine-setup script is also used during the oVirt Engine installation process, and it stores the configuration values supplied. During an update, the stored values are displayed when previewing the configuration, and might not be up to date if engine-config was used to update configuration after installation. For example, if engine-config was used to update SANWipeAfterDelete to true after installation, engine-setup will output "Default SAN wipe after delete: False" in the configuration preview. However, the updated values will not be overwritten by engine-setup.

    The update process might take some time. Do not stop the process before it completes.

  6. Update the base operating system and any optional packages installed on the Engine:

    # yum update

    If any kernel packages were updated:

    1. Disable global maintenance mode

    2. Reboot the machine to complete the update.

If the update from oVirt 4.2.7 to oVirt 4.2.8 fails with a message indicating a dependency error with an eap7-jboss-server-migration-wildfly package:

  1. Check if all the required repositories are enabled.

  2. Update the eap7-jboss-server-migration-wildfly packages installed on the Manager:

    # yum update eap7-jboss-server-migration-wildfly*
  3. Run engine-setup again.

Related Information

Disabling Global Maintenance Mode

Disabling Global Maintenance Mode

Procedure
  1. Log in to the Engine virtual machine and shut it down.

  2. Log in to one of the self-hosted engine nodes and disable global maintenance mode:

    # hosted-engine --set-maintenance --mode=none

    When you exit global maintenance mode, ovirt-ha-agent starts the Engine virtual machine, and then the Engine automatically starts. It can take up to ten minutes for the Engine to start.

  3. Confirm that the environment is running:

    # hosted-engine --vm-status

    The listed information includes Engine Status. The value for Engine status should be:

    {"health": "good", "vm": "up", "detail": "Up"}

    When the virtual machine is still booting and the Engine hasn’t started yet, the Engine status is:

    {"reason": "bad vm status", "health": "bad", "vm": "up", "detail": "Powering up"}

    If this happens, wait a few minutes and try again.

Updating All Hosts in a Cluster

You can update all hosts in a cluster instead of updating hosts individually. This is particularly useful during upgrades to new versions of oVirt. See https://github.com/oVirt/ovirt-ansible-cluster-upgrade/blob/master/README.md for more information about the Ansible role used to automate the updates.

oVirt recommends updating one cluster at a time.

Limitations
  • On oVirt Node, the update only preserves modified content in the /etc and /var directories. Modified data in other paths is overwritten during an update.

  • If migration is enabled at the cluster level, virtual machines are automatically migrated to another host in the cluster.

  • In a self-hosted engine environment, the Engine virtual machine can only migrate between self-hosted engine nodes in the same cluster. It cannot migrate to standard hosts.

  • The cluster must have sufficient memory reserved for its hosts to perform maintenance. Otherwise, virtual machine migrations will hang and fail. You can reduce the memory usage of host updates by shutting down some or all virtual machines before updating hosts.

  • You cannot migrate a pinned virtual machine (such as a virtual machine using a vGPU) to another host. Pinned virtual machines are shut down during the update, unless you choose to skip that host instead.

Procedure
  1. In the Administration Portal, click Compute  Clusters and select the cluster.

  2. Click Upgrade.

  3. Select the hosts to update, then click Next.

  4. Configure the options:

    • Stop Pinned VMs shuts down any virtual machines that are pinned to hosts in the cluster, and is selected by default. You can clear this check box to skip updating those hosts so that the pinned virtual machines stay running, such as when a pinned virtual machine is running important services or processes and you do not want it to shut down at an unknown time during the update.

    • Upgrade Timeout (Minutes) sets the time to wait for an individual host to be updated before the cluster upgrade fails with a timeout. The default is 60. You can increase it for large clusters where 60 minutes might not be enough, or reduce it for small clusters where the hosts update quickly.

    • Check Upgrade checks each host for available updates before running the upgrade process. It is not selected by default, but you can select it if you need to ensure that recent updates are included, such as when you have configured the Engine to check for host updates less frequently than the default.

    • Reboot After Upgrade reboots each host after it is updated, and is selected by default. You can clear this check box to speed up the process if you are sure that there are no pending updates that require a host reboot.

    • Use Maintenance Policy sets the cluster’s scheduling policy to cluster_maintenance during the update. It is selected by default, so activity is limited and virtual machines cannot start unless they are highly available. You can clear this check box if you have a custom scheduling policy that you want to keep using during the update, but this could have unknown consequences. Ensure your custom policy is compatible with cluster upgrade activity before disabling this option.

  5. Click Next.

  6. Review the summary of the hosts and virtual machines that will be affected.

  7. Click Upgrade.

You can track the progress of host updates in the Compute  Hosts view, and in the Events section of the Notification Drawer (EventsIcon).

You can track the progress of individual virtual machine migrations in the Status column of the Compute  Virtual Machines view. In large environments, you may need to filter the results to show a particular group of virtual machines.

You can also update hosts individually:

Updating Individual Hosts

Use the host upgrade manager to update individual hosts directly from the Administration Portal.

The upgrade manager only checks hosts with a status of Up or Non-operational, but not Maintenance.

Limitations
  • On oVirt Node, the update only preserves modified content in the /etc and /var directories. Modified data in other paths is overwritten during an update.

  • If migration is enabled at the cluster level, virtual machines are automatically migrated to another host in the cluster. Update a host when its usage is relatively low.

  • In a self-hosted engine environment, the Engine virtual machine can only migrate between self-hosted engine nodes in the same cluster. It cannot migrate to standard hosts.

  • The cluster must have sufficient memory reserved for its hosts to perform maintenance. Otherwise, virtual machine migrations will hang and fail. You can reduce the memory usage of host updates by shutting down some or all virtual machines before updating hosts.

  • Do not update all hosts at the same time, as one host must remain available to perform Storage Pool Manager (SPM) tasks.

  • You cannot migrate a pinned virtual machine (such as a virtual machine using a vGPU) to another host. Pinned virtual machines must be shut down before updating the host.

Procedure
  1. Ensure that the correct repositories are enabled. To view a list of currently enabled repositories, run yum repolist.

  2. In the Administration Portal, click Compute  Hosts and select the host to be updated.

  3. Click Installation  Check for Upgrade and click OK.

    Open the Notification Drawer (EventsIcon) and expand the Events section to see the result.

  4. If an update is available, click Installation  Upgrade.

  5. Click OK to update the host. Running virtual machines are migrated according to their migration policy. If migration is disabled for any virtual machines, you are prompted to shut them down.

    The details of the host are updated in Compute  Hosts and the status transitions through these stages:

    Maintenance > Installing > Reboot > Up

    If the update fails, the host’s status changes to Install Failed. From Install Failed you can click Installation  Upgrade again.

Repeat this procedure for each host in the oVirt environment.

oVirt recommends updating the hosts from the Administration Portal. However, you can update the hosts using yum update instead:

Manually Updating Hosts

You can use the yum command to update your hosts. Update your systems regularly, to ensure timely application of security and bug fixes.

Limitations
  • On oVirt Node, the update only preserves modified content in the /etc and /var directories. Modified data in other paths is overwritten during an update.

  • If migration is enabled at the cluster level, virtual machines are automatically migrated to another host in the cluster. Update a host when its usage is relatively low.

  • In a self-hosted engine environment, the Engine virtual machine can only migrate between self-hosted engine nodes in the same cluster. It cannot migrate to standard hosts.

  • The cluster must have sufficient memory reserved for its hosts to perform maintenance. Otherwise, virtual machine migrations will hang and fail. You can reduce the memory usage of host updates by shutting down some or all virtual machines before updating hosts.

  • Do not update all hosts at the same time, as one host must remain available to perform Storage Pool Manager (SPM) tasks.

  • You cannot migrate a pinned virtual machine (such as a virtual machine using a vGPU) to another host. Pinned virtual machines must be shut down before updating the host.

Procedure
  1. Ensure the correct repositories are enabled. You can check which repositories are currently enabled by running yum repolist.

  2. In the Administration Portal, click Compute  Hosts and select the host to be updated.

  3. Click Management  Maintenance.

  4. Update the host:

    # yum update
  5. Reboot the host to ensure all updates are correctly applied.

    Check the imgbased logs to see if any additional package updates have failed for a oVirt Node. If some packages were not successfully reinstalled after the update, check that the packages are listed in /var/imgbased/persisted-rpms. Add any missing packages then run rpm -Uvh /var/imgbased/persisted-rpms/*.

Repeat this process for each host in the oVirt environment.

Appendix B: Updating the Local Repository for an Offline oVirt Engine Installation

If your oVirt Engine is hosted on a machine that receives packages via FTP from a local repository, you must regularly synchronize the repository to download package updates from the Content Delivery Network, then update or upgrade that machine. Updated packages address security issues, fix bugs, and add enhancements.

  1. On the system hosting the repository, synchronize the repository to download the most recent version of each available package:

    # reposync -l --newest-only /var/ftp/pub/ovirtrepo

    This command might download a large number of packages, and take a long time to complete.

  2. Ensure that the repository is available on the Engine machine, and then update or upgrade the machine. See Updating the oVirt Engine for information on updating the Engine between minor versions. See oVirt Upgrade Overview for information on upgrading between major versions.

Certain portions of this text first appeared in the Red Hat Virtualization 4.4-beta Upgrade Guide. Copyright © 2020 Red Hat, Inc. Licensed under a Creative Commons Attribution-ShareAlike 3.0 Unported License.