No relevant resource is found in the selected language.

This site uses cookies. By continuing to browse the site you are agreeing to our use of cookies. Read our privacy policy>

Reminder

To have a better experience, please upgrade your IE browser.

upgrade

Multi-core VMs backup fails by eBackup in FusionSphere 6.0

Publication Date:  2017-05-19 Views:  670 Downloads:  0
Issue Description

Customer has reported his backup tasks fail for several VMs with Red Hat Enterprise Linux Server release 7.2 and CentOS Linux release 7.2 installed. The same time with backup creation failure affected VM hangs and becomes unresponsive to console. Only forced reboot helps to bring the VM back to operation.

Alarm Information

Following error message is returned by eBackup

Handling Process

 To troubleshoot the issue the following steps done:

1. Since eBackup uses snapshot function of FusionSphere, we checked with customer snapshot creation of affected VMs, it also fails.

2. Analyzed the configuration of affected VMs, all of them are configured with multiple CPU cores. VMs with the same OS versions but single CPU core are not affected. If to change multi-core configuration of affected VM to single-core, snapshot creation succeeds.

3. Huawei R&D confirmed this issue is connected with software bug in kernel of used RHEL and CentOS linux versions.

4. Huawei R&D developed a workaround using linux PV driver to create disk snapshots of multiple core VMs. Disk snapshot creation is sufficient for eBackup operation.

Root Cause

The root cause of this issue is the Linux kernel bug. When use the snapshot function, the VM internal reconfigure the “vcpu_info”, but configure all the VCPU “vcpu_info” to part of the “HYPERVISOR_shared_info” and not inform xen kernel. So the VCPU0 and xen kernel record the same location but other VCPU caused the location address wrong to recover abnormal. Xen send the interrupt message to VM,but the VM CPU can not handle this interrupt message except the VCPU0 then caused the VM stuck.

Solution

The solution is implemented as workaround in PV driver for Linux OS. Disk snapshots of multi-core VM are available, memory snapshots are not implemented.

To resolve the problem update PV driver on affected VMs to V1.3.10.160 and later.

http://support.huawei.com/enterprise/en/cloud-computing/fusionsphere-sia-pid-19916941/software/22368298

Since the snapshot creation issue is resolved by this solution, backup creation issue in eBackup is also resolved.

END