No relevant resource is found in the selected language.

This site uses cookies. By continuing to browse the site you are agreeing to our use of cookies. Read our privacy policy>

Reminder

To have a better experience, please upgrade your IE browser.

upgrade

CH121 V3 suffered VMware PSOD

Publication Date:  2017-09-06 Views:  98 Downloads:  0
Issue Description

Due to server down some of the VMs are rebooted. 

Alarm Information

Basic Information  

CH121 V3 Suffered VMware PSOD




Handling Process

Problem Description  

Fault Symptom


On Sep. 02, CH121 V3 suffered VMware PSOD. The PSOD screenshot is as following image



Problem Analysis

Exception Analysis

1.         SEL Analysis

The hardware SEL only has records of 2016, which indicates that this time of issue is VMware PSOD (VMware PSOD: Purple Screen Of Death).



1.         OS Analysis

OS log had printed vmnic0: transmit timed out during the PSOD, as follows:

esx-inp44pphv016a.vsnl.co.in-2017-09-02--22.00\esx-inp44pphv016a.vsnl.co.in-2017-09-02--22.00\var\log\ vmkernel.log

2017-09-02T17:05:43.491Z cpu29:33573)WARNING: LinNet: netdev_watchdog:3478: NETDEV WATCHDOG: vmnic0: transmit timed out

2017-09-02T17:05:43.491Z cpu29:33573)WARNING: at vmkdrivers/src_92/vmklinux_92/vmware/linux_net.c:3507/netdev_watchdog() (inside vmklinux)

2017-09-02T17:05:43.491Z cpu29:33573)Backtrace for current CPU #29, worldID=33573, ebp=0x4123cc95de10

2017-09-02T17:05:43.491Z cpu29:33573)0x4123cc95de10:[0x418014ecd2db]watchdog_work_cb@com.vmware.driverAPI#9.2+0x28f stack: 0x410800000d9

2017-09-02T17:05:43.491Z cpu29:33573)0x4123cc95de90:[0x418014ecd2db]watchdog_work_cb@com.vmware.driverAPI#9.2+0x28f stack: 0x410b0f3fb4e

2017-09-02T17:05:43.491Z cpu29:33573)0x4123cc95df30:[0x418014ef44a6]vmklnx_workqueue_callout@com.vmware.driverAPI#9.2+0xde stack: 0x0, 0

2017-09-02T17:05:43.491Z cpu29:33573)0x4123cc95dfd0:[0x41801486173a]helpFunc@vmkernel#nover+0xab6 stack: 0x0, 0x0, 0x0, 0x0, 0x0

2017-09-02T17:05:43.491Z cpu29:33573)0x4123cc95dff0:[0x418014a56872]CpuSched_StartWorld@vmkernel#nover+0xfa stack: 0x0, 0x0, 0x0, 0x0, 0

According to the system log, the NIC card information is as follows:

NIC:  vmnic0

NICInfo:

   Advertised Auto Negotiation: true

   Advertised Link Modes: 1000baseT/Full, 10000baseT/Full

   Auto Negotiation: true

   Cable Type: Twisted Pair

   Current Message Level: -1

   Driver Info:

      NICDriverInfo:

         Bus Info: 0000:04:00.0

         Driver: be2net

         Firmware Version: 1.1.43.34

         Version: 4.6.100.0v

   Link Detected: true



Root Cause

Root Cause

The VMware PSOD is caused by default ESXi 5.5 driver (4.6.100.0v) of the MZ910 card.

Solution

Conclusion 

MZ910 driver version is an inbox driver which was VMWare integrated and it is lower than Huawei recommended version, which caused the PSOD.


Solution 

Update all blades MZ910 card’s driver and firmware to the latest version.

1. Recommend firmware:

MZ910&MZ512 FW: Z510&MZ512&MZ910-FW(be3-11.1.240.0&Lancer-11.1.245.5)-Upgrade-Tool_V1.0.iso

2. Recommend driver:

MZ910&MZ512 driver: FusionServer iDriver-Vmware-Driver-V323.zip


END