No relevant resource is found in the selected language.

This site uses cookies. By continuing to browse the site you are agreeing to our use of cookies. Read our privacy policy>Search

Reminder

To have a better experience, please upgrade your IE browser.

upgrade

FusionServer G5500 Server G560 V5 Compute Node Maintenance and Service Guide 03

Rate and give feedback:
Huawei uses machine translation combined with human proofreading to translate this document to different languages in order to help you better understand the content of this document. Note: Even the most advanced machine translation cannot match the quality of professional translators. Huawei shall not bear any responsibility for translation accuracy and it is recommended that you refer to the English document (a link for which has been provided).
Removing a 2.5-inch Drive

Removing a 2.5-inch Drive

Removing a SAS/SATA Drive

NOTE:
  • If no OS is installed on the drive, the compute node does not need to be powered off.
  • If an OS is installed on the drive, and the drive is configured in a RAID with redundancy, the compute node does not need to be powered off.
  • If an OS is installed on the drive, and the drive is not configured in a RAID or is configured in a RAID without redundancy (for example, RAID 0), the compute node needs to be powered off.
Procedure
  1. Check whether the SAS/SATA drive to be removed is in a RAID array:

    • If the SAS/SATA drive is not in a RAID array or in a RAID array without redundancy settings (such as a RAID 0 array), back up the data on the SAS/SATA drive before removing it.
    • If the SAS/SATA drive to be removed is in a RAID array with redundancy, you can remove the SAS/SATA drive without backing up the data.

  2. Determine the SAS/SATA drive to be removed. Figure 5-27 shows the locations of SAS/SATA drives.

    Figure 5-27 Locations of SAS/SATA drives and NVMe SSDs

  3. Press the release button on the drive. See (1) in Figure 5-28.

    The ejector lever automatically ejects.

    Figure 5-28 Removing a drive

  4. Open the drive ejector lever completely. See (2) in Figure 5-28.
  5. Hold the drive ejector lever, and pull out the drive for about 3 cm (1.18 in.). Wait for at least 30 seconds, and remove the drive from the slot. See (3) in Figure 5-28.
  6. Place the removed drive in an ESD bag.
  7. (Optional) Install a filler module in the drive slot if you do not want to install a drive immediately.

Removing an NVMe SSD

The following describes how to remove an NVMe SSD in different OSs.

NOTE:
  • Back up data on the NVMe SSD to be removed.
  • Run commands to uninstall the SSD driver.
  • Observe the SSD indicators. If the green indicator is off and the yellow indicator blinks at 0.5 Hz, the SSD can be removed.
  • Remove the NVMe SSDs one by one. Wait 5 seconds after the removal of an NVMe SSD and then proceed to the next one.
Windows
  1. Stop read/write access to the NVMe SSD to be removed.
  2. Query the mapping between slot IDs and drive letters of NVMe SSDs.

    The following uses Windows Server 2012 R2 as an example.

    1. Choose Server Manager > Computer Management > Disk Management.
    2. Right-click a drive and choose Properties from the shortcut menu. In the displayed dialog box, view the slot ID of the drive.
    3. Record the drive letter of the NVMe SSD according to the slot ID of the drive. See Figure 5-29.
      Figure 5-29 Mapping between the slot ID and drive letter of an NVMe SSD
    4. Find the physical slot of the drive according to the slot ID. For the mapping between the slot ID and the physical slot of the drive, see Table 5-1.
      Table 5-1 Mapping among the physical slot numbers, B/D/F numbers, and slot IDs

      PCIe Device

      CPU

      Physical Slot Number

      Root Port B/D/F

      Device B/D/F

      Slot ID

      NVMe SSD 4

      CPU 1

      Slot 4

      17/2/0

      19/0/0

      84

      NVMe SSD 5

      CPU 1

      Slot 5

      17/3/0

      1A/0/0

      85

      NVMe SSD 6

      CPU 2

      Slot 6

      85/0/0

      86/0/0

      86

      NVMe SSD 7

      CPU 2

      Slot 7

      85/1/0

      87/0/0

      87

      NVMe SSD 8

      CPU 2

      Slot 8

      85/2/0

      88/0/0

      88

      NVMe SSD 9

      CPU 2

      Slot 9

      85/3/0

      89/0/0

      89

      Huawei official website

    5. Install an NVMe SSD driver and NVMe SSD management tool. For Huawei ES3000 V3 NVMe SSDs, you need to install the NVMe SSD driver hiodriver and the NVMe SSD management tool NVMe Toolbox. NVMe Toolbox is used to query NVMe SSD information, upgrade firmware, and hot remove and insert NVMe SSDs.
      NOTE:

      After installing the ES3000 V3 NVMe SSDs, install the packages required by the OS to manage the SSDs. NVMe Toolbox is incompatible with the built-in NVMe driver of Windows. Therefore, install Huawei's NVMe driver before using the tool. You can obtain the installation packages from the Huawei official website. For details about how to install the packages, see the HUAWEI ES3000 V3 NVMe PCIe SSD V100R003C50SPC200 Upgrade Guide (ES3500, ES3600, and ES3620).

  3. In the program(x86)\hioadm directory of drive C, run NVMe Toolbox to view running NVMe SSDs.

    Figure 5-30 Viewing running NVMe SSDs

  4. Select the NVMe SSD to be ejected and click eject.

    After the SSD is ejected, a success message is displayed, as shown in the following figure.
    Figure 5-31 Ejection success message

    After the NVMe SSD is ejected, observe the NVMe SSD indicators. When the green indicator is off and the yellow indicator blinks at 0.5 Hz, you can slowly remove the NVMe SSD.

    For details about how to manually remove an NVMe SSD, see Removing a SAS/SATA Drive.

Linux

Stop I/O operations on the NVMe SSD to be removed and unmount any file systems mounted to the SSD. If the SSD is used by the mdadm utility, you also need to set the SSD to the fail state.

Procedure
  1. Set a kernel parameter.

    1. Add pciehp.pciehp_force=1 pci=pcie_bus_perf to the grub.cfg file. After the configuration, restart the compute node for the configuration to take effect.
    2. Log in to the OS, and open /boot/efi/EFI/redhat/grub.cfg (in UEFI mode) or /boot/grub2/grub.cfg (in Legacy mode).
      • Example 1: RHEL 7.3 in UEFI mode
        1. Run the cat /proc/cmdline command. In the displayed message, look for pciehp.pciehp_force=1pci=pcie_bus_perf. If it cannot be found, proceed with the following steps.

        2. Run the vi /boot/efi/EFI/redhat/grub.cfg command. On the displayed screen, locate the message displayed after you run cat /proc/cmdline in 1.b.i, and enter pciehp.pciehp_force=1 pci=pcie_bus_perf after it. Between the added content and its preceding content, a space is required and no line feed is allowed.

        3. Save the changes and restart the compute node.
        4. After the restart, run the cat /proc/cmdline command to check whether the kernel parameter is set successfully.

      • Example 2: SUSE 12.2 in UEFI mode
        1. Run the cat /proc/cmdline. In the displayed message, look for pciehp.pciehp_force=1pci=pcie_bus_perf. If it cannot be found, proceed with the following steps.

        2. Run the vi /boot/efi/EFI/sles/grub.cfg command. On the displayed screen, locate the message displayed after you run cat /proc/cmdline in 1.b.i, and enter pciehp.pciehp_force=1 pci=pcie_bus_perf after it. Between the added content and its preceding content, a space is required and no line feed is allowed.

        3. Save the changes and restart the compute node.
        4. After the restart, run the cat /proc/cmdline command to check whether the kernel parameter is set successfully.

  2. Upgrade the kernel.

    • Example 1: RHEL 7.3

      The kernel of the RHEL 7.3 OS must be upgraded to kernel-3.10.0-514.26.2.e17.x86_64 or later to support orderly hot swap of NVMe SSDs.

      1. Run the uname -r command to view the current kernel version.
        Figure 5-32 Original kernel version
      2. Log in to the Red Hat official website and download the kernel upgrade package (kernel-3.10.0-514.26.2.e17.x86_64 is used as an example).
      3. Run the following command to upload the upgrade package to the OS:

        rpm -ivh kernel-3.10.0-514.26.2.e17.x86_64.rpm

      4. Install the upgrade package. Restart the OS after the kernel upgrade.
      5. Run the uname -r command to verify the upgrade.

        If "3.10.0-514.26.2.el7.x86_64" is displayed, the upgrade is successful.

        Figure 5-33 New kernel version
    • Example 2: SUSE 12.2

      The kernel of the SUSE 12.2 OS must be upgraded to kernel-default-4.4.74-92.32.1.x86_64 or later to support orderly hot swap of NVMe SSDs.

      1. Run the uname -r command to view the current kernel version.
        Figure 5-34 Original kernel version
      2. Log in to the SUSE official website and download the kernel upgrade package (kernel-3.10.0-514.26.2.e17.x86_64 is used as an example).
      3. Run the following command to upload the upgrade package to the OS:

        rpm -ivh kernel-default-4.4.74-92.32.1.x86_64.rpm

        Figure 5-35 Installing the upgrade package
      4. Restart the OS after the upgrade.
      5. Run the uname -r command to verify the upgrade.

        If it is still the original kernel version, modify the version in the grub.cfg file.

  3. Remove the NVMe SSD in orderly hot swap mode.

    1. Obtain the slot number, B/D/F number, and slot ID of the NVMe SSD in the OS.

      Generally, the slot number is the same as that marked on the compute node panel. If they are inconsistent:

      • Run the find /sys -name nvme0n1 command to obtain the bus number of the SSD.
      • Run the lspci -vvv -xxx -s BDF |grep -i slot command to obtain the slot number.
        Figure 5-36 Querying the slot number, B/D/F number, and slot ID
      Table 5-2 Mapping among the physical slot numbers, B/D/F numbers, and slot IDs

      PCIe Device

      CPU

      Physical Slot Number

      Root Port B/D/F

      Device B/D/F

      Slot ID

      NVMe SSD 4

      CPU 1

      Slot 4

      17/2/0

      19/0/0

      84

      NVMe SSD 5

      CPU 1

      Slot 5

      17/3/0

      1A/0/0

      85

      NVMe SSD 6

      CPU 2

      Slot 6

      85/0/0

      86/0/0

      86

      NVMe SSD 7

      CPU 2

      Slot 7

      85/1/0

      87/0/0

      87

      NVMe SSD 8

      CPU 2

      Slot 8

      85/2/0

      88/0/0

      88

      NVMe SSD 9

      CPU 2

      Slot 9

      85/3/0

      89/0/0

      89

    2. Change the value of the a8 register. Otherwise, NVMe SSDs do not support orderly hot swap.
      1. Run the following command to view the value of the a8 register and record the value (the PCI data in row a0, column 9).

        lspci -xxx -s <B/D/F>

        <B/D/F>: Enter the root port (B/D/F) of the NVMe SSD. For details, see Table 5-2.

        Figure 5-37 Original value of a8 register
        NOTE:

        If the value of the a8 register is not f1, contact Huawei engineers.

      2. Run the following command to change the value to e1:

        setpci -s <B/D/F> a8.B=e1

        <B/D/F>: Enter the root port (B/D/F) of the NVMe SSD. For details, see Table 5-2.

      3. Run the following command to check whether the change takes effect:

        lspci -s <B/D/F> -xxx

        <B/D/F>: Enter the root port (B/D/F) of the NVMe SSD. For details, see Table 5-2.

        Figure 5-38 New value of a8 register
    3. Stop the read/write operations on the NVMe SSD.

      Stop service I/Os.

      • If the drive has been mounted to a path, unmount it first.
      • If the drive is used in the RAID software mdadm of the linux OS, set the drive to fail.

      Command:

      umount /dev/NVMe SSD driver letter

    4. Run the following command to safely remove the SSD.

      echo n > /sys/bus/pci/slots/Slot ID/power

      NOTE:

      Parameter description:

      • n: The value can be 0 or 1. The value 0 indicates removal, and the value 1 indicates insertion. The value 1 can be used only when a removal command is run but no removal operation is performed.

      For example, to hot remove the NVMe SSD in slot 2 in an orderly manner, run the following command:

      echo 0 > /sys/bus/pci/slots/81/power

    5. Observe the NVMe SSD indicators. If the green indicator is off and the yellow indicator blinks at 0.5 Hz, the NVMe SSD can be removed.

      Remove the NVMe SSD by referring to the instructions provided in Removing a SAS/SATA Drive.

    6. Place the removed drive in an ESD bag.
    7. After the orderly hot removal, run the following command to restore the value of the a8 register to f1:

      setpci -s < B/D/F> a8.B=f1

      < B/D/F>: Enter the root port (B/D/F) of the NVMe SSD. For details, see Table 5-2.

      If the value of the register is not restored to the original one, the hot insertion of the NVMe SSD may be abnormal.

      Figure 5-39 Restoring the value of a8 register
    8. If you do not install another drive into the slot immediately, install a filler module.

Translation
Download
Updated: 2018-12-14

Document ID: EDOC1100031432

Views: 33867

Downloads: 48

Average rating:
This Document Applies to these Products
Related Documents
Related Version
Share
Previous Next