No relevant resource is found in the selected language.
Enterprise
Worldwide
Huawei Global - English
Search
Support Documentation
Atlas 800 Inference Server Maintenance and Service Guide (Model 3010) 16
Rate and give feedback:
Huawei uses machine translation combined with human proofreading to translate this document to different languages in order to help you better understand the content of this document. Note: Even the most advanced machine translation cannot match the quality of professional translators. Huawei shall not bear any responsibility for translation accuracy and it is recommended that you refer to the English document (a link for which has been provided).
Removing an NVMe SSD

Removing an NVMe SSD

  • If the VMD function is enabled and the latest VMD driver is installed, the NVMe drives support surprise hot swap.
  • If the VMD function is disabled, NVMe SSDs support only orderly hot swapping.

Removing a Front NVMe Drive

Procedure (VMD Enabled)

  • Use the Computing Product Compatibility Checker to obtain information about the operating systems supported by the VMD function.
  • The VMD function requires the following software versions:
    • BIOS 090 or later
    • BMC 312 or later
    • Latest CPLD version of the drive backplane
  • Before removing an NVMe drive:
    • Stop accessing the NVMe drive.
    • Back up data of the NVMe drive to prevent data loss.
  • Remove the NVMe drives one by one. Remove the next NVMe drive only after the previous one is removed completely.
  • If the OS is RHEL 7.3 and the VMD function is enabled, the OS kernel 3.10.0-514.el7.x86_64 must be used for surprise hot swap of NVMe drives. Otherwise, the operating system may restart.
  1. Check whether a front bezel is installed.

  2. Remove the front bezel.

    For details, see Removing the Front Bezel.

  3. Determine the slot and position of the drive.

    For details, see Drive Configurations and Drive Numbers.

  4. Check the running status of the drive by observing its indicators.

    For details, see SAS/SATA Drive Indicators.

  5. Determine whether the drive needs to be removed.

    • If yes, go to Step 6.
    • If no, rectify the drive fault.

  6. Remove the drive.

    1. Press the ejector release button on the drive panel to release the ejector lever. See (1) in Figure 6-32.
    2. Open the drive ejector lever completely. See (2) in Figure 6-32.
    3. Hold the ejector lever of the hard drive, and pull out the hard drive for about 3 cm (1.18 in.). Wait for at least 30 seconds and remove the drive from the server. See (3) in Figure 6-32.

      Do not insert or remove drives frequently. If a drive is frequently removed and installed in an interval shorter than 30 seconds, there are risks that the drive cannot be identified.

    Figure 6-32 Removing a drive

  7. Place the removed part in an ESD bag.
  8. Install a drive filler.

    Perform this operation only when a drive is not installed immediately.

Procedure (VMD Disabled)

  • Before removing an NVMe drive:
    • Stop accessing the NVMe drive.
    • Back up data of the NVMe drive to prevent data loss.
  • Remove the NVMe drives one by one. Remove the next NVMe drive only after the previous one is removed completely.
  • Run commands to uninstall the drive driver.

Windows OS

Use the Computing Product Compatibility Checker to obtain information about the Windows operating systems supported.

  1. Check whether a front bezel is installed.

  2. Remove the front bezel.

    For details, see Removing the Front Bezel.

  3. Determine the slot and position of the drive.

    For details, see Drive Configurations and Drive Numbers.

  4. Check the running status of the drive by observing its indicators.

    For details, see NVMe Drive Indicators.

  5. Determine whether the drive needs to be removed.

    • If yes, go to Step 6.
    • If no, rectify the drive fault.

  6. Install the tool package for the Windows operating system.

    For details, see Installation and Configuration > Installing the Tool Package > Installing the Tool Package (Windows) in the ES3000 V5 NVMe PCIe SSD User Guide.

  7. Double-click the HioadmGUI_DotNet_x.0.exe file in the tool installation directory.

    • If the .NET Framework version is 2.0 or 3.5, double-click the HioadmGUI_DotNet_2.0.exe file in the tool installation directory.
    • If the .NET Framework version is 4.0 or 4.5, double-click the HioadmGUI_DotNet_4.0.exe file in the tool installation directory.
    Figure 6-33 BaseInfo page

  8. Select the NVMe drive to be removed.
  9. Click eject.

    After the drive is removed, "Eject Succeed" will be displayed.

    Figure 6-34 BaseInfo page

    If a failure message is displayed indicating that the NVMe drive is being accessed, stop the related services and click eject again.

  10. Click OK.
  11. Observe the indicator status of the NVMe drive. If the green indicator is off and the yellow indicator blinks at 0.5 Hz, remove the NVMe drive slowly.

    For details about the drive indicator status, see NVMe Drive Indicators.

    For details about how to manually remove an NVMe drive, see Removing an NVMe SSD.

  12. Place the removed part in an ESD bag.
  13. Install a drive filler.

    Perform this operation only when a drive is not installed immediately.

Linux OS

  • Use the Computing Product Compatibility Checker to obtain information about the Linux operating systems supported.
  • If the OS is RHEL 7.3 and the VMD function is disabled, the OS kernel must be upgraded to kernel-3.10.0-514.26.2.e17.x86_64 or later to support orderly hot swap of NVMe drives.
  • The following uses RHEL 7.3 as an example.
  1. Check whether a front bezel is installed.

  2. Remove the front bezel.

    For details, see Removing the Front Bezel.

  3. Determine the slot and position of the drive.

    For details, see Drive Configurations and Drive Numbers.

  4. Check the running status of the drive by observing its indicators.

    For details, see NVMe Drive Indicators.

  5. Determine whether the drive needs to be removed.

    • If yes, go to Step 6.
    • If no, rectify the drive fault.

  6. If the operating system is RHEL 7.3, upgrade the kernel to kernel-3.10.0-514.26.2.e17.x86_64 or later.

    1. Log in to the server whose OS kernel is to be upgraded as the root user.
    2. Check the current kernel version.

      uname -a

    3. Download the kernel upgrade package to the local PC.

      Log in to the official Red Hat website using a customer portal account and download the kernel upgrade package (kernel-3.10.0-514.26.2.el7.x86_64.rpm for example).

      Link: https://access.redhat.com/downloads/content/rhel---7/x86_64/2456/kernel/3.10.0-514.26.2.el7/x86_64/fd431d51/package

    4. Use the KVM to mount the directory where the upgrade package is located.
      Figure 6-35 Mounting the directory
    5. Mount the upgrade package to a directory on the OS, for example, /home.

      mount /dev/sr0 /home

    6. Check the upgrade package in the /home directory.

      cd /home

      ls kernel*

    7. Run the following command to upgrade the system kernel:

      rpm -ivh kernel*.x86_64.rpm

    8. Run the following command to reboot the server and go to the new kernel:

      reboot

    9. Run the following command to check whether the upgrade is successful:

      uname -r

      The system displays the following information:

      kernel-3.10.0-514.26.2.el7.x86_64

  7. Configure kernel parameters.

    1. Log in to the operating system and open the /etc/default/grub file.
    2. Press Insert.
    3. Locate GRUB_CMDLINE_LINUX="crashkernel=auto rd.lvm.lv=rhel/root rd.lvm.lv=rhel/swap rhgb quiet and add a space and then pciehp.pciehp_force=1 pci=pcie_bus_perf.

      Do not add the content in a new line.

      Figure 6-36 Configuring kernel parameters
    4. Press Esc.
    5. Enter :wq.

      Save the modifications and exit.

    6. Run the following command for the modification to take effect:
      • Legacy mode: run grub2-mkconfig -o /boot/grub2/grub.cfg
      • UEFI mode: run grub2-mkconfig -o /boot/efi/EFI/redhat/grub.cfg
    7. Restart the server.

  8. If the operating system is RHEL 7.3 or RHEL 7.4, change the value of the a8 register. Otherwise, the NVMe PCIe drives do not support orderly hot swap.

    Skip this step if the server has 24 x 2.5" NVMe drives. You do not need to change the a8 register value for the server with 24 x 2.5" NVMe drives.

    1. Run the following command to record the original a8 register value (the PCI data in column 9 of row a0), as shown in Figure 6-37.

      lspci -s <B/D/F> -xxx

      <B/D/F>: indicates the root port (B/D/F) of the NVMe drive. Table 6-3 lists the B/D/F information.

      Table 6-3 Drive B/D/F information

      Configuration

      Drive Configuration

      Root Port (B/D/F)

      Device (B/D/F)

      8 x 2.5" drive configuration

      44

      d7:00.0

      d8:00.0

      45

      d7:01.0

      d9:00.0

      46

      d7:02.0

      da:00.0

      47

      d7:03.0

      db:00.0

      20 x 2.5" (8 x SAS/SATA + 12 x NVMe) drive configurationa

      8

      3A:00.0

      3B:00.0

      9

      3A:01.0

      3C:00.0

      10

      3A:02.0

      3D:00.0

      11

      3A:03.0

      3E:00.0

      12

      5D:00.0

      5E:00.0

      13

      5D:01.0

      5F:00.0

      14

      85:00.0

      86:00.0

      15

      85:01.0

      87:00.0

      16

      85:02.0

      88:00.0

      17

      85:03.0

      89:00.0

      18

      AE:02.0

      AF:00.0

      19

      AE:03.0

      B0:00.0
      • a: If no FlexIO card is configured, Device (B/D/F) is AF:00.0 for drive 18 and B0:00.0 for drive 19. If FlexIO cards are configured, Device (B/D/F) is B1:00.0 for drive 18 and B2:00.0 for drive 19.
      • The Device (B/D/F) values vary depending on the setting of the VMD function. The values in this table are based on the assumption that the VMD function is disabled.

      For example, if the drive No. is 44, run the following command:

      lspci -s d7:00.0 -xxx

      Figure 6-37 Querying the original register value

      If the default a8 register value is not f1, contact Huawei technical support.

    2. Run the following command to change the a8 register value to e1.

      setpci -s <B/D/F> a8.B=e1

      <B/D/F>: indicates the root port (B/D/F) of the NVMe drive. Table 6-3 lists the B/D/F information.

    3. Run the following command to check the modification result:

      lspci -s <B/D/F> -xxx

      <B/D/F>: indicates the root port (B/D/F) of the NVMe drive. Table 6-3 lists the B/D/F information.

      Figure 6-38 Querying the new register value

  9. Stop accessing the NVMe drive.

    If a file system is mounted to the NVMe drive, run the following command to unmount the file system from it:

    umount /dev/NVMe drive letter

  10. Obtain the mapping between the server drive IDs and the slot IDs on the operating system.

    • For details about the drive numbers and positions, see Drive Numbers.
    • The slot ID is the sum of the drive No. plus 80. For example, if the NVMe drive No. is 5, the slot ID is 85.
    Table 6-4 Mapping

    Configuration

    Drive Configuration

    Slot ID

    8 x 2.5" drive configuration

    44 to 47

    124 to 127

    20 x 2.5" (8 x SAS/SATA + 12 x NVMe) drive configuration

    8 to 19

    88 to 99

    24 x 2.5" NVMe drive configuration

    0 to 23

    80 to 103

  11. Remove the NVMe SSD.

    echo n > /sys/bus/pci/slots/Slot ID/power

    The parameters are described as follows:

    • The value of n can be 0 or 1. The value 0 indicates unloading a drive, and the value 1 indicates loading a drive. You can load a drive only before the unloaded drive is removed.
    • Slot ID: see Table 6-4.

    For example, to orderly hot remove the NVMe drive in slot 44, run the following command:

    echo 0 > /sys/bus/pci/slots/124/power

  12. If the OS uses the irqbalance service to balance CPU interrupts, restart the service after performing a hot swap.

    systemctl restart irqbalance.service

    For details about configuration and management of the irqbalance service, see the related OS document.

  13. Observe the indicator status of the NVMe drive. If the green indicator is off and the yellow indicator blinks at 0.5 Hz, remove the NVMe drive slowly.

    For details about the drive indicator status, see NVMe Drive Indicators.

    For details about how to manually remove an NVMe drive, see Removing an NVMe SSD.

  14. Place the removed part in an ESD bag.
  15. Run the following command to restore the value of the a8 register to the original value:

    setpci -s <B/D/F> a8.B=Original a8 register value

    <B/D/F>: indicates the root port (B/D/F) of the NVMe drive. Table 6-3 lists the B/D/F information.

    • Skip this step if the server has 24 x 2.5" NVMe drives. You do not need to change the a8 register value for the server with 24 x 2.5" NVMe drives.
    • If the original register value is not restored, the orderly hot swap function of the NVMe SSD in the slot is abnormal.

  16. Install a drive filler.

    Perform this operation only when a drive is not installed immediately.

Removing a Rear NVMe Drive

Procedure (VMD Enabled)

  • Use the Computing Product Compatibility Checker to obtain information about the operating systems supported by the VMD function.
  • The VMD function requires the following software versions:
    • BIOS 090 or later
    • BMC 312 or later
    • Latest CPLD version of the drive backplane
  • Before removing an NVMe drive:
    • Stop accessing the NVMe drive.
    • Back up data of the NVMe drive to prevent data loss.
  • Remove the NVMe drives one by one. Remove the next NVMe drive only after the previous one is removed completely.
  • If the OS is RHEL 7.3 and the VMD function is enabled, the OS kernel 3.10.0-514.el7.x86_64 must be used for surprise hot swap of NVMe drives. Otherwise, the operating system may restart.
  1. Determine the slot and position of the drive.

    For details, see Drive Configurations and Drive Numbers.

  2. Check the running status of the drive by observing its indicators.

    For details, see SAS/SATA Drive Indicators.

  3. Determine whether the drive needs to be removed.

    • If yes, go to Step 4.
    • If no, rectify the drive fault.

  4. Remove the drive.

    1. Press the ejector release button on the drive panel to release the ejector lever. See (1) in Figure 6-39.
    2. Open the drive ejector lever completely. See (2) in Figure 6-39.
    3. Hold the ejector lever of the hard drive, and pull out the hard drive for about 3 cm (1.18 in.). Wait for at least 30 seconds and remove the drive from the server. See (3) in Figure 6-39.

      Do not insert or remove drives frequently. If a drive is frequently removed and installed in an interval shorter than 30 seconds, there are risks that the drive cannot be identified.

    Figure 6-39 Removing a drive

  5. Place the removed part in an ESD bag.
  6. Install a drive filler.

    Perform this operation only when a drive is not installed immediately.

Procedure (VMD Disabled)

  • Before removing an NVMe drive:
    • Stop accessing the NVMe drive.
    • Back up data of the NVMe drive to prevent data loss.
  • Remove the NVMe drives one by one. Remove the next NVMe drive only after the previous one is removed completely.
  • Run commands to uninstall the drive driver.

Windows OS

Use the Computing Product Compatibility Checker to obtain information about the Windows operating systems supported.

  1. Determine the slot and position of the drive.

    For details, see Drive Configurations and Drive Numbers.

  2. Check the running status of the drive by observing its indicators.

    For details, see NVMe Drive Indicators.

  3. Determine whether the drive needs to be removed.

    • If yes, go to Step 4.
    • If no, rectify the drive fault.

  4. Install the tool package for the Windows operating system.

    For details, see Installation and Configuration > Installing the Tool Package > Installing the Tool Package (Windows) in the ES3000 V5 NVMe PCIe SSD User Guide.

  5. Double-click the HioadmGUI_DotNet_x.0.exe file in the tool installation directory.

    • If the .NET Framework version is 2.0 or 3.5, double-click the HioadmGUI_DotNet_2.0.exe file in the tool installation directory.
    • If the .NET Framework version is 4.0 or 4.5, double-click the HioadmGUI_DotNet_4.0.exe file in the tool installation directory.
    Figure 6-40 BaseInfo page

  6. Select the NVMe drive to be removed.
  7. Click eject.

    After the drive is removed, "Eject Succeed" will be displayed.

    Figure 6-41 BaseInfo page

    If a failure message is displayed indicating that the NVMe drive is being accessed, stop the related services and click eject again.

  8. Click OK.
  9. Observe the indicator status of the NVMe drive. If the green indicator is off and the yellow indicator blinks at 0.5 Hz, remove the NVMe drive slowly.

    For details about the drive indicator status, see NVMe Drive Indicators.

    For details about how to manually remove an NVMe drive, see Removing an NVMe SSD.

  10. Place the removed part in an ESD bag.
  11. Install a drive filler.

    Perform this operation only when a drive is not installed immediately.

Linux OS

  • Use the Computing Product Compatibility Checker to obtain information about the Linux operating systems supported.
  • If the OS is RHEL 7.3 and the VMD function is disabled, the OS kernel must be upgraded to kernel-3.10.0-514.26.2.e17.x86_64 or later to support orderly hot swap of NVMe drives.
  • The following uses RHEL 7.3 as an example.
  1. Determine the slot and position of the drive.

    For details, see Drive Configurations and Drive Numbers.

  2. Check the running status of the drive by observing its indicators.

    For details, see NVMe Drive Indicators.

  3. Determine whether the drive needs to be removed.

    • If yes, go to Step 4.
    • If no, rectify the drive fault.

  4. If the operating system is RHEL 7.3, upgrade the kernel to kernel-3.10.0-514.26.2.e17.x86_64 or later.

    1. Log in to the server whose OS kernel is to be upgraded as the root user.
    2. Check the current kernel version.

      uname -a

    3. Download the kernel upgrade package to the local PC.

      Log in to the official Red Hat website using a customer portal account and download the kernel upgrade package (kernel-3.10.0-514.26.2.el7.x86_64.rpm for example).

      Link: https://access.redhat.com/downloads/content/rhel---7/x86_64/2456/kernel/3.10.0-514.26.2.el7/x86_64/fd431d51/package

    4. Use the KVM to mount the directory where the upgrade package is located.
      Figure 6-42 Mounting the directory
    5. Mount the upgrade package to a directory on the OS, for example, /home.

      mount /dev/sr0 /home

    6. Check the upgrade package in the /home directory.

      cd /home

      ls kernel*

    7. Run the following command to upgrade the system kernel:

      rpm -ivh kernel*.x86_64.rpm

    8. Run the following command to reboot the server and go to the new kernel:

      reboot

    9. Run the following command to check whether the upgrade is successful:

      uname -r

      The system displays the following information:

      kernel-3.10.0-514.26.2.el7.x86_64

  5. Configure kernel parameters.

    1. Log in to the operating system and open the /etc/default/grub file.
    2. Press Insert.
    3. Locate GRUB_CMDLINE_LINUX="crashkernel=auto rd.lvm.lv=rhel/root rd.lvm.lv=rhel/swap rhgb quiet and add a space and then pciehp.pciehp_force=1 pci=pcie_bus_perf.

      Do not add the content in a new line.

      Figure 6-43 Configuring kernel parameters
    4. Press Esc.
    5. Enter :wq.

      Save the modifications and exit.

    6. Run the following command for the modification to take effect:
      • Legacy mode: run grub2-mkconfig -o /boot/grub2/grub.cfg
      • UEFI mode: run grub2-mkconfig -o /boot/efi/EFI/redhat/grub.cfg
    7. Restart the server.

  6. If the operating system is RHEL 7.3 or RHEL 7.4, change the value of the a8 register. Otherwise, the NVMe PCIe drives do not support orderly hot swap.

    Skip this step if the server has 24 x 2.5" NVMe drives. You do not need to change the a8 register value for the server with 24 x 2.5" NVMe drives.

    1. Run the following command to record the original a8 register value (the PCI data in column 9 of row a0), as shown in Figure 6-44.

      lspci -s <B/D/F> -xxx

      <B/D/F>: indicates the root port (B/D/F) of the NVMe drive. Table 6-5 lists the B/D/F information.

      Table 6-5 Drive B/D/F information

      Configuration

      Drive Configuration

      Root Port (B/D/F)

      Device (B/D/F)

      8 x 2.5" drive configuration

      44

      d7:00.0

      d8:00.0

      45

      d7:01.0

      d9:00.0

      46

      d7:02.0

      da:00.0

      47

      d7:03.0

      db:00.0

      20 x 2.5" (8 x SAS/SATA + 12 x NVMe) drive configurationa

      8

      3A:00.0

      3B:00.0

      9

      3A:01.0

      3C:00.0

      10

      3A:02.0

      3D:00.0

      11

      3A:03.0

      3E:00.0

      12

      5D:00.0

      5E:00.0

      13

      5D:01.0

      5F:00.0

      14

      85:00.0

      86:00.0

      15

      85:01.0

      87:00.0

      16

      85:02.0

      88:00.0

      17

      85:03.0

      89:00.0

      18

      AE:02.0

      AF:00.0

      19

      AE:03.0

      B0:00.0
      • a: If no FlexIO card is configured, Device (B/D/F) is AF:00.0 for drive 18 and B0:00.0 for drive 19. If FlexIO cards are configured, Device (B/D/F) is B1:00.0 for drive 18 and B2:00.0 for drive 19.
      • The Device (B/D/F) values vary depending on the setting of the VMD function. The values in this table are based on the assumption that the VMD function is disabled.

      For example, if the drive No. is 44, run the following command:

      lspci -s d7:00.0 -xxx

      Figure 6-44 Querying the original register value

      If the default a8 register value is not f1, contact Huawei technical support.

    2. Run the following command to change the a8 register value to e1.

      setpci -s <B/D/F> a8.B=e1

      <B/D/F>: indicates the root port (B/D/F) of the NVMe drive. Table 6-5 lists the B/D/F information.

    3. Run the following command to check the modification result:

      lspci -s <B/D/F> -xxx

      <B/D/F>: indicates the root port (B/D/F) of the NVMe drive. Table 6-5 lists the B/D/F information.

      Figure 6-45 Querying the new register value

  7. Stop accessing the NVMe drive.

    If a file system is mounted to the NVMe drive, run the following command to unmount the file system from it:

    umount /dev/NVMe drive letter

  8. Obtain the mapping between the server drive IDs and the slot IDs on the operating system.

    • For details about the drive numbers and positions, see Drive Numbers.
    • The slot ID is the sum of the drive No. plus 80. For example, if the NVMe drive No. is 5, the slot ID is 85.
    Table 6-6 Mapping

    Configuration

    Drive Configuration

    Slot ID

    8 x 2.5" drive configuration

    44 to 47

    124 to 127

    20 x 2.5" (8 x SAS/SATA + 12 x NVMe) drive configuration

    8 to 19

    88 to 99

    24 x 2.5" NVMe drive configuration

    0 to 23

    80 to 103

  9. Remove the NVMe SSD.

    echo n > /sys/bus/pci/slots/Slot ID/power

    Parameter description:

    • The value of n can be 0 or 1. The value 0 indicates unloading a drive, and the value 1 indicates loading a drive. You can load a drive only before the unloaded drive is removed.
    • Slot ID: see Table 6-6.

    For example, to orderly hot remove the NVMe drive in slot 44, run the following command:

    echo 0 > /sys/bus/pci/slots/124/power

  10. If the OS uses the irqbalance service to balance CPU interrupts, restart the service after performing a hot swap.

    systemctl restart irqbalance.service

    For details about configuration and management of the irqbalance service, see the related OS document.

  11. Observe the indicator status of the NVMe drive. If the green indicator is off and the yellow indicator blinks at 0.5 Hz, remove the NVMe drive slowly.

    For details about the drive indicator status, see NVMe Drive Indicators.

    For details about how to manually remove an NVMe drive, see Removing an NVMe SSD.

  12. Place the removed part in an ESD bag.
  13. Run the following command to restore the value of the a8 register to the original value:

    setpci -s <B/D/F> a8.B=Original a8 register value

    <B/D/F>: indicates the root port (B/D/F) of the NVMe drive. Table 6-5 lists the B/D/F information.

    • Skip this step if the server has 24 x 2.5" NVMe drives. You do not need to change the a8 register value for the server with 24 x 2.5" NVMe drives.
    • If the original register value is not restored, the orderly hot swap function of the NVMe SSD in the slot is abnormal.

  14. Install a drive filler.

    Perform this operation only when a drive is not installed immediately.

Translation
简体中文
Download
Updated: 2023-01-31

Document ID: EDOC1100115123

Views: 236300

Downloads: 56

Average rating:
This Document Applies to these Products

Related Version

Related Documents

Digital Signature File

Digital Signature Authentication Mode
Share
Previous Next

Copyright © 2023 Huawei Technologies Co., Ltd. All rights reserved.

You are going to visit :