No relevant resource is found in the selected language.

This site uses cookies. By continuing to browse the site you are agreeing to our use of cookies. Read our privacy policy>Search

Reminder

To have a better experience, please upgrade your IE browser.

upgrade

CH225 V3 Compute Node V100R001 User Guide 07

This document describes the appearance, functions, and structure of the CH225 V3 storage expansion compute node (CH225 V3 for short), and methods of installing, removing, and configuring the CH225 V3.
Rate and give feedback:
Huawei uses machine translation combined with human proofreading to translate this document to different languages in order to help you better understand the content of this document. Note: Even the most advanced machine translation cannot match the quality of professional translators. Huawei shall not bear any responsibility for translation accuracy and it is recommended that you refer to the English document (a link for which has been provided).
Removing a Hard Disk

Removing a Hard Disk

This section describes the precautions and procedure for removing a hard disk from a CH225 V3.

Scenarios

Remove a hard disk in any of the following scenarios:

  • The hard disk is faulty.
  • The hard disk needs to be replaced with a newer model.
  • The hard disk has no free space.
NOTE:
  • For details about how to erase data from a hard disk, see Erasing Storage Media Data.
  • Before removing a hard disk, check the following:
    • If no OS is installed on the hard disk, the compute node does not need to be powered off.
    • If an OS is installed on the hard disk, and the hard disk is configured in a RAID with redundancy, the compute node does not need to be powered off.
    • If an OS is installed on the hard disk, and the hard disk is not configured in a RAID or is configured in a RAID without redundancy (for example, RAID 0), the compute node needs to be powered off.

The CH225 V3 supports a maximum of two 2.5-inch HDDs/SSDs and twelve 2.5-inch NVMe PCIe SSDs, which are shown in Figure 8-1.

Figure 8-1 Locations of HDDs/SSDs and NVMe PCIe SSDs
Table 8-2 CH225 V3 hard disk slot numbering

Disk0

Disk2

Disk3

Disk4

Disk5

Disk1

Disk6

Disk7

Disk8

Disk9

Disk10

Disk11

Disk12

Disk13

Prerequisites

Conditions

You have determined whether data backup is required.

  • If the hard disk to be removed is not configured in a RAID or is configured in a RAID without redundancy (for example, RAID 0), you have need to back up data on the hard disk.
  • If the hard disk to be removed is configured in a RAID with redundancy, you can remove the hard disk without data backup.

Data

Data preparation is not required for this operation.

Tools

  • Electrostatic discharge (ESD) gloves.
  • Packing materials, for example, an ESD bag.

Procedure

Remove a SAS or SATA HDD/SSD

  1. Determine the HDD/SSD to be removed.
  2. Figure 8-1 shows the positions of HDDs/SSDs.
  3. Push the release button that fastens the HDD/SSD ejector lever. See step (1) in Figure 8-2.

    The ejector lever automatically ejects.

    Figure 8-2 Removing the HDD/SSD

  4. Fully open the ejector lever. See step (2) in Figure 8-2.
  5. Hold the ejector lever of the hard disk, and pull out the hard disk for about 3 cm (1.18 in.). Wait for at least 30s until the hard disk stops working. Remove the hard disk from the slot. See step (3) in Figure 8-2.
  6. Place the removed hard disk in an ESD bag.
  7. (Optional) Install a filler module in the hard disk slot if you do not want to install a hard disk immediately.

Removing an NVMe PCIe SSD

NOTE:
  • Back up data on the NVMe SSD to be removed.
  • Run commands to uninstall the SSD driver.
  • Observe the SSD indicators. If the green indicator is off and the yellow indicator blinks at 0.5 Hz, the SSD can be removed.
  • Remove the NVMe SSDs one by one. Wait 5 seconds after the removal of an NVMe SSD and proceed to the next NVMe SSD.

Windows

  1. Stop read/write access to the NVMe SSD to be removed.
  2. Query the mapping between slot IDs and drive letters of NVMe SSDs.

    In this step, Windows Server 2012 is used as an example.
    1. Choose Server Manager > Computer Management > Disk Management.
    2. Right-click a hard disk and choose Properties from the shortcut menu. In the displayed dialog box, view the slot ID of the hard disk.
    3. Determine whether the hard disk is an NVMe SSD according to slot ID. If it is an NVMe SSD, record its slot ID (see Figure 8-3). Locate the physical slot number of the NVMe SSD based on the slot ID. For details about the mapping between slot IDs and physical slot numbers, see Table 8-3.
      Figure 8-3 Mapping between the slot ID and drive letter of an NVMe SSD
      Table 8-3 Mapping among the physical slot number, B/D/F number, and slot ID

      Configuration

      Physical Slot Number

      Root Port (B/D/F)

      Device (B/D/F)

      Slot ID

      With two 2.5-inch NVMe SSDs

      Slot2

      00:01.1

      0f:00.0

      2

      Slot3

      00:02.2

      07:00.0

      3

      Slot4

      80:00.0

      83:00.0

      4

      Slot5

      82:02.0

      84:00.0

      5

      Slot6

      00:01.1

      10:00.0

      6

      Slot7

      00:02.0

      05:00.0

      7

      Slot8

      80:01.0

      82:00.0

      8

      Slot9

      80:02.3

      87:00.0

      9

      Slot10

      00:02.3

      08:00.0

      10

      Slot11

      00:02.1

      06:00.0

      11

      Slot12

      80:02.1

      85:00.0

      12

      Slot13

      80:02.2

      86:00.0

      13

    4. The following uses Huawei ES3000 V3 NVMe SSDs as an example to describe how to install the NVMe SSD driver hiodriver and the NVMe hard disk management tool NVMe Toolbox. NVMe Toolbox is used to query NVMe SSD information, upgrade firmware, and hot remove and insert NVMe SSDs.
      NOTE:

      After installing the ES3000 V3 NVMe SSDs, install the packages required by the OS to manage the SSDs. NVMe Toolbox is incompatible with the built-in NVMe driver of Windows. Therefore, install Huawei's NVMe driver before using the tool. You can obtain the installation packages from the Huawei official website. For details about how to install the packages, see the Huawei ES3000 V3 NVMe PCIe SSD V100R003C50SPC200 Upgrade Guide (ES3500&ES3600&ES3620).

  3. In the program(x86)\hioadm directory of drive C, run NVMe Toolbox to view the running NVMe SSDs.

    Figure 8-4 Viewing running NVMe SSDs

  4. Select the NVMe SSD to be ejected and click eject.

    After the SSD is ejected, a success message is displayed, as shown in the following figure.
    Figure 8-5 Ejection success message

    After the NVMe SSD is ejected, observe the NVMe SSD indicators. When the green indicator is off and the yellow indicator blinks at 0.5 Hz, you can slowly remove the NVMe SSD.

    Remove the SSD manually. The removal is the same as removing a SAS or SATA HDD/SSD.

Linux

Stop I/O operations on the NVMe SSD to be removed and unmount any file systems mounted to the SSD. If the SSD is used by the mdadm utility, you also need to set the SSD to the fail state.

  1. Set a kernel parameter.

    Add pciehp.pciehp_force=1 pci=pcie_bus_perf to the grub.cfg file. After the configuration, restart the compute node for the configuration to take effect.

    Log in to the OS, and open /boot/grub2/grub.cfg (in Legacy mode).

    Take the installation of Red Hat Enterprise Linux (RHEL) 7.1 in Legacy mode as an example.
    1. Run the cat /proc/cmdline command. In the command output, look for pciehp.pciehp_force=1 pci=pcie_bus_perf. If it cannot be found, you need to configure kernel parameters.
      Figure 8-6 Command output
    2. Run the vi /boot/grub2/grub.cfg command. On the displayed screen, locate the message displayed after you run cat /proc/cmdline in 1.a, and enter pciehp.pciehp_force=1 pci=pcie_bus_perf after it. Between the added content and its preceding content, a space is required and no line feed is allowed.
      Figure 8-7 Setting a kernel parameter
    3. Save the changes and restart the compute node.
    4. After the restart, run the cat /proc/cmdline command to check whether the kernel parameter is set successfully.
      Figure 8-8 Checking whether the kernel configuration is successful

  2. Upgrade the kernel or driver.

    The following takes RHEL 7.1 as an example.

    Take the Huawei-developed ES3000 V3 as an example, the kernel or driver must be upgraded. Otherwise, the NVMe SSD does not support the orderly hot swap function.

    1. Run the uname -r command to view the current kernel version.
      Figure 8-9 Querying the kernel or driver version
    2. Download the kernel or driver upgrade package. (kmod-hiodriver-4.0.2.1-3.10.0_229.el7.redhat.x86_64.rpm is used as an example.)

      Download link: http://support.huawei.com/enterprise/en/server/es3000-v3-pid-21500649/software

    3. Upload the upgrade package to the OS and run the following command to install it:

      rpm -ivh kmod-hiodriver-4.0.2.1-3.10.0_229.el7.redhat.x86_64.rpm

    4. Install the upgrade package. Restart the OS after the kernel upgrade.
    5. Run the uname -r command to verify the upgrade.

      If "3.10.0-229.el7.x86_64" is displayed, the upgrade is successful.

      If the upgrade does not take effect, you need to manually modify the grub.cfg file.

      Figure 8-10 Querying the kernel or driver version

  3. Remove the NVMe SSD in scheduled hot swap mode.

    1. Obtain the slot number, B/D/F number, and slot ID of the NVMe SSD in the OS.

      Generally, the slot number is the same as that marked on the compute node panel. If they are inconsistent:

      • Run the find /sys -name nvme0n1 command to obtain the bus number of the SSD.
      • Run the lspci -vvv -xxx-s BDF command to obtain the slot number.
      Figure 8-11 Querying the slot number, B/D/F number, and slot ID
      Table 8-4 Mapping among the physical slot number, B/D/F number, and slot ID

      Configuration

      Physical Slot Number

      Root Port (B/D/F)

      Device (B/D/F)

      Slot ID

      With two 2.5-inch NVMe SSDs

      Slot2

      00:01.1

      0f:00.0

      2

      Slot3

      00:02.2

      07:00.0

      3

      Slot4

      80:00.0

      83:00.0

      4

      Slot5

      82:02.0

      84:00.0

      5

      Slot6

      00:01.1

      10:00.0

      6

      Slot7

      00:02.0

      05:00.0

      7

      Slot8

      80:01.0

      82:00.0

      8

      Slot9

      80:02.3

      87:00.0

      9

      Slot10

      00:02.3

      08:00.0

      10

      Slot11

      00:02.1

      06:00.0

      11

      Slot12

      80:02.1

      85:00.0

      12

      Slot13

      80:02.2

      86:00.0

      13

    2. Change the value of the a8 register. Otherwise, NVMe SSDs do not support scheduled hot swap.
      1. Run the following command to view the value of the a8 register and record the value (the PCI data in row a0, column 9).

        lspci -xxx -s <B/D/F>

        <B/D/F>: Enter the root port (B/D/F) of the NVMe SSD. For details, see Table 8-4.

        Figure 8-12 Original value of a8 register
        NOTE:

        If the value of the a8 register is not f1, contact Huawei engineers.

      2. Run the following command to change the value to e1:

        setpci -s <B/D/F> a8.B=e1

        <B/D/F>: Enter the root port (B/D/F) of the NVMe SSD. For details, see Table 8-4.

      3. Run the following command to check whether the change takes effect:

        lspci -xxx-s <B/D/F>

        <B/D/F>: Enter the root port (B/D/F) of the NVMe SSD. For details, see Table 8-4.

        Figure 8-13 New value of a8 register
    3. If the SSD is used by the mdadm utility, you also need to set the SSD to the fail state. Stop I/O operations on the NVMe SSD to be removed and unmount any file systems mounted to the SSD.

      Run the following command:

      umount /dev/NVMe SSD drive letter

    4. Determine the SSD to be removed.
      • NVMe PCIe SSDs do not support violence hot-swap.
      • The interval between removing or inserting two hard disks must be greater than 30 seconds.

      Figure 8-1 shows the positions of SSDs.

    5. Query the relationship between slot IDs and NVMe SSD IDs as follows.
      1. On the Linux terminal, run the cd /sys/bus/pci/slots command to find the corresponding slot IDs. Record the slot IDs that do not start with 0 (for example, 8, 9, 10, and 11 in Figure 8-14).
        Figure 8-14 Slot IDs
      2. Run the following command to obtain the bus IDs of SSDs in the recorded slots. For example, the bus ID of slot 8 in Figure 8-15 is 0000:81:00.

        Command: cat /sys/bus/pci/slots/$slot/address

        Parameter description: $slot indicates the SSD slot number and ranges from 2 to 13.

        Figure 8-15 Bus IDs that correspond to SSD slot IDs
      3. Run the following command to display the NVMe SSD IDs, PCIe bus IDs, and 80 IDs under /sys/class/block/, as shown in Figure 8-16. Work out the relationships between NVMe SSD IDs and slot IDs by referring to the relationships between slot IDs and bus IDs.

        For example, for nvme0n1, the bus ID is 0000:81:00.0 and the 80 ID is 0000:80:02.0. By referring to the mapping relationships obtained in 3.e.ii, the slot ID of nvme0n1 is 8.

        Command: ls -l /sys/class/block/

        Figure 8-16 Relationships between NVMe SSD IDs, PCIe bus IDs, and 80 IDs
      NOTE:

      Theoretically, slot IDs in 3.e.i are the same as the slot numbers (ranging from 2 to 13) on the servers. However, different systems may have different start numbers, so the 80 IDs in 3.e.iii can be used to determine the slot numbers because the dev.func IDs of the 80 IDs correspond to the slot numbers.

    6. Run the following command to safely remove the SSD.

      echo n > /sys/bus/pci/slots/Slot ID/power

      Parameter description:

      • n: The value can be 0 or 1. The value 0 indicates removal, and the value 1 indicates insertion. The value 1 can be used only when a removal command is run but no removal operation is performed.
      • Slot ID: For details, see Table 8-4.

      For example, to hot remove the NVMe SSD in slot 1 in scheduled mode, run the following command:

      echo 0 > /sys/bus/pci/slots/3/power

      3: slot ID of slot 1.

    7. Observe the NVMe SSD indicators. If the green indicator is off and the yellow indicator blinks at 0.5 Hz, the NVMe SSD can be removed.

      Remove the SSD manually. The removal is the same as removing a SAS or SATA HDD/SSD.

    8. Place the removed hard disk into an ESD bag.
    9. (Optional) If you do not install another hard disk into the slot immediately, install a filler module.

Translation
Download
Updated: 2019-07-10

Document ID: EDOC1000095351

Views: 6832

Downloads: 73

Average rating:
This Document Applies to these Products
Related Documents
Related Version
Share
Previous Next