Cabinet Layout and Connection Planning

Context

• Equipment dimension

  The dimensions of storage devices vary according to the model. Table 4-2 lists the dimensions of the related storage devices.

  Table 4-2 Device dimensions

  Device	Dimensions
  Controller enclosure	Depth: 865 mm Width: 447 mm Height: 175 mm
  2 U SAS disk enclosure (25 disk slots)	Depth: 410 mm Width: 447 mm Height: 86.1 mm
  4 U SAS disk enclosure (24 disk slots)	Depth: 488 mm Width: 447 mm High: 175 mm
  2 U smart SAS disk enclosure (25 disk slots)	Depth: 520 mm Width: 447 mm Height: 86.1 mm
  2 U smart SAS disk enclosure (12 disk slots)	Depth: 600 mm Width: 447 mm Height: 86.1 mm
  2 U smart NVMe disk enclosure	Depth: 620 mm Width: 447 mm Height: 86.1 mm

• U-shaped brackets

  The distance between the front mounting bar and the inner side of the front door varies according to the cabinet type. Since the system enclosure needs to be fixed on the front mounting bars, a longer distance between the front mounting bars and the inner side of the front door can result in less cabling space at the rear of the device. In such a case, you can install U-shaped brackets to move the system enclosure forward for more cabling space.

  A U-shaped bracket is a mechanical part installed between the mounting ear of the enclosure and mounting bar of the cabinet to adjust the relative position between the enclosure and the cabinet. A maximum of two U-shaped brackets can be stacked.

  U-shaped brackets are classified into three types: 25 mm, 50 mm, and 75 mm, as shown in Figure 4-2.

  Figure 4-2 U-shaped bracket of different specifications
  
  • If U-shaped brackets are not installed, the system enclosure is fixed to the front mounting bar by using captive screws. The distance between the front panel of the system enclosure and the inner side of the front door is fixed.
  • Install the U-shaped brackets on the front mounting bars, and then secure the system enclosure to U-shaped brackets of the respective size. This allows you to adjust the distance between the front panel of the system enclosure and the inner side of the front door within a certain range.

Installation Planning

Before installing storage devices in a cabinet, ensure that:

• The front panel of the system enclosure does not affect closing of the cabinet door.
• The distance between the rear of the system enclosure and the rear door of the cabinet is greater than or equal to 90 mm. Otherwise, the cables at the rear of the system enclosure may affect routing of cables or closing of the cabinet door.

If the distance between the front mounting bars and the inner side of the front door does not meet the installation requirements, use U-shaped brackets to shorten the distance between the front panel of the system enclosure and the inner side of the front door. Figure 4-3 and Figure 4-4 show the positions of the storage system before and after the U-shaped brackets are installed.

• A measuring scale is contained in the auxiliary material package. You can use it to determine the type of U-shaped brackets for your storage device. For details about how to use the measuring scale, see the instructions on it.
• For details on how to install the U-shaped brackets, see (Optional) Installing U-shaped Brackets and Adjustable Guide Rails.
• The U-shaped brackets decrease the distance between the front panel of the system enclosure and the inner side of the front door (L1 in the figures) but increase the distance between the rear of the system enclosure and the rear door of the cabinet (L2 in the figures).

Figure 4-3 Top view of the cabinet (no U-shaped brackets installed)

Figure 4-4 Top view of the cabinet (U-shaped brackets installed)

The preceding cabinet space requirements do not consider the space that may be contained in the front and rear doors of the cabinet.

Checking the Installation Conditions in a Third-Party Cabinet

If the storage device needs to be installed in a third-party cabinet provided by the customer, extra requirements must be met in addition to the preceding requirements. For the check items, refer to Checking a Third-Party Cabinet.

Connection Principles (Applicable to the OceanStor 18000 V5 Series)

Connection Principles (Applicable to the OceanStor 18000F V5 Series)

Before the planning, note the following:

• An expansion loop cannot contain both SAS disk enclosures and smart disk enclosures.
• Connect the expansion interface module in slot H to expansion module A on the disk enclosure and the expansion interface module in slot L to expansion module B on the disk enclosure.
• The disk enclosure type and the number of cascaded disk enclosures supported by a controller enclosure must not exceed the specifications. For details, visit Specifications Query (http://support-it.huawei.com/storagespecification/#/home).

Expansion Ports and Slots

The highlighted slots in the following figures can be used to connect expansion modules. Determine the slots for installing interface modules and ports for connecting the disk enclosures based on the actual number of interface modules and disk enclosures, and the maximum number of disk enclosures supported by the storage system.

• Connecting SAS disk enclosures

  A 12 Gbit/s SAS interface module used to connect a SAS disk enclosure can be installed in a slot between IOM H5/L5 and IOM H8/L8, as shown in Figure 4-5.

  The recommended slot sequence is as follows: IOM H6/L6 > IOM H7/L7 > IOM H5/L5 > IOM H8/L8, where the configuration disk enclosure (the enclosure with coffer disk identifiers) must be installed in slots IOM H6/L6.

  Figure 4-5 Slots for installing interface modules used to connect SAS disk enclosures
  

• Connecting smart disk enclosures

  An RDMA interface module used to connect a smart disk enclosure can be installed in a slot among IOM H2/L2 to IOM H11/L11 (except for IOM H3/L3 and IOM H10/L10), as shown in Figure 4-6.

  The recommended slot sequence is as follows: IOM H6/L6 > IOM H7/L7 > IOM H5/L5 > IOM H8/L8 > IOM H4/L4 > IOM H9/L9 > IOM H2/L2 > IOM H11/L11 > IOM H1/L1 > IOM H12/L12 > IOM H3/L3 > IOM H10/L10, where the configuration disk enclosure (the enclosure with coffer disk identifiers) must be installed in slots IOM H6/L6.

  The SO 100 Gbit/s RDMA interface module used for adding controller enclosures can be installed only in slots IOM H3/L3 and IOM H10/L10. When connecting disk enclosures, do not use these slots unless other slots are not enough for disk enclosure connection.

  Figure 4-6 Slots for installing interface modules used to connect smart disk enclosures
  
• Connecting SAS and smart NVMe disk enclosures

  SAS disk enclosures can be mixed with smart NVMe disk enclosures. A controller enclosure can connect to both SAS and smart NVMe disk enclosures in different loops.

  In the mixed configuration, the slot sequence for installing RDMA interface modules used to connect smart disk enclosures is IOM H4/L4 > IOM H9/L9, and that for installing 12 Gbit/s SAS interface modules used to connect SAS disk enclosures is IOM H6/L6 > IOM H7/L7 > IOM H5/L5 > IOM H8/L8, as shown in Figure 4-7. The configuration disk enclosure (the enclosure with coffer disk identifiers) must be installed in slots IOM H6/L6.

  Figure 4-7 Available slots
  

Connection Planning

To ensure network reliability, connect disk enclosures in the same expansion loop in forward redundancy mode so that two independent links are set up between the disk enclosures for mutual backup. In the expansion loops shown in Figure 4-8, Figure 4-9, and Figure 4-10, the orange and blue lines represent two independent links in mutual backup. A controller failure does not cause all expansion loops to fail.

The following networking diagrams of disk enclosures are for your reference only. You can access Huawei Storage Product Networking Assistant (https://support.huawei.com/onlinetoolsweb/sna/#/home) for more networking diagrams.

• Connecting a 4 U controller enclosure to three 2 U SAS disk enclosures:
  Figure 4-8 Example of connecting a 4 U controller enclosure to three 2 U SAS disk enclosures
  
  • Connecting a 4 U controller enclosure to three 4 U SAS disk enclosures:
    Figure 4-9 Example of connecting a 4 U controller enclosure to three 4 U SAS disk enclosures
    

• Connecting a 4 U controller enclosure to three smart disk enclosures:
  Figure 4-10 Example of connecting a 4 U controller enclosure to three smart disk enclosures
  

After disk enclosures are connected and powered on, do not change their positions in the storage system. Otherwise, IDs of disk enclosures may be displayed incorrectly, service performance may deteriorate, or some storage resources may be unavailable.

Context

• Application servers and storage systems support various network modes. An application server is usually connected to a storage system over multiple paths for enhanced data transfer security and reliability.
  • In planning block services, hosts require multipathing software to select and manage paths between application servers and storage systems. In planning file services, hosts do not require multipathing software.
  • To connect storage systems to application servers, both iSCSI and Fibre Channel networks are supported for block services while only the NAS network is supported for file services. The principles for planning iSCSI and NAS networks between storage systems and application servers are the same.
  • This section uses the block service network planning as an example. For details about how to configure services, see the Basic Storage Service Configuration Guide for Block and Basic Storage Service Configuration Guide for File specific to your product model.

  On a network with multiple links for block services, multipathing software is required to select and manage paths between application servers and storage systems. This section uses UltraPath, Huawei-developed multipathing software, as an example. For details about how to install and configure UltraPath, see the OceanStor UltraPath User Guide. For the operating systems supporting UltraPath, see the Release Notes corresponding to your UltraPath version. If your operating system does not support UltraPath, use the operating system's native multipathing software.

• A 4 U controller enclosure has four quadrants: A, B, C, and D, as shown in Figure 4-19.
  Figure 4-19 Quadrants on a 4 U controller enclosure
  

Planning Rules

To improve system reliability, comply with the following rules for cable connection:

• Figure 4-20 shows a 4 U controller enclosure. From left to right, the slot IDs are from IOM H0 to IOM H13 in the upper half and are from IOM L0 to IOM L13 in the lower half.
  Figure 4-20 Layout of interface module slots in a 4 U controller enclosure
  
  • In Fibre Channel networking:
    • If 12 Gbit/s SAS interface modules are configured, the recommended slot sequence for installing front-end interface modules is as follows: IOM H0/L0 > IOM H13/L13 > IOM H1/L1 > IOM H12/L12 > IOM H2/L2 > IOM H11/L11 > IOM H4/L4 > IOM H9/L9 > IOM H3/L3 > IOM H10/L10.
    • If BE 100 Gbit/s RDMA interface modules are configured, the recommended slot sequence for installing front-end interface modules is as follows: IOM H0/L0 > IOM H13/L13 > IOM H1/L1 > IOM H12/L12 > IOM H2/L2 > IOM H11/L11 > IOM H4/L4 > IOM H9/L9 > IOM H5/L5 > IOM H8/L8 > IOM H7/L7 > IOM H3/L3 > IOM H10/L10 > IOM H7/L7.
  • In Ethernet networking:
    • If 12 Gbit/s SAS interface modules are configured:
    1. When the storage system is equipped with two controllers, the recommended slot sequence for installing front-end interface modules is as follows: IOM H0/L0 > IOM H1/L1 > IOM H2/L2 > IOM H4/L4 > IOM H3/L3.
    2. When the storage system is equipped with four controllers, the recommended slot sequence for installing front-end interface modules is as follows: IOM H0/L0 > IOM H13/L13 > IOM H1/L1 > IOM H12/L12 > IOM H2/L2 > IOM H11/L11 > IOM H4/L4 > IOM H9/L9 > IOM H3/L3 > IOM H10/L10.
    • If BE 100 Gbit/s RDMA interface modules are configured:
    1. When the storage system is equipped with two controllers, the recommended slot sequence for installing front-end interface modules is as follows: IOM H0/L0 > IOM H1/L1 > IOM H2/L2 > IOM H4/L4 > IOM H5/L5 > IOM H3/L3.
    2. When the storage system is equipped with four controllers, the recommended slot sequence for installing front-end interface modules is as follows: IOM H0/L0 > IOM H13/L13 > IOM H1/L1 > IOM H12/L12 > IOM H2/L2 > IOM H11/L11 > IOM H4/L4 > IOM H9/L9 > IOM H5/L5 > IOM H8/L8 > IOM H3/L3 > IOM H10/L10 > IOM H7/L7.
• Interface modules of the same type must be inserted in sequence.
• Insert the interface modules used for Ethernet networking in ascending order of their port rates. Then insert the interface modules used for Fibre Channel networking in ascending order of their port rates.
• The front-end port connections are symmetric between slots H0 to H13 and slots L0 to L13 on the same storage system. That is, the interface modules reside in slots with the same slot ID and use the ports in the same positions.
• In the Ethernet port bonding scenario, the member bond ports are symmetric between slots H0 to H13 and slots L0 to L13 on the same storage system. That is, the interface modules reside in slots with the same slot ID and use the ports in the same positions.

For the GE electrical interface module:

• If the rate of a GE electrical port is set to 1000 Mbit/s, the GE electrical port supports only the full-duplex mode.
• If GE electrical ports on a GE electrical interface module or network ports on an application server (or a switch) are not in autonegotiation mode, you must use crossover cables to connect the controller enclosure to the application server (or switch). Otherwise, the controller enclosure may fail to communicate with the application server (or switch).
• A GE electrical port uses a CAT5 network cable or CAT6A shielded network cable.

Connection Plans

A storage system can be connected to an application server directly or through two switches.

Other connection modes between storage systems and application servers include single-switch connection, dual-switch connection in a cluster environment, and dual-switch connection in a HyperMetro cluster environment. For details, see "Typical UltraPath Applications" in the OceanStor UltraPath User Guide.

• Direct connection (single controller enclosure)
  • iSCSI network
    1. When the controller enclosure has two controllers, the application server is directly connected to the storage system through two paths. UltraPath automatically calculates and selects the optimal path for data transmission based on the operating status of the storage system. Figure 4-21, Figure 4-22, and Figure 4-23 show the connection diagrams.

       If a controller enclosure is configured with two controllers, interface modules can be installed only in slots in quadrants A and B.

       Figure 4-21 Direct connection diagram (two controllers)
       
       Figure 4-22 Directly connecting the storage device (two controllers) and the application server over GE electrical ports
       
       Figure 4-23 Directly connecting the storage device (two controllers) and the application server over ETH ports
       
    2. When the controller enclosure has four controllers, the application server is directly connected to the storage system through four paths. UltraPath automatically calculates and selects the optimal path for data transmission based on the operating status of the storage system. Figure 4-24, Figure 4-25, and Figure 4-26 show the connection diagrams.
       Figure 4-24 Direct connection diagram (four controllers)
       
       Figure 4-25 Directly connecting the storage device (four controllers) and the application server over GE electrical ports
       
       Figure 4-26 Directly connecting the storage device (four controllers) and the application server over ETH ports
       
  • Fibre Channel network

    It is recommended that you connect the application server to the storage system through four paths, no matter whether the controller enclosure has two or four controllers. UltraPath automatically calculates and selects the optimal path for data transmission based on the operating status of the storage system. Figure 4-27 and Figure 4-28 show the connection diagrams.

    Alternatively, you can choose to connect the storage system and the application server through two paths. You only need to configure two interface modules in the same slots of the H and L planes, respectively.

    Figure 4-27 Direct connection diagram
    
    Figure 4-28 Directly connecting the storage device and the application server over Fibre Channel ports
    
• Dual-switch connection (single controller enclosure)

  In this mode, two switches are added to the direct connections. Each controller is connected to each switch.

  The switches increase the number of ports to allow more access paths. Moreover, the switches extend the transmission distance by connecting remote application servers to the storage system. The two switches are redundant of each other to prevent single point of failure and improve the forwarding capability.

  When setting up connections through switches, note the following:

  • If Ethernet switches are used, plan the network segment of the switches in advance and assign ports to each network segment.
  • If Fibre Channel switches are used, plan switch zones in advance and assign ports to each zone.
  • The first and last ports on a switch are typically used to connect to other switches.
  • iSCSI network
    1. When the controller enclosure has two controllers, connect the storage system to each switch through two paths, as shown in Figure 4-29, Figure 4-30, and Figure 4-31.

       If a controller enclosure is configured with two controllers, interface modules can be installed only in slots in quadrants A and B.

       Figure 4-29 Dual-switch connection diagram (two controllers)
       
       Figure 4-30 Connecting the storage device (two controllers) and the application server over GE electrical ports through switches
       
       Figure 4-31 Connecting the storage device (two controllers) and the application server over ETH ports through switches
       
    2. When the controller enclosure has four controllers, connect the storage system to each switch through four paths, as shown in Figure 4-32, Figure 4-33, and Figure 4-34.
       Figure 4-32 Dual-switch connection diagram (four controllers)
       
       Figure 4-33 Connecting the storage device (four controllers) and the application server over GE electrical ports through switches
       
       Figure 4-34 Connecting the storage device (four controllers) and the application server over ETH ports through switches
       
  • Fibre Channel network

    It is recommended that you connect the storage system to each switch through four paths, no matter whether the controller enclosure has two or four controllers. Figure 4-35 and Figure 4-36 show the connection diagrams.

    Alternatively, you can choose to connect the storage system and each switch through two paths. You only need to configure two interface modules in the same slots of the H and L planes, respectively.

    Figure 4-35 Dual-switch connection diagram
    
    Figure 4-36 Connecting the storage device and the application server over Fibre Channel ports through switches
    

    Table 4-6 provides an example of zone planning for a Fibre Channel network.

    Table 4-6 Zone configuration example

    Zone Name	Application Server	Storage System	Switch
    Zone 1	HBA 0, P0	Quadrant A, P0	Switch 1
    Zone 2	HBA 0, P0	Quadrant B, P0	Switch 1
    Zone 3	HBA 0, P0	Quadrant C, P0	Switch 1
    Zone 4	HBA 0, P0	Quadrant D, P0	Switch 1
    Zone 5	HBA 1, P0	Quadrant A, P1	Switch 2
    Zone 6	HBA 1, P0	Quadrant B, P1	Switch 2
    Zone 7	HBA 1, P0	Quadrant C, P1	Switch 2
    Zone 8	HBA 1, P0	Quadrant D, P1	Switch 2

• Dual-switch connection (multiple controller enclosures)

  When the storage system has two controller enclosures, ensure that front-end interface modules on every enclosure are all connected to the application server. The connection between each controller enclosure and the application server is the same as that when a single controller enclosure is deployed, as shown in Figure 4-37, Figure 4-38, and Figure 4-39.

  Figure 4-37 Dual-switch connection diagram (two controllers on an iSCSI network)
  
  Figure 4-38 Dual-switch connection diagram (four controllers on an iSCSI network)
  
  Figure 4-39 Dual-switch connection diagram (Fibre Channel network)
  

  On Fibre Channel networks, you can choose to connect the storage system and each switch through two paths. You only need to configure two interface modules in the same slots of the H and L planes on each controller enclosure, respectively.

  Table 4-7 provides an example of zone planning for a Fibre Channel network.

  Table 4-7 Zone configuration example

  Zone Name	Application Server	Storage System	Switch
  Zone 1	HBA 0, P0	Controller enclosure 0, quadrant A, P0	Switch 1
  Zone 2	HBA 0, P0	Controller enclosure 0, quadrant B, P0	Switch 1
  Zone 3	HBA 0, P0	Controller enclosure 0, quadrant C, P0	Switch 1
  Zone 4	HBA 0, P0	Controller enclosure 0, quadrant D, P0	Switch 1
  Zone 5	HBA 1, P0	Controller enclosure 0, quadrant A, P1	Switch 2
  Zone 6	HBA 1, P0	Controller enclosure 0, quadrant B, P1	Switch 2
  Zone 7	HBA 1, P0	Controller enclosure 0, quadrant C, P1	Switch 2
  Zone 8	HBA 1, P0	Controller enclosure 0, quadrant D, P1	Switch 2
  Zone 9	HBA 0, P0	Controller enclosure 1, quadrant A, P0	Switch 1
  Zone 10	HBA 0, P0	Controller enclosure 1, quadrant B, P0	Switch 1
  Zone 11	HBA 0, P0	Controller enclosure 1, quadrant C, P0	Switch 1
  Zone 12	HBA 0, P0	Controller enclosure 1, quadrant D, P0	Switch 1
  Zone 13	HBA 1, P0	Controller enclosure 1, quadrant A, P1	Switch 2
  Zone 14	HBA 1, P0	Controller enclosure 1, quadrant B, P1	Switch 2
  Zone 15	HBA 1, P0	Controller enclosure 1, quadrant C, P1	Switch 2
  Zone 16	HBA 1, P0	Controller enclosure 1, quadrant D, P1	Switch 2

Follow-up Procedure

After planning ports, you must formulate port correlation tables for connections between the application server and switch ports, and for connections between the storage system and switch ports. Table 4-8 shows port correlation between the application server and switches, and Table 4-9 shows port correlation between the storage system and switches.