Interconnection Between 10GE Interfaces of S12708 and S5700 Switches Fails

Publication Date:  2015-10-13 Views:  539 Downloads:  0
Issue Description
In the customer's central equipment room in building 3, two S12708 switches set up a cluster using CSS2 technology to act as the core node. After all 100M access switches in building 3 are replaced by S5700-52X-PWR-LI-AC gigabit access switches, the access switches fail to communicate with the S12708 cluster through 10GE interfaces.

Network topology:



Symptom:

1. Interconnection test in the equipment room: An S5700 switch can normally communicate with another S5700 switch (loopback connection) and the S12708 cluster through 10GE interfaces.

2. Interconnection test between the equipment room and a different floor (optical distribution frames used, less than 150 m of distance): An S5700 switch cannot communicate with another S5700 switch or the S12708 cluster through 10GE interfaces.

3. Interconnection test between the equipment room and a different floor (no optical distribution frames used, less than 150 m of distance): Indicators on the interconnected 10GE interfaces between two S5700 switches or between an S5700 switch and the S12708 cluster may stay on for a short period, during which many packets are dropped and the transmission delay is long.
Alarm Information
The interconnected interfaces cannot go Up, or the interfaces go Up but their indicators blink abnormally.
Handling Process
Collect and save device information after obtaining permission from the customer.

Connect the 10GE interfaces using new optical fibers (model: DTT-GJ-12A1, multimode, 62.5/125 um). The following figure shows specifications of the optical fibers.



1. When 10GE interfaces of the S5700 and S12708 switches are connected using 150 m new fibers, the 10GE interfaces should not go Up. Theoretically, fibers of this model can transmit data at 10 Gbit/s only within a distance of 33 m. In the interconnection test, however, the interfaces connected using 150 m fibers of this model go Up. This test result is abnormal.

2. When 10GE interfaces of the S5700 and S12708 switches are connected using 20 m new fibers, the 10GE interfaces can go Up and can be pinged successfully. The test result is normal because the distance between the 10GE interfaces is within the maximum transmission distance (33 m) supported by the fibers.

3. When 10GE interfaces of two S5700 switches are connected using 150 m new fibers, the 10GE interfaces cannot go Up. The test result is normal because the distance between the 10GE interfaces exceeds the maximum transmission distance supported by the fibers.

4. When 10GE interfaces of two S5700 switches are connected using 20 m new fibers, the 10GE interfaces can go Up. The test result is normal because the distance between the 10GE interfaces is within the maximum transmission distance supported by the fibers.

5. When GE interfaces of the S5700 and S12708 switches are connected using 150 m new fibers, the GE interfaces can go Up. Theoretically, fibers of this model can transmit data at 1 Gbit/s within a distance of 275 m. The actual distance between the GE interfaces is within this range, so the test result is normal.

6. When GE interfaces of the S5700 and S12708 switches are connected using 20 m new fibers, the GE interfaces can go Up. Theoretically, fibers of this model can transmit data at 1 Gbit/s within a distance of 275 m. The actual distance between the GE interfaces is within this range, so the test result is normal.

7. When GE interfaces of two S5700 switches are connected using 150 m new fibers, the GE interfaces can go Up. Theoretically, fibers of this model can transmit data at 1 Gbit/s within a distance of 275 m. The actual distance between the GE interfaces is within this range, so the test result is normal.

8. When GE interfaces of two S5700 switches are connected using 20 m new fibers, the GE interfaces can go Up. Theoretically, fibers of this model can transmit data at 1 Gbit/s within a distance of 275 m. The actual distance between the GE interfaces is within this range, so the test result is normal.

Use original optical fibers (of the same model as the new optical fibers) used in the equipment room to perform the tests:

9. When 10GE interfaces of the S5700 and S12708 switches are connected using 150 m original fibers, the 10GE interfaces cannot go Up. The test result is normal because the distance between the 10GE interfaces exceeds the maximum transmission distance supported by the fibers.

10. When GE interfaces of the S5700 and S12708 switches are connected using 150 m original fibers, the GE interfaces cannot go Up. Theoretically, fibers of this model can transmit data at 1 Gbit/s within a distance of 275 m. The actual distance between the GE interfaces is within this range, so the test result is abnormal.

11. When 10GE interfaces of two S5700 switches are connected using 150 m original fibers, the 10GE interfaces cannot go Up. The test result is normal because the distance between the 10GE interfaces exceeds the maximum transmission distance supported by the fibers.

12. When GE interfaces of two S5700 switches are connected using 150 m original fibers, the GE interfaces cannot go Up. Theoretically, fibers of this model can transmit data at 1 Gbit/s within a distance of 275 m. The actual distance between the GE interfaces is within this range, so the test result is abnormal.
Root Cause
1. In test scenario 10, interconnected GE interfaces between S5700 and S12708 switches cannot go Up. The receive power (Current Rx Power(dBm) field) on GigabitEthernet2/2/0/36 of the S12708 cluster is -26.58, which is below the normal range considering the receiver sensitivity of the optical transceiver on the interface. Although the fiber length (150 m) is shorter than the maximum transmission distance, the optical fiber may attenuate to a low value due to other factors on the link.

2. In test scenario 12, interconnected GE interfaces between S5700 switches cannot go Up. The receive power on XGigabitEthernet0/0/3 is -26.02, below the normal range. The optical power may attenuate due to other factors on the fiber link.

3. It has confirmed that optical transceivers used on the interfaces have been certified by Huawei and no quality issues were found on the optical transceivers. The model of optical transceivers on the switches is: OMXD30000 optical transceiver, SFP+, 10G, multi-mode module (850 nm, 0.3 km, LC). If the required 10 Gbit/s transmission distance exceeds 100 m, OM3 (50/125 um) or OM4 optical fibers must be used. The current optical fibers used in the equipment room do not meet the requirement.

4. Datang DTT-GJ-12A1 optical fibers (12-core, multimode, 62.5/125 um) are used in the equipment room, which are not OM3 or OM4 optical fibers.

According to the preceding test results, the current optical fibers cannot meet transmission requirements in the equipment room.
Solution
Replace the optical fibers with OM3 optical fibers.
Suggestions
1. If you cannot determine whether an interface-Down issue is caused by optical transceivers or fibers, test the interface connections in different scenarios to locate the problem.

2. When interconnected interfaces cannot go Up, check whether the parameters of the optical fibers match the optical transceivers or whether the optical fibers and optical transceivers are qualified.

3. You also need to check whether the transmit power and receive optical power on the interconnected interfaces are within the normal ranges or whether the power values are close to the receiver sensitivity.

4. Ensure consistent configurations on the interconnected interfaces.

5. Ensure consistent working modes (duplex mode and rate) on the interconnected interfaces.

END