Troubleshooting Interface Down Problems

Table 1-2 describes the causes and recovery measures for interface down events.

If an interface is down and no prompt message is displayed, check whether the prerequisites for bringing the interface up are met. Whether an interface is enabled is determined by a combination of multiple actions, including the [ undo ] shutdown command. An interface is enabled only when all the conditions are met. To check diagnostic information about whether interfaces are enabled, run the display system internal device port command.

# Check diagnostic information about 10GE1/0/1.

<HUAWEI> display system internal device port 10ge 1/0/1
Port create related check:
--------------------------------------------------------------------------------
Item             LogicCfg         PhyCfg           Picm             IsPass
--------------------------------------------------------------------------------
board module     0x3000001f       0x3000001f       N/A              YES
board device     0x14000207       0x14000207       N/A              YES
lfe device       0x80000000       0x80000000       N/A              YES
pic module       0x50000020       0x50000020       N/A              YES
pic device       0x43000006       0x43000006       N/A              YES
panelport 1      0x50000004       0x50000004       N/A              YES
media type 1     --               --               N/A              NO
port device      0x6000002f       0x6000002f       N/A              YES
--------------------------------------------------------------------------------

Port enable related check:
--------------------------------------------------------------------------------
DevType          AttrName         AttrValue        ExpectValue      IsPass
--------------------------------------------------------------------------------
board            isFastUpgrade    0                == 0             YES
PhyLpuBrd        isIssuUpgrade    0                == 0             YES
PhyLfe           Status           0x10001          != 0             YES
port             isAvailable      0x1              == 1             YES
port             isshut           0x1              == 1             YES
port             portlfeisup      0x1              == 1             YES
port             portissuup       0                == 0             YES
port             triggerShut      0x1              == 1             YES
port             port12x100gDown  0                == 0             YES
port             phyportisshut    0x1              == 1             YES
--------------------------------------------------------------------------------

Port physical related check:
--------------------------------------------------------------------------------
Link    Enable    Speed    Negotiation     Loopback
--------------------------------------------------------------------------------
DOWN    DISABLE   100000   DISABLE         PHY
--------------------------------------------------------------------------------

In the preceding command output, if any value of the IsPass field in Port enable related check is not YES, the interface does not go up.

For details about the possible causes and solutions of the failed check items, see Troubleshooting Guide or collect logs and contact Huawei technical support.

If the interface is enabled but remains in down state, check the transmission medium status of the interface to determine whether the fault is caused by hardware or environment factors.

• For an Ethernet optical interface
  1. Check whether the optical module is properly installed.

     Run the display interface transceiver command. If no command output is displayed or the command output is abnormal, the optical module is not properly installed. In this case, remove and reinstall the module.

     You can run the display device slot <slot-id> command to check whether optical modules on a device are properly installed in a batch.

  2. Check whether the optical module matches.

     Run the display interface interface-type interface-number transceiver verbose command to check optical module information.

     <HUAWEI> display interface 10ge 1/0/1 transceiver verbose  
       
       10GE1/0/1 transceiver information: 
     -------------------------------------------------------------------
       Common information:  
       Transceiver Type                      :1000BASE_SX   //Indicates the optical module type.
       Connector Type                        :LC            //Indicates the type of the optical fiber connector.
       Wavelength (nm)                       :850           //Indicates the optical wavelength.
       Transfer Distance (m)                 :150(62.5um/125um OM1)    
                                              300(50um/125um OM2)    
                                              400(50um/125um OM4)  
       Digital Diagnostic Monitoring         :YES   
       Vendor Name                           :SumitomoElectric   
       Vendor Part Number                    :SCP6F86-GL-CWH  
       Ordering Name                         : 
     ------------------------------------------------------------------- 
      Manufacture information:  
       Manu. Serial Number                   :7YK056C08623  
       Manufacturing Date                    :2007-11-13  
       Vendor Name                           :SumitomoElectric 
     ------------------------------------------------------------------- 
      Alarm information:  
        Non-Huawei-Ethernet-Switch-Certified Transceiver  
        LOS alarm 
     -------------------------------------------------------------------

     • View the Transceiver Type field to check whether the local optical module type matches the remote optical module type. For example, if a GE optical module is installed on the remote interface and a 10GE optical module is installed on the local interface, the interfaces will not go up. You can replace the optical module on the local or remote interface based on actual requirements to ensure that local and remote optical modules have the same rate.

       View the Manu. Serial Number field to check whether the serial number of the copper cable connected to the local interface is the same as that connected to the remote interface. If the serial numbers are inconsistent, you can replace the copper cable connected to the local or remote interface based on actual requirements to ensure that the serial numbers of the copper cables at both ends are consistent.

     • View the Transfer Distance field to check the transmission distance of the optical module, and determine whether the optical fiber length is within the allowed transmission distance range of the optical module based on the optical fiber type. For example, the transmission distance supported by OM1 optical fibers in the preceding command output is 150 m. If the actual transmission distance exceeds 150 m, use an optical fiber with a longer transmission distance.

     • View the Alarm information field to check whether the optical module is a non-Huawei-certified optical module. If the alarm message "Non-Huawei-Certified Transceiver" or "Non-Huawei-Ethernet-Switch-Certified Transceiver" is displayed, the optical module is a non-Huawei-certified optical module. Replace it with a Huawei-certified optical module.

     • If Alarm information displays LOS Alarm, the local optical module does not receive optical signals. The remote interface, optical fiber, or optical module may be abnormal. Run the display this command in the interface view to check whether the interfaces at both ends are shut down. If so, run the undo shutdown command. If not, check whether the optical modules and optical fibers at both ends are normal.

  3. Check whether the receive optical power and transmit optical power are normal.

     Run the display interface interface-type interface-number transceiver verbose command to check whether the receive optical power and transmit optical power are within the normal range.

     <HUAWEI> display interface 10ge 1/0/1 transceiver verbose 
     
     ...... 
     ------------------------------------------------------------------- 
      Diagnostic information:  
        Temperature (Celsius)                 :33.68    
        Voltage (V)                           :3.29   
        Bias Current (mA)                     :7.97  
        Bias High Threshold (mA)              :13.20  
        Bias Low Threshold (mA)               :4.00  
        Current RX Power (dBm)                :-2.15 
        Default RX Power High Threshold (dBm) :1.00  
        Default RX Power Low Threshold (dBm)  :-11.90 
        Current TX Power (dBm)                :-2.07  
        Default TX Power High Threshold (dBm) :1.00  
        Default TX Power Low Threshold (dBm)  :-9.30 
     -------------------------------------------------------------------

     • If the receive optical power is low (Current RX Power has a smaller value than Default RX Power Low Threshold), the transmit signal strength of the remote optical module is poor. As a result, the local interface may go down or discard packets after it goes up. Check whether the distance between the local and remote devices exceeds the maximum transmission distance of the remote optical module. If not, check whether the optical module and optical fiber on the remote interface are damaged. If they are damaged, replace them.

     • If the receive optical power is high (Current RX Power has a larger value than Default RX Power High Threshold), the transmit signal strength of the remote optical module is too high. The possible reason is that the distance between the local and remote devices is short but a long-distance optical module is used. In this case, install an optical attenuator on the remote optical module to reduce the transmit power.

     • If the transmit optical power is low (Current TX Power has a smaller value than Default TX Power Low Threshold), the transmit signal strength of the local optical module is poor or the local optical module is faulty. As a result, the local interface may go down or discard packets after it goes up. Replace the local optical module or contact Huawei technical support.

     • If the transmit optical power is high (Current TX Power has a larger value than Default TX Power High Threshold), the transmit signal strength of the local optical module is too high. This may cause a continuous high receive power on the remote optical module and the remote optical module may be burnt. The possible cause is that the local optical module is faulty. You are advised to replace the local optical module.

  4. Check whether the optical module matches the optical fiber.
     • Check the optical module type.

       Generally, an optical module has a label attached, identifying the rate, center wavelength, and mode (single-mode or multimode) of the optical module, as shown in Figure 1.

       Figure 1-1 400GE optical module
       
       You can obtain more information about optical modules in the following ways:

       • See What Is an Optical Module and Its FAQs.

       • Install an optical module on an interface of a matching type, and then run the display interface transceiver command to check optical module information.

     • Check whether the optical module matches the optical fiber.

       • A single-mode optical module (typically with a center wavelength of 1310 nm or 1550 nm) must be used with single-mode optical fibers (typically yellow).

       • A multimode optical module (typically with a center wavelength of 850 nm) must be used with multimode optical fibers (typically orange).

     • DWDM optical module: If a DWDM optical module is used, run the wavelength-channel channel-number command to manually configure the wave channel number corresponding to the center wavelength of the DWDM optical module and check whether the wavelengths of the optical modules on the interfaces at both ends are the same.

       The center wavelength of a DWDM optical module can be configured only on a 10GE or 25GE interface that has a 10GE DWDM optical module installed or has a 10GE DWDM optical module pre-configured.

  5. Perform an external loopback test on the optical module.

     Use an optical fiber to connect the Tx and Rx ends of the optical module to perform an external loopback test on the optical module (use an optical attenuator for a long-distance optical module). If the interface indicator is steady on and the interface can go up, the local interface and optical module are normal. Otherwise, the local interface and optical module may be abnormal. You are advised to use a normal optical module or interface for test.

     Avoid loops when performing the external loopback test. The optical module will work abnormally if the receive optical power is too high. Therefore, avoid high receive optical power on the optical module. In most cases, a short-distance optical module and a multimode optical fiber are used in an external loopback test. In addition, run a command to check the receive optical power and ensure that the receive optical power is lower than the upper threshold.

  6. Perform a replacement test.

     Perform a replacement test for the optical module and optical fiber link, and locate the failure point based on the test result. You can replace the local interface, local optical module, optical fiber link (including the optical fiber, optical distribution frame, fiber splice point, optical splitter, and intermediate wavelength division multiplexing (WDM) transmission device), remote optical module, and remote interface.

     • If an intermediate WDM transmission device exists, bypass or replace the intermediate device. If the local and remote interfaces go up normally, the intermediate device is faulty.

     • Replace the optical fiber and optical distribution frame. If the local and remote interfaces go up normally, the optical fiber link is faulty.

     • If the local and remote interfaces go up normally after the local optical module is replaced, the local optical module is faulty.

     • If the local and remote interfaces go up normally after the local interface is replaced, the local interface is faulty.

     • If the local and remote interfaces go up normally after the remote optical module is replaced, the remote optical module is faulty.

     • If the local and remote interfaces go up normally after the remote interface is replaced, the remote interface is faulty.

     Replace the optical module and optical fiber preferentially. If a WDM or other transmission device exists on the optical fiber link, remove the intermediate device to directly connect the local and remote devices or replace the intermediate device. Then replace the interfaces and devices to check whether the fault occurs on an interface or a device.

• For an Ethernet electrical interface
  1. Check the cable status. Run the virtual-cable-test command in the interface view to check the cable status. If the cable status is displayed as Open or Short rather than OK, replace the cable.
  2. Check whether the local interface is connected to the correct remote interface.

If the transmission medium status of the interface is normal, check whether the fault is caused by the interface configuration. The following factors will affect the state of an interface:

1. Negotiation status

   The negotiation status of two connected interfaces must be the same. Otherwise, the interfaces may go down. In this case, ensure that the negotiation status of the two interfaces is the same.

   Run the display interface interface-type interface-number command to check whether the negotiation status of two connected interfaces on both ends is the same.

   <HUWEI> display interface 10ge 1/0/1 
   10GE1/0/1 current state : DOWN (ifindex: 52) 
   Line protocol current state : DOWN  
   Description:  
   Switch Port, PVID :    1, TPID : 8100(Hex), The Maximum Frame Length is 9216 
   Internet protocol processing : disabled 
   IP Sending Frames' Format is PKTFMT_ETHNT_2, Hardware address is e468-a357-cbc1 
   Port Mode:    COMMON COPPER,     Port Split/Aggregate:         DISABLE 
   Speed:                 1000,     Loopback:                        NONE 
   Duplex:                FULL,     Negotiation:                   ENABLE 
   Input Flow-control: DISABLE,     Output Flow-control:          DISABLE 
   Mdi:                   AUTO,     Fec:                             NONE 
   Last physical up time   : - 
   ...

   • After an SFP-GE copper or optical module is installed on a GE or 10GE optical interface, or a high-speed cable is connected to a 40GE optical interface, the interface works in auto-negotiation mode by default and can be configured to work in non-auto-negotiation mode using the negotiation disable command. If you replace the module after the non-auto-negotiation mode takes effect and the new module does not support the negotiation disable command configuration, the interface goes down. For example, if an SFP-FE optical module is installed on a GE optical interface, an SFP+ optical module is installed on a 10GE optical interface, or a QSFP+ optical module is installed on a 40GE optical interface, the interface goes down. You can run the undo negotiation disable command in the interface view to delete the original non-auto-negotiation mode configuration to make the interface go up again.

   • A 40GE optical interfaces work in auto-negotiation mode by default when it has a high-speed cable connected. If the interface on the peer device does not support auto-negotiation, run the negotiation disable command to configure the local 40GE interface to work in non-auto-negotiation mode.

   • When a 40GE interface on the CE-L36LQ-EG card is connected to an interface on some types of network adapters (such as Mellanox network adapters) using a passive QSFP+ high-speed cable, the two interfaces may fail to go up due to different 40GBASE-CR4 negotiation mechanisms used on the interfaces. In this case, disable the auto-negotiation function on the 40GE interface so that the two interfaces can go up and communicate properly.

   # Disable auto-negotiation on 10GE1/0/1.

   <HUAWEI> system-view 
   [~HUAWEI] interface 10ge 1/0/1 
   [~HUAWEI-10GE1/0/1] negotiation disable 
   [*HUAWEI-10GE1/0/1] commit

   CloudEngine series switches provide the following methods to check the default auto-negotiation status, interface rate, and whether attributes such as auto-negotiation and flow control auto-negotiation can be configured, based on the interface type and transmission medium:

   • Use the Interface Query Tool (https://info.support.huawei.com/network/ptmngsys/Web/DC/en/interface_query.html).
   • See "Which Ethernet Interfaces of CE Switches Support Auto-Negotiation?" in FAQs - Interface Management - Ethernet Interface in Troubleshooting Guide.
2. Interface rate

   If the rates of two connected interfaces are different, negotiation between the two interfaces fails. In this case, set the interface rate to ensure that the rates of the two interfaces are the same. If the rate displays auto, the lower-layer link is not established. Check other possible causes or connect the interface to other interfaces to locate the fault.

   CloudEngine series switches provide the following methods to check the default auto-negotiation status, interface rate, and whether attributes such as auto-negotiation and flow control auto-negotiation can be configured, based on the interface type and transmission medium:

   • Use the Interface Query Tool (https://info.support.huawei.com/network/ptmngsys/Web/DC/en/interface_query.html).
   • See "Which Ethernet Interfaces of CE Switches Support Auto-Negotiation?" in FAQs - Interface Management - Ethernet Interface in Troubleshooting Guide.
3. FEC status

   If the FEC status of two connected interfaces is different, the interfaces cannot go up. To check the FEC status of interfaces, run the display interface command. If the FEC status of the two interfaces is different, disable the auto-negotiation function and run the fec mode command to configure the FEC mode.

4. Duplex mode

   An Ethernet interface works in either half-duplex or full-duplex mode. Interfaces on both ends must use the same duplex mode. Otherwise, the interfaces cannot go up.

   Only 10GE electrical interfaces on the CE6850-48T4Q-EI in V100R003C10 or later versions support the duplex mode configuration when the interface rate is 100 Mbit/s. All Ethernet interfaces on other CloudEngine series switches work in full-duplex mode and do not support the half-duplex mode.

5. Interface split configuration

   If interface split is not correctly configured on the interface or an inappropriate cable is used after interface split, the interface may go down. When a 40GE interface that is not split is connected to four remote 10GE interfaces over a one-to-four cable, the 40GE interface cannot go up and its indicator is off, whereas the remote 10GE interfaces may be up and their indicators are on but they cannot work properly. You need to correctly configure interface split or replace the cable used after interface split with an appropriate one.

   For details about whether interfaces on a device support interface split, interface split types, and cables used after interface split, see Interface Split Query Tool.

6. Fault signal detection and filtering

   The fault signal filtering function prevents frequent transitions between up and down due to reasons such as link signal jitter, improving link stability. After the fault signal filtering function is enabled on an interface, the interface may go up and the remote interface may go down for a short period of time. If two connected devices on both ends support the fault signal filtering function, you are advised to enable this function on the two devices. If one end supports the function but the other end does not, you are advised to disable the function.

   For more details about the fault signal filtering function, see "When Two CE Switches Are Connected, Why Does the Interface on One End Go Up and the Interface on the Other End Go Down Temporarily?" in FAQs - Interface Management - Ethernet Interface in Troubleshooting Guide.

7. Loopback detection

   If the loopback detection function is configured on the remote interface, the local interface may go down. Disable the loopback detection function on the remote interface.

   • Before testing some special functions such as locating an Ethernet fault, enable loopback detection on the desired Ethernet interface to check whether the interface is working properly.

   • The internal loopback detection function affects other functions and may prevent the interface or link from working properly. When the test is complete, run the undo loopback command to disable loopback detection. The original configuration is restored after loopback detection is disabled.

   • You are not advised to configure the auto-negotiation, flow control, flow control auto-negotiation, interface rate, FEC, and EEE of electrical interfaces together with the loopback detection function on the interface. Otherwise, the configuration may not take effect.

8. Unidirectional single-fiber communication

   A single fiber means that two optical modules are connected by only one fiber, and unidirectional communication means that packets can be sent in only one direction from the sender to the receiver.

   An optical module provides a TX end and an RX end. The TX and RX ends of one optical module are respectively connected to the RX and TX ends of another module. A device transmits and receives packets through two independent fibers. If the unidirectional single-fiber communication function is disabled, two devices cannot communicate with each other through a single fiber. After the single-fiber enable command is run on interfaces, the devices can communicate with each other unidirectionally.

   After the unidirectional single-fiber communication function is configured on an interface, the interface will be in down state if no optical module is inserted, or if a single-fiber bidirectional optical module or high-speed cable is inserted into the interface. In this case, replace the optical module (use an optical module that is not a single-fiber bidirectional optical module) to rectify the fault. If no optical module is inserted into the interface, run the device transceiver fiber command to pre-configure the transmission medium type of the interface as fiber.

   For details about the types of single-fiber bidirectional optical modules, see "Do the CloudEngine Series Switches Support BIDI Optical Modules?" in FAQs - Interface Management - Optical Module in Troubleshooting Guide. Optical modules except single-fiber bidirectional optical modules support unidirectional single-fiber connections.

9. Energy saving protocols
   • Automatic laser shutdown (ALS): can be configured for optical interfaces to control the pulse of an optical module's laser. If a fiber is not properly installed on an optical interface or an optical fiber link fails, ALS shuts down the laser to save energy and prevent eye damage.
   • Energy Efficient Ethernet (EEE): is supported only on electrical interfaces (except the Ethernet management interface). After EEE is configured on an interface, the system adjusts the power supply of the interface when the interface is idle. The interface then enters the low power mode (sleeping mode), which reduces the total power consumption of the system. When the interface starts to transmit data, its power supply is restored to the normal state.
   • Interface dormancy: can be configured only on GE electrical interfaces (except the Ethernet management interface). After interface dormancy is configured on an interface, the interface is automatically shut down to save power when it is idle.

   If one or more of the preceding energy saving protocols are configured on an interface, check whether the configuration is correct.