No relevant resource is found in the selected language.

This site uses cookies. By continuing to browse the site you are agreeing to our use of cookies. Read our privacy policy>Search

Reminder

To have a better experience, please upgrade your IE browser.

upgrade

ME60 Troubleshooting Guide V1.0 (VRPv8)

This document provides the maintenance guide of the device, including daily maintenance, emergence maintenance, and typical troubleshooting.
Rate and give feedback :
Huawei uses machine translation combined with human proofreading to translate this document to different languages in order to help you better understand the content of this document. Note: Even the most advanced machine translation cannot match the quality of professional translators. Huawei shall not bear any responsibility for translation accuracy and it is recommended that you refer to the English document (a link for which has been provided).
Load Balancing Faults Troubleshooting

Load Balancing Faults Troubleshooting

This section provides troubleshooting procedures for a fault in load balancing.

Common Causes

This fault is commonly caused by one of the following:

  • The protocol status of an outbound interface is not Up.

  • The number of outbound interfaces exceeds the specified threshold.

  • Routes were not been delivered to interfaces correctly.

  • Didn't choose suitable hash factors.

Troubleshooting Flowchart

Figure 4-44 Troubleshooting flowchart for fault in load balancing

Troubleshooting Procedure

Procedure

  1. Check whether the protocol status of all outbound interfaces is Up and whether the number of outbound interfaces for load balancing exceeds the specified threshold.

    Run the display ip interface brief command in the system view and check the physical and protocol status of the outbound interfaces. If the physical and protocol status of the outbound interfaces is not Up, check whether the IP addresses of the outbound interfaces are correct, whether the physical links are Up, and whether the basic protocols are configured correctly.

    If all the preceding conditions are met, check whether the number of outbound interfaces configured for load balancing has exceeded the specified threshold. If the number of configured outbound interfaces in a group for load balancing reaches the specified threshold and a new outbound interface is added to the group, the path that traverses an interface in the group will be prohibited from performing load balancing. To enable the interface to perform load balancing, choose an alternative interface already in the group, delete the equal-cost route that passes through the replaced interface, and configure an equal-cost route on the interface that is to participate in load balancing.=

    If the fault persists, go to step 2.

  2. Check whether direct routes are delivered to the outbound interfaces.

    Run the display ip routing-table command in the system view and check whether a correct route is delivered to each outbound interface. If no corresponding route is delivered to an outbound interface, check whether the ME60 has learned or advertised a correct route. For routing troubleshooting, see the Troubleshooting - IP Routing.

    If the fault persists, go to step 3.

  3. Check whether the configured hash factors vary evenly. Uneven variation of hash factors causes load imbalance. If this case occurs, ensure that hash factors vary evenly.

    Run the load-balance hash-fields command in the slot view to configure a new hash factor on the current interface.

    Select one of the following commands to configure a hash factor as required:

    • IP forwarding: load-balance hash-fields ip { l3 | l4 }

    • VPLS and MAC forwarding: load-balance hash-fields mac { l2 | l3 | l4 }

    • MPLS forwarding: load-balance hash-fields mpls { payload-header | label | ip | ip-tos | mac }

    If the fault persists, go to step 4.

  4. Collect trap, log, and configuration information, and contact Huawei technical support personnel.

Relevant Alarms and Logs

None.

Troubleshooting Cases

No OSPF Equal-Cost Routes Are Available Between the Two Links That Connect an ABR to a Node in an NSSA

Fault Symptom
Figure 4-45 OSPF NSSA

On the network shown in Figure 4-45, area 10 is the NSSA. The ABR that belongs to both area 0 and area 10 is connected to the RTA in the NSSA. After the nssa default-route-advertise command is run on the ABR, there is only one static route to the RTA, with 10.0.0.17 as the next hop.

The configurations on the ABR are as follows:

 area 0.0.0.10
  network 10.0.0.16 0.0.0.3
  network 10.0.0.192 0.0.0.3
  nssa default-route-advertise
Cause Analysis

After the nssa default-route-advertise command is run on a Huawei ME60, the device generates a Type 7 LSA and adds an FA to the LSA.

The address of the local loopback interface through which default routes are advertised is preferentially selected as the FA. If no such loopback interface is available, the smallest address of the physical interfaces through which default routes are advertised is used as the FA. In this example, the LSA is advertised to only two areas connected to the RTA, and the FA is 10.0.0.17.

When calculating routes, the RTA selects 10.0.0.17 as the next hop of the default route because the FA of the LSA is the address of an interface connected to the RTA.

Troubleshooting
  1. Configure a loopback interface on the ABR and assign an IP address for the loopback interface.

  2. Advertise the loopback interface IP address to area 10.

After the preceding operations are performed, the fault is rectified.

Solution
Choose either of the following solutions:
  • Configure a loopback interface on the ABR, assign an IP address for the loopback interface, and advertise the IP address to area 10.
  • Delete the nssa default-route-advertise command configuration from the ABR and run the nssa no-summary command so that the LSAs (Type 3 LSAs) to be advertised do not carry an FA. If more than one ABR exists in the area, run the nssa no-summary command on each ABR so that equal-cost routes to each ABR are available.
Follow-up Procedure

Run the save command to save the current configuration to the configuration file when a set of configuration is finished and the expected functions have been achieved.

Summary

Configure a loopback address on the ABR in the NSSA so that the loopback address is selected as the FA. If equal-cost routes to the ABR are available, they can load-balance traffic.

IS-IS Routes Fail to Load-balance Traffic

Fault Symptom
Figure 4-46 Networking with IS-IS route load balancing

On the network shown in Figure 4-46, Core-1 and Core-2 in the core layer are Huawei ME60. SR-1 (Huawei device) is attached to Core-1, and SR-2 (non-Huawei device) is attached to Core-2. All the devices are located in an IS-IS Level-2 area. The same service segment is configured on both SRs, and the SRs add static routes to the IS-IS routing table. However, Core-1 and Core-2 can receive routes from SR-2 only.

The configurations of SR-1 are as follows:

isis 1
 is-level level-2
 network-entity 10.0222.0422.5358.00
 is-name SR-1
 import-route direct
 import-route static
Cause Analysis

The cost-type configured on SR-2 when importing routes is different from that configured on SR-1.

Troubleshooting

Change cost-type to internal on SR-1 and check the routing tables on both Core-1 and Core-2. Each routing table has two IS-IS routes to the same service segment.

isis 1
 is-level level-2
 network-entity 10.0222.0422.5358.00
 is-name SR-1
 import-route direct cost-type internal
 import-route static cost-type internal

After cost-type is changed to internal, the default cost of imported static routes is 0.

Solution

Change cost-type to internal on SR-1 when IS-IS imports routes.

Follow-up Procedure

Run the save command to save the current configuration to the configuration file when a set of configuration is finished and the expected functions have been achieved.

Summary

When Huawei devices and non-huawei devices co-exist on the same network, pay attention to the implementation differences.

OSPF Routes Fail to Load-balance Traffic

Fault Symptom
Figure 4-47 Networking

On the network shown in Figure 4-47, a static route to 10.53.80.0/24 is configured on AR2 and added to the OSPF routing table so that traffic from AR1 to 10.53.80.0/24 can be load-balanced between AR1 -> BAS -> 10.53.80.0/24 and AR1 -> AR2 -> BAS -> 10.53.80.0/24.

After the configuration, the traffic from AR1 to 10.53.80.0/24 is not load-balanced, and AR1 receives a route from BAS, but not from AR2.

Cause Analysis

Possible causes are as follows:

  • The FA is unreachable. As a result, the LSA is not calculated.
  • The conditions for OSPF route load balancing are not met.
Troubleshooting
  1. Check the configuration files. No exceptions exist.
  2. Check the OSPF LSDB on AR1. AR1 receives routes to 10.53.80.0/24 from BAS and AR2, and the FA of the LSA advertised by AR2 is 10.53.55.26.
    <AR1> display ospf lsdb ase 10.53.80.0
    
              OSPF Process 1 with Router ID 10.53.53.23
                      Link State Database
    
    
      Type      : External
      Ls id     : 10.53.80.0
      Adv rtr   : 10.53.53.21
      Ls age    : 1327
      Len       : 36
      Options   :  DC
      seq#      : 80000037
      chksum    : 0x3fc3
      Net mask  : 255.255.255.0
      TOS 0  Metric: 1
      E type    : 2
      Forwarding Address : 0.0.0.0
      Tag       : 1
    
      Type      : External
      Ls id     : 10.53.80.0
      Adv rtr   : 10.53.53.1
      Ls age    : 1157
      Len       : 36
      Options   :  E
      seq#      : 80000001
      chksum    : 0x2288
      Net mask  : 255.255.255.0
      TOS 0  Metric: 1
      E type    : 2
      Forwarding Address : 10.53.55.26
      Tag       : 1
    
  3. Check information about the routes to the FA in the LSA with a set FA and the neighbor. Such routes are available, indicating that the FA is reachable.
    <AR1> display ip routing-table 10.53.55.26
    
    
        10.53.55.24/30  OSPF   10   101         D  10.53.55.45     GigabitEthernet10/0/5
                        OSPF   10   101         D  10.53.55.49     GigabitEthernet10/0/6
    
    <AR1> display ip routing-table 10.53.53.1
    Route Flags: R - relay, D - download to fib
    ------------------------------------------------------------------------------  
    Routing Table : Public 
    Summary Count : 2
    
    Destination/Mask    Proto  Pre  Cost     Flags NextHop         Interface
    
         10.53.53.1/32  O_ASE  150  1           D  10.53.55.45     GigabitEthernet10/0/5
                        O_ASE  150  1           D  10.53.55.49     GigabitEthernet10/0/6
    
  4. The conditions for OSPF route load balancing are as follows:
    • The routes are of the same type (intra-area, inter-area, Type-1 external, or Type-2 external).
    • The routes have different direct next hops.
    • The routes have the same cost.
    • If the routes are Type-2 external routes, the costs of the links to the ASBR or forwarding address are the same.
    • If OSPF route selection specified in relevant standards is implemented, the routes have the same area ID.
    The routes sent by BAS and AR2 are carried in ASE LSPs. The route types are the same, but the next hops are different. The command output in step 2 shows that the metrics are both 1.
  5. Check the cost of the route to 10.53.53.21 in the first LSA. The cost is 1. The command outputs in step 2 and step 3 show that the cost of the route to the FA is 101. The costs of the two routes are different. As a result, OSPF routes fail to load-balance traffic.
Solution

Advertise the next hop of the route to 10.53.80.0 on BAS to AR1 using OSPF and set the cost of the outbound interface to 100 so that AR1 receives two LSAs with FAs and the costs of the routes to the FAs are the same (101).

Follow-up Procedure

Run the save command to save the current configuration to the configuration file when a set of configuration is finished and the expected functions have been achieved.

Summary

If OSPF routes fail to load-balance traffic, troubleshoot the problem based on the conditions for OSPF route load balancing.

After One of the AC Link That Load-balance L3VPN Traffic Fails, Services Are Interrupted

Fault Symptom
Figure 4-48 L3VPN load balancing

Figure 4-48 shows a service bearer network, on which the RNC is connected to two CEs and each CE is connected to an AR. The two CEs carry media services of the RNC. A static route to the logical IP address (service IP address) of the RNC is configured on each CE, and BFD for static routes is configured to detect the two static routes.

After the link between CE1 and the RNC fails, CE1 fails to ping the logical IP address of the RNC.

Cause Analysis

After the link between CE1 and the RNC fails, the traffic from AR1 to the RNC is supposed to travel along the AR1 -> CE1 -> CE2 -> RNC path. However, the traffic does not pass through the link between the CEs because CE1 does not select the route to CE2 or the route is discarded.

Troubleshooting

The configurations on the CEs are as follows:

  • An OSPF multi-instance is configured on both CEs, and the CEs are configured as MCEs.

  • The following black-hole route is configured on both CEs:
    ip route-static vpn-instance IuPS 10.10.10.0 255.255.255.0 NULL0
    NOTE:
    The CEs use black-hole routes to advertise routes because the CEs need to aggregate the services before advertising them to the AR bearer network.
  • An IP prefix list is configured for a route-policy to add the routes that match the route-policy to the OSPF routing table.

  • The IP prefix list configuration is as follows:
    ip ip-prefix prefix_iups index 10 permit  10.10.10.0 24

Only the routes that match the route-policy can be added to the OSPF routing table. After the link between CE1 and the RNC fails, 10.10.10.0 is the most precise route in the routing table of CE1 and therefore discarded based on the black-hole route.

Solution
  • Solution 1: Adjust the priority of the black-hole route to 200 so that the priority of the imported static route is higher than that of the black-hole route. Because the mask length of the service network segment is smaller than the aggregation network segment, a routing loop may occur if one host route is unreachable. Therefore, solution 1 is not recommended.
  • Solution 2: Modify the IP prefix list to match the routes with a mask range.
    ip-prefix prefix_iups index 10 permit 10.10.10.0 24 greater-equal 24 less-equal 32
    After the link between CE1 and the RNC fails, CE1 selects the route that matches the route-policy so that traffic can reach the RNC through CE2.
Follow-up Procedure

Run the save command to save the current configuration to the configuration file when a set of configuration is finished and the expected functions have been achieved.

Summary

None

Traffic Is Imbalanced along LDP over Tunnels

Fault Symptom

On the network shown in Figure 4-49, RT1 and RT2 are directly connected through three physical interfaces and three TE tunnels, and RT1 and RT2 are MPLS LDP over TE neighbors. Traffic is imbalanced among the tunnels.

Figure 4-49 LDP over TE networking
Cause Analysis

The traffic model mismatches the load balancing algorithm on the live network.

Troubleshooting
  1. Check the load balancing entries. The following command output shows that three tunnels are used for load balancing.
    [Huawei] display mpls lsp include 10.12.40.3 32
    -------------------------------------------------------------------------------
                     LSP Information: LDP LSP
    -------------------------------------------------------------------------------
    FEC                In/Out Label  In/Out IF                      Vrf Name
    10.12.40.3/32      NULL/3        -/Tun0/0/4059
    10.12.40.3/32      NULL/3        -/Tun0/0/4058
    10.12.40.3/32      NULL/3        -/Tun0/0/4068
    10.12.40.3/32      1270/3        -/Tun0/0/4059
    10.12.40.3/32      1270/3        -/Tun0/0/4058
    10.12.40.3/32      1270/3        -/Tun0/0/4068
    
  2. Check the traffic model of the device that receives the traffic. The traffic model is in the format of MAC address+double MPLS labels+IP packet.
  3. Check the information about the board that receives the traffic. Hash calculation is performed based on the destination and source IP addresses of traffic to implement load balancing on the board by default.

When hash calculation is performed based on the destination and source IP addresses of traffic and there are only a small number of IP addresses or several traffic flows of a large volume exist, traffic may be imbalanced.

Solution

Use the source IP address, destination IP address, source interface number, destination interface number, and protocol number as hash factors.

Summary

If traffic is imbalanced, change the hash factors.

BGP Routes Fail to Load-balance Traffic After a Network Cutover

Fault Symptom
Figure 4-50 Networking before the cutover

Figure 4-50 shows the networking before the cutover. EBGP peer relationships are established between the two egresses of each city of a MAN and two core nodes of the backbone network through loopback interfaces, and their connection forms a quadrangle-shaped network.

Figure 4-51 Networking after the cutover

After the cutover, the two egresses of each city of a MAN and two core nodes of the backbone network form a crossed network, as shown in Figure 4-50. The EBGP peers exchange routing information through loopback interfaces as they do before the cutover.

After the cutover, traffic is imbalanced between the two links connected to the two outbound interfaces of each egress.

Cause Analysis
Possible causes of the traffic imbalance are as follows:
  • BGP routes fail to load-balance traffic.
  • The load balancing algorithm mismatches the traffic model.
Troubleshooting
  1. Check the BGP routing table on each core node. No exceptions exist.

  2. Check the IP routing table on each egress. No exceptions exist.

  3. There are two BGP routes destined for the same IP address in the BGP routing table of each egress. However, only the one with a smaller router ID is selected.

  4. Check the configurations. The maximum load-balancing command is not run in the BGP view.

By default, BGP selects only one optimal route, and load-balancing is not supported.

Solution

Run the maximum load-balancing command in the BGP view.

Follow-up Procedure

Run the save command to save the current configuration to the configuration file when a set of configuration is finished and the expected functions have been achieved.

Summary

Run the maximum load-balancing command in the BGP view to enable load balancing if the networking of the live network is similar to that in this instance.

BGP VPN Equal-Cost Routes Fail to Load-balance Traffic

Fault Symptom
Figure 4-52 Load balancing among BGP routes

On the network shown in Figure 4-52, the upstream interface of RT3 is bound to VPN 2, and the downstream interface of RT3 is bound to VPN 1. RT3 is a BGP peer of RT1 and RT2 in AS 100, and RT1 and RT2 send one route to the same destination network segment (10.58.32.128 in this example) to RT3. Equal-cost routes exist in the VPN 2 routing table.

<RT3> display ip routing-table vpn-instance vpn2 10.58.32.128
Route Flags: R - relay, D - download to fib
------------------------------------------------------------------------------
Routing Table : vpn2
Summary Count : 2
Destination/Mask    Proto   Pre  Cost      Flags NextHop         Interface
   10.58.32.128/26  EBGP    255  0           D   10.46.85.97     GigabitEthernet1/0/0
                    EBGP    255  0           D   10.46.87.173    GigabitEthernet2/0/0

Traffic flows from VPN 1 to AS 100 through the interfaces that are established to VPN 2.

RT4-to-AS 100 traffic is imbalanced between the interfaces that are established to VPN 2, one of which does not receive traffic.

Cause Analysis
  • The source and destination IP addresses of some traffic are the same.
  • A level 2 hash operation is performed.
  • No BGP equal-cost routes are available.
  • The load balancing algorithm mismatches the traffic model.
Troubleshooting
  1. Check the traffic type. The traffic is transmitted by multiple L2TP tunnels whose ingress and egress have different IP addresses.
  2. Traffic from RT4 reaches RT3 through an Eth-Trunk interface and is not load-balanced between interfaces on RT4. Therefore, no level 2 hash calculation is performed.
  3. Check the path along which packets are transmitted. After reaching RT3, traffic is forwarded based on the VPN 1 routing table. There is only one route to the destination in the VPN 1 routing table, and the outbound interface of the route is the one that forwards traffic.

Only the optimal route in VPN 2 routing table is added to the VPN 1 routing table. As a result, there is only one route to the destination in the VPN 1 routing table.

Solution

Configure two equal-cost routes in the VPN 1 routing table.

  • Method 1: Configure a new VPN instance and change the VPN 2 that is bound to one of the interfaces that load-balance traffic to the new VPN instance. After the operation, each optimal route of the new VPN routing table and VPN2 routing table can be added to the VPN1 routing table.
  • Method 2: Configure two equal-cost routes in the VPN 1 routing table. This method is applicable when a small number of equal-cost routes are required.
    ip route-static vpn-instance1 10.58.32.128 26 vpn-instance vpn2 10.46.85.97
    ip route-static vpn-instance1 10.58.32.128 26 vpn-instance vpn2 10.46.87.173
Follow-up Procedure

Run the save command to save the current configuration to the configuration file when a set of configuration is finished and the expected functions have been achieved.

Summary

None

Traffic Is Imbalanced Because the Masks of Sub-interface IP addresses Are Different Although Per-Flow Load Balancing Is Configured

Fault Symptom
Figure 4-53 Networking

On the network shown in Figure 4-53, each site has two PEs, and the two sites are connected through sub-interfaces for dot1q VLAN tag termination across a transport network. PE1 and PE2 allocate shared bandwidth resources to four VLANs: VLAN 101, 102, 103, and 104. For example, 32 Mbit/s is allocated to VLAN 101, indicating that the sum of the bandwidth of VLAN 101 used by PE1, PE2, PE3, and PE4 is 32 Mbit/s. Bandwidth usage = Sum of bandwidth of each sub-interface/Total bandwidth.

OSPF runs on all PEs for communication.

The 2 G traffic volume of VLAN 101 is almost the same as that of VLAN 102, and the 2 G traffic volume of VLAN 103 is almost the same as that of VLAN 104. However, The 2 G traffic volume of VLAN 101 or VLAN 102 almost doubles that of VLAN 103 or VLAN 104.

Cause Analysis

Possible causes are as follows:

  • No routes are available for load balancing.
  • The load balancing algorithm mismatches the traffic model.
Troubleshooting
  1. Most of the 2 G traffic is transmitted along the routes in VPN-instance 2G_Traffic. Check the VPN-instance 2G_Traffic routing table. There are six routes in the routing table, in two of which the outbound interface is the sub-interface corresponding to VLAN 103 or VLAN 104.
    <PE1> display ip routing-table vpn-instance 2G_Traffic 10.67.0.0
    Route Flags: R - relay, D - download to fib
    ------------------------------------------------------------------------------
    Routing Table : 2G_Traffic
    Summary Count : 6
    Destination/Mask    Proto  Pre  Cost       Flags NextHop         Interface
      10.67.0.0/16      O_ASE  150  1            D   10.16.1.9       GigabitEthernet1/0/0.103
                        O_ASE  150  1            D   10.16.2.129     GigabitEthernet2/0/0.102
                        O_ASE  150  1            D   10.16.2.5       GigabitEthernet1/0/0.101
                        O_ASE  150  1            D   10.16.1.13      GigabitEthernet2/0/0.104
                        O_ASE  150  1            D   10.16.2.1       GigabitEthernet1/0/0.101
                        O_ASE  150  1            D   10.16.2.133     GigabitEthernet2/0/0.102
    
  2. Check the configurations of sub-interfaces in both sites. The masks of the sub-interfaces on the PEs are different (either 29 bits or 30 bits), as shown in Figure 4-54.

    Figure 4-54 Sub-interface IP addresses

    For VLAN 101 and VLAN 102, the four PEs are in the same network segment; for VLAN 103 and VLAN 104, P2P connections are used.

For PE1, VLAN 101 and VLAN 102 are connected to PE3 and PE4, but VLAN 103 and VLAN 104 are connected to only PE1. Therefore, there are six equal-cost routes in the routing table: two routes in both VLAN 101 and VLAN 102, and one route in both VLAN 103 and VLAN 104. Because the number of equal-cost routes in VLAN 101 and VLAN 102 is twice that in VLAN 103 and VLAN 104, the possibility that traffic runs through VLAN 101 and VLAN 102 also doubles.

Solution

Change the masks of the sub-interfaces on the PEs to 30 bits.

Follow-up Procedure

Run the save command to save the current configuration to the configuration file when a set of configuration is finished and the expected functions have been achieved.

Summary

Configure the same masks for the sub-interfaces that load-balance traffic.

SNMP Traffic Is Interrupted Because Strict URPF Is Enabled in a Load Balancing Scenario

Fault Symptom
Figure 4-55 Networking

On the network shown in Figure 4-55, two links load-balance traffic between Switch and RC, and strict URPF is enabled on RA and RB to prevent DDoS attacks from the Switch side.

After the configurations, service traffic is not affected, but SNMP traffic is interrupted. NMS Server can ping Port B but not Port A.

Cause Analysis

SNMP traffic is interrupted after strict URPF is enabled, indicating that packet loss due to strict URPF causes the interruption.

Troubleshooting
  1. Check the number of discarded packets on RB and record the number.
  2. Run the undo ip urpf command on RA and RB to disable URPF.
  3. Check the number of discarded packets on RB after URPF is disabled. The number does not increase.
  4. Re-enable URPF. After URPF is re-enabled, the number of discarded packets on RB keeps increasing.
  5. Run the snmp local interface loopback 0 command on Switch to enable it to responds with SNMP packets through a loopback interface. After the command is run, the problem is cleared.
  6. Ping Port A from NMS Server. The ping operation fails again.
  7. Tracert Port A from NMS Server. The tracert packets reach RB but are not returned by Switch.

Do not configure strict URPF in a load balancing scenario with two upstream links.

Solution

Run the ip urpf loose command to enable loose URPF on RA and RB.

After the command is run, SNMP traffic recovers, and SNMP Server succeeds in pinging Port A.

Follow-up Procedure

Run the save command to save the current configuration to the configuration file when a set of configuration is finished and the expected functions have been achieved.

Summary

Do not configure strict URPF in a load balancing scenario with two upstream links.

Translation
Download
Updated: 2019-06-11

Document ID: EDOC1000175918

Views: 5009

Downloads: 210

Average rating:
This Document Applies to these Products
Related Documents
Related Version
Share
Previous Next