How Do I Adjust Eth-Trunk Configurations When Eth-Trunk Load Balancing Is Uneven

This document describes how to determine whether Eth-Trunk load balancing is uneven and how to adjust Eth-Trunk configurations in this scenario.

This document uses S6720EI running V200R010C00 as an example. There may be differences in the licensing requirements and limitations and command restrictions of different switch models and versions. For details, see the corresponding product documentation. For licensing requirements and limitations, see "Licensing Requirements and Limitations for Link Aggregation" under CLI-based Configuration Guide > Ethernet Switching Configuration Guide > Link Aggregation Configuration. For command restrictions, see Command Reference.

The InUti and OutUti fields in the display interface brief command output show the average bandwidth usages of an interface in the last 300 seconds.

<HUAWEI> display interface brief
InUti/OutUti: input utility/output utility
Interface                   PHY   Protocol InUti OutUti   inErrors  outErrors
GigabitEthernet0/0/1        up    up       0.06%   0.06%       315        408
GigabitEthernet0/0/2        up    up       0.06%   0.07%       229        306

Normally, traffic is load balanced over multiple member links after an Eth-Trunk is deployed for load balancing. In this case, the bandwidth usage of each link does not exceed 80%, and the bandwidth usage difference between links does not exceed 30%. This effectively improves service reliability. If traffic is transmitted over only one link or load balanced over some links, but not all links of an Eth-Trunk, uneven load balancing occurs. As a result, the bandwidth usage of a single link may exceed 80% and services may be affected.

Eth-Trunk load balancing can be classified as packet-based load balancing and flow-based load balancing. Since the data volume of each traffic flow is different, no matter which load balancing mode is used, traffic cannot be absolutely evenly distributed to all member links of an Eth-Trunk. As per-packet load balancing may cause packet disorder, it is not recommended for services that are sensitive to packet sequence. In this case, you are advised to use flow-based load balancing which ensures the correct packet order. In V200R011C10, S series switches support only flow-based load balancing. For details, see Understanding Eth-Trunk Load Balancing of S Series Switches.

If Eth-Trunk load balancing is uneven, perform the following steps to troubleshoot the fault.

1. Check whether the interface with unbalanced traffic loads belongs to an inter-chassis Eth-Trunk in a CSS or stack.

   If so, adjust the configuration according to Configuring Load Balancing on an Inter-Chassis Eth-Trunk.

   If not, go to step 2.

2. Check whether equal and weighted cost multi-path (ECMP) and Eth-Trunk load balancing are configured together on the device.

   If so, adjust the configuration according to Configuring ECMP and Eth-Trunk Load Balancing.

   If not, go to step 3.

3. Check whether packets being load balanced are known or non-known unicast packets. If the destination MAC address of a packet is not in the MAC address table, the packet is a non-known unicast packet.
   NOTE:

   The load balancing mode takes effect only on the outbound interface of traffic. If traffic of a single type accounts for a much larger proportion than traffic of other types, you need to change the traffic path on the outbound interface of the upstream device to evenly distribute traffic to the local device or expand the outbound interface capacity of the local device, for example, changing a GE port to an XGE port.

   If so, adjust the Eth-Trunk load balancing configuration according to Configuring Load Balancing for Non-known Unicast Packets.

   If not, adjust the Eth-Trunk load balancing configuration according to Configuring Loading Balancing for Known Unicast Packets.

For known unicast traffic, you can preferentially configure common load balancing. If common load balancing cannot meet your requirements, configure enhanced load balancing. The recommended load balancing mode varies according to the traffic type. Select a proper load balancing mode according to Table 1-1 and perform the corresponding configuration. Unless otherwise specified, the number of member interfaces in an Eth-Trunk must be a multiple of two. For details about how to configure Eth-Trunk load balancing, see Configuring Common Load Balancing and Configuring Enhanced Load Balancing. For details about how to view the Protocol field in a packet, see Obtaining Packet Headers.

Figure 1-1 Typical MPLS networking

<HUAWEI> system-view
[HUAWEI] interface eth-trunk 1
[HUAWEI-Eth-Trunk1] load-balance src-dst-mac
[HUAWEI-Eth-Trunk1] quit
[HUAWEI] display eth-trunk 1 load-balance
Eth-Trunk1's load-balance information:
 Load-balance Configuration: SA-XOR-DA
 Load-balance options used per-protocol:
  L2  : Source XOR Destination MAC address, Vlan ID, Ethertype, Ingress-port
  IPv4: Source XOR Destination MAC address, Vlan ID, Ethertype, Ingress-port
  IPv6: Source XOR Destination MAC address, Vlan ID, Ethertype, Ingress-port
  MPLS: Source XOR Destination MAC address, Vlan ID, Ethertype, Ingress-port

<HUAWEI> system-view
[HUAWEI] load-balance-profile a
[HUAWEI-load-balance-profile-a] ipv6 field sip protocol
[HUAWEI-load-balance-profile-a] quit
[HUAWEI] interface eth-trunk 1
[HUAWEI-Eth-Trunk1] load-balance enhanced profile a
[HUAWEI-Eth-Trunk1] quit
[HUAWEI] display eth-trunk 1 load-balance
Eth-Trunk1's load-balance information:
 Load-balance Configuration: ENHANCED
 Load-balance enhanced profile: a
 Load-balance options used per-protocol:
  L2  : Source XOR Destination MAC address
  IPv4: Source XOR Destination IP address
  IPv6: Source IP address, IP protocol
  MPLS: Top XOR Second label

In a CSS or stack, an inter-chassis Eth-Trunk is configured as the outbound interface of traffic to ensure reliable transmission. When the CSS or stack forwards traffic, the Eth-Trunk may select an inter-chassis member interface based on the hash algorithm. The cable bandwidth between devices in the CSS or stack is limited, so inter-chassis traffic forwarding occupies bandwidth resources between devices, lowering traffic forwarding efficiency. To solve this problem, inter-chassis Eth-Trunks on S series switches preferentially forward local traffic by default. It is normal that the bandwidth usage difference between interfaces on different devices exceeds 30%.

If active member interfaces of an Eth-Trunk have insufficient bandwidth to forward local traffic and the bandwidth usage of a single link exceeds 80%, disable the Eth-Trunk from preferentially forwarding local traffic; otherwise, traffic may be discarded.

If services are insensitive to forwarding performance, you can run the undo local-preference enable command in the Eth-Trunk view to disable the Eth-Trunk from preferentially forwarding local traffic, although member interfaces have sufficient bandwidth to forward local traffic. This enhances reliability and ensures the Eth-Trunk to carry burst traffic when a link fault occurs.

ECMP first distributes traffic to multiple Eth-Trunks and then Eth-Trunks further load balance traffic over member links. Ensure that ECMP evenly distributes traffic and then perform the following steps to adjust the Eth-Trunk load balancing configuration.

1. If traffic is evenly distributed after common ECMP load balancing is performed, there is no limit on the Eth-Trunk load balancing mode. Continue to check for other possible causes that lead to uneven load balancing.
2. If traffic is unevenly distributed after common ECMP load balancing is performed, configure enhanced ECMP load balancing. In this case, common Eth-Trunk load balancing is preferred.
3. If traffic is unevenly distributed after enhanced ECMP load balancing and common Eth-Trunk loading balancing are performed, configure enhanced Eth-Trunk load balancing. In this case, the Eth-Trunk must have more effective hash factors than ECMP.
4. If traffic is extremely unevenly distributed on the live network and a large number of hash factors are configured for both enhanced ECMP and Eth-Trunk load balancing, adjust the numbers of ECMP member paths and Eth-Trunk member links. Ensure that the numbers of ECMP member paths and Eth-Trunk member links do not have any common divisor. For example, the numbers cannot be 4 and 4, 4 and 2, or 6 and 3, but can be set to 3 and 4, or 4 and 5.

Non-known unicast packets include unknown unicast, multicast, and broadcast packets. When a large number of non-known unicast packets are transmitted and unevenly distributed on interfaces, you can run the unknown-unicast load-balance enhanced command in the system view to apply an enhanced load balancing profile to implement enhanced load balancing.

NOTE:

To enable enhanced load balancing of non-known unicast traffic on modular switches, run the unknown-unicast load-balance enhanced lbid command. In V200R010C00, when the ES0D0G24SA00 and ES0D0G24CA00 of the S7700 or the EH1D2G24SSA0 and EH1D2S24CSA0 of the S9700 are used with other types of cards together, the unknown-unicast load-balance enhanced lbid command cannot be run. Otherwise, packet loss may occur or excess packets may be received.

The following provides two methods to obtain the Portal field in a packet header to determine the packet type or view other information carried in the packet header.

<HUAWEI> system-view
[HUAWEI] observe-port 1 interface GigabitEthernet 1/0/1
[HUAWEI] interface eth-trunk 1
[HUAWEI-Eth-Trunk1] port-mirroring to observe-port 1 outbound

Obtaining Packet Headers Using the capture-packet Command

If you cannot obtain packet headers in mirroring mode, run the capture-packet interface interface-typeinterface-number destinationterminal command in the system view to obtain packet headers. You are advised to obtain packet headers in more than 50 groups at a specified interval and then summarize and analyze all the obtained packet headers.

NOTE:

The number of packet headers obtained by using the capture-packet command is relatively small and sometimes cannot reflect the actual traffic model on the live network.

Understanding Eth-Trunk Load Balancing of S Series Switches

S1720, S2700, S5700, and S6720 V200R011C10 Configuration Guide - Ethernet Switching