A Service Anomaly Occurs Because the Switch Interface Is Not Bound to the Eth-Trunk

Publication Date:  2015-07-02 Views:  201 Downloads:  0
Issue Description
Network Topology:



Symptom:

The USG connects to the NE40E, S8500, and S9300, and all USG interfaces are Eth-Trunk interfaces. The USG operates in transparent mode. Three Eth-Trunk interfaces are added to the same VLAN. The IP addresses of the corresponding Vlanif interfaces reside on the same network segment as that of the S8500 and NE40E. The S9300 is connected lately.

Services are normal before the S9300 is connected. After the S9300 is connected, ping the Vlanif address of the USG from the S8500. The ping operation sometimes succeeds and sometimes fails. Collect traffic statistics on the USG. The statistics show that the number of received ICMP request packets is only half of the transmitted ICMP reply packets. Some ICMP reply packets are discarded because the inbound and outbound interfaces are the same.

[USG-diagnose] display firewall statistic acl 
Current Show sessions count: 1
Protocol(ICMP) SourceIp(1.1.2.2) DestinationIp(1.1.2.1)   
SourcePort(43982) DestinationPort(43982) VpnIndex(public)   
           Receive           Forward           Discard   
Obverse : 4          pkt(s) 4          pkt(s) 0          pkt(s)   
Reverse : 8          pkt(s) 4          pkt(s) 8          pkt(s)
Discard detail information:
  DP_FW_L2Distribute            :exit 4:     4
  FPATH_TP_Input                :exit 3:     4
Handling Process
1. Before the S9300 is not connected, services are normal. The S8500 has correct ARP entries of the NE40E and USG. The USG has correct ARP entries of the S8500 and NE40E.

2. The S9300 without any configuration is connected. When the ARP entry of the USG on the S8500 ages, the S8500 sends an ARP request to the USG. Because ARP requests are broadcast packets, the USG broadcasts the ARP request packet to the S9300 and NE40E, returns an ARP reply to the S8500, and learns proper ARP entries. The S9300 is not configured. Therefore, the S9300 broadcasts the packet to all interfaces in Up state. Therefore, the packet is returned to the USG. The ARP entry on the USG is refreshed again, and the outbound interface becomes the Eth-Trunk interface connecting to the S9300.

3. On the S8500, ping the Vlanif address of the USG. The USG sends a reply not to the S8500, but to the S9300. Because the S9300 learns all MAC addresses from the USG, the S9300 sends the reply back to the USG. Therefore, the number of reply packets is doubled on the USG.

4. The returned packet is not destined to the USG. Therefore, the USG looks up the MAC forwarding table. If the interface with the specified MAC address is on the S9300 side, the USG considers the packet illegitimate because the inbound interface of the packet is also the outbound interface specified in the MAC forwarding table. Then the USG discards the packet, and the ping operation fails on the S8500. If the interface with the specified MAC address in the MAC forwarding table is on the S8500 side, the packet can be correctly forwarded, and the ping operation succeeds on the S8500.

5. The S8500 determines whether the MAC forwarding table on the USG is correct or not. If the S8500 sends a broadcast packet, the USG broadcasts the packet, and the S9300 sends it back. Therefore, the MAC forwarding table on the USG becomes incorrect. The interface corresponds to the MAC address of the S8500 changes to that of the S9300. If the S8500 sends a unicast packet, the USG does not broadcast the packet, but forwards the packet based on the MAC forwarding table (unless no entry is found in the MAC forwarding table). At the same time, the USG refreshes the MAC forwarding table. Then the MAC forwarding table on the USG becomes correct, and the interface corresponds to the MAC address of the S8500 changes to that of the S8500.
Root Cause
According to the traffic statistics, the ICMP reply packets that the USG sends are returned by a certain device. Therefore, the ICMP reply packets are counted twice. If the reply packets are destined to the S8500, the destination MAC address is that of the S8500 and the S8500 will not send the reply packets back.

Therefore, the reply packets must be sent back by either the S9300 or NE40E. Because services are normal before the S9300 is connected, it is possible that the S9300 sends the reply packets back. Then why does the S9300 send the ICMP packets back? The only reason is that the USG sends the ICMP reply packets to the S9300. If so, the outbound interface corresponding to the MAC address of the S8500 in the ARP entry on the USG is the interface connecting to the S9300.

Further analysis shows that the ARP request to the S8500 is from the S9300 or the USG broadcasts the request but the S9300 returns it back. Because the S9300 and S8500 are not directly connected, the later is more possible. The analysis result indicates that the S9300 returns the packets broadcast by the USG. Therefore, it is deduced that the interface on the S9300 connecting to USG is not bound to any Eth-Trunk. As is known, the S9300 is not configured before being connected to the USG. Construct a similar networking environment in the lab. The symptom is the same as that on the live network.
Solution
Configure Eth-Trunk binding on the S9300.

END