No relevant resource is found in the selected language.

This site uses cookies. By continuing to browse the site you are agreeing to our use of cookies. Read our privacy policy>Search

Reminder

To have a better experience, please upgrade your IE browser.

upgrade

Lots of Large-Sized Packets Are Discarded When a Third-Party Device Ping an NE40E

Publication Date:  2019-03-27 Views:  94 Downloads:  0

Issue Description



On the network shown in the preceding figure, an NE40E is directly connected to a Maipu device, and the two devices are on the same network segment.

https://support.huawei.com/enterprisecase/product/images/8a0d68c464aa4b14a077d26d801042dc

The customer feedback shows that a lot of large-sized packets are discarded when the Maipu device pings the NE40E. When the NE40E pings the Maipu device, the packet size is set to 9600 bytes, and no packets are lost. When the Maipu device pings the NE40E, the packet size is set to 10000 bytes. In this case, packet loss occurs. When 1000 packets are pinged, two packets or even dozens of packets are discarded.

Handling Process

1. Check that the MTU of the interface connecting the Maipu device to the NE40E is 1500, and jumbo packets are processed as fragments.

The default MTU of the NE40E is 1500 bytes, and the value ranges from 46 to 9600.

<HUAWEI>display interface
  GigabitEthernet 2/0/1

GigabitEthernet2/0/1 current state : UP
  (ifindex: 50)

Line protocol current state : UP

Last line protocol up time : 2018-08-27
  14:53:39

Link quality grade : GOOD

Description:

Route Port,The Maximum Transmit Unit is 1500

Internet Address is 12.12.12.1/30

IP Sending Frames' Format is PKTFMT_ETHNT_2,
  Hardware address is f86e-ee98-cb79

The Vendor PN is SP7041-E

The packet header obtaining result shows that the 1000-byte ping packets sent by the Maipu device contain seven fragments. Each of the first six fragments contain 1500 bytes, and the last fragment contains 1120 bytes. Therefore, the MTU of the packets sent by the Maipu device is also 1500 bytes.

https://support.huawei.com/enterprisecase/product/images/78b4a4a659824044aa192da15be62a19

The payload and a 20-byte IP packet header constitute the IP packet length.


     2. Obtain packet headers on the NE40E. It is found that the packets are discarded on the NE40E.
    Obtain packet headers on GigabitEthernet 2/0/1 of the NE40E in both the inbound and outbound directions.
    The Maipu device sends packets to the NE40E: seq 0~999  no packet loss

 https://support.huawei.com/enterprisecase/product/images/001de36a0a50483f9248b951c5fe7ffd

    The NE40E times out in receiving some fragments and fails in reassembly. The NE40E does not respond to the Maipu device.

https://support.huawei.com/enterprisecase/product/images/8865554027634fb0966753d285073b06
    The analysis shows that the problem occurs on the NE40E. GigabitEthernet 2/0/1 receives all the packets from the Maipu device in the inbound direction. However, some fragments of the packets sent to the CPU are discarded, leading to a CPU reassembly failure. The NE40E does not send a reply packet to the Maipu device. This is why jumbo ping packets are discarded.
    3. Check that the packet loss on the NE40E is caused by the threshold-crossing CPU configuration.
    To optimize delay processing, the NE40E is enabled with ICMP fast reply by default when the Maipu device pings the NE40E. Specifically, data packets are not forwarded to the CPU.
However, if the received data packets are ping packets, reassembly is required, and the packets need to be sent to the CPU. To prevent the CPU from being attacked, the NE40E implements CPU for such packets. The data packets with a threshold-crossing CAR value are discarded.
It is suspected that the Maipu device sends packets at a threshold-crossing rate, and the packets are discarded if the CAR value exceeds the threshold. The test result shows that among the 1000 10000-byte ping packets sent by the Maipu device to the NE40E, two packets are lost. Before the test, reset packet statistics.

<HUAWEI>reset cpu-defend car fragment statistics slot 2
<HUAWEI>display cpu-defend car fragment statistics slot 2
 Slot               : 2
 Application switch : Open
 Default Action     : Min-to-cp
--------------------------------------------
 Fragment Packet
 Protocol switch: N/A
 Packet information:
  Passed packet(s)  : 6993
  Dropped packet(s) : 7
 Configuration information:
  Configured CIR : 512       kbps       Actual CIR in NP : 512       kbps
  Configured CBS : 9000000   bytes      Actual CBS in NP : 9000000   bytes
  Priority : af3
  Min-packet-length : 128 bytes
 History information:
  Last drop:
   Start time: 2018-08-27 17:22:00
   End time  : 2018-08-27 17:24:00
   Last drop rate(pps): 0
   Total dropped packet(s): 6
  Peak rate:
   Time: 2018-08-27 15:05:00
   Peak rate(pps): 392
    When the Maipu device sends a 10000-byte ping packet, the packet is divided into seven fragments. Therefore, a total of 7000 packets are sent. The NE40E receives 6993 packets, and seven packets are discarded. The NE40E discards packets at random if the CAR value exceeds the threshold. If the discarded seven packets contain fragments of two 100000-byte packets, the two 10000-byte packets are discarded due to a reassembly failure. Consequently, two ping packets are discarded.
    4. Check that packet loss occurs due to threshold-crossing CAR on the packets received by the NE40E.
Check the rate at which GI 2/0/1 of the NE40E receives packets.

<HUAWEI>display traffic policy statistics interface gi 2/0/1 inbound verbose rule-based 
Interface: GigabitEthernet2/0/1
Traffic policy inbound: 3334
Traffic policy applied at 2018-08-27 17:03:12
Statistics enabled at 2018-08-27 17:03:12
Statistics last cleared: 2018-08-27 17:08:55
Rule number: 4 IPv4, 0 IPv6
Current status: OK!

Classifier: 3334 operator or
 if-match acl 3334
 rule 5 permit ip source 12.12.12.2 0 destination 12.12.12.1 0
    10,218,000 bytes, 7,000 packets
    Last 30 seconds rate 219 pps, 2,547,688 bps
    The packet receiving rate is about 2547 kbps, which is greater than the rate specified in the cpu car 512kbps configuration. Therefore, packet loss occurs due to threshold-crossing CAR.

Root Cause

When the Maipu device pings the ping NE40E, fragmented packets are sent to the CPU for processing, and the packet receiving rate exceeds the specified CPU CAR value. Therefore, some fragmented packets are discarded, causing a reassembly failure. The NE40E does not respond to the Maipu device, leading to discarding of some ping packets.

Solution

CPU CAR is configured to prevent lots of attack packets from being sent to the CPU. CPU CAR, however, does not need to be configured in a ping test which is performed to check network connectivity. Therefore, increasing the CPU CAR value is not recommended. Keep the default value of CPU CAR.

END