No relevant resource is found in the selected language.

This site uses cookies. By continuing to browse the site you are agreeing to our use of cookies. Read our privacy policy>

Reminder

To have a better experience, please upgrade your IE browser.

upgrade

Because of attack makes CPU high on NE40E

Publication Date:  2013-07-15 Views:  128 Downloads:  0
Issue Description
CPU on NE40E is high. And service is impacted. We checked CPU information and found CPU is high on LPU1 board.
#Automatic record log end,current health information as follows:
Slot                    CPU Usage     Memory Usage (Used/Total)
---------------------------------------------------------------
5       MPU(System Master)  6%           18%  340MB/1877MB
1       LPU                92%           33%  274MB/818MB
2       LPU                21%           33%  274MB/818MB
4       MPU                 5%           17%  319MB/1877MB
#DateTime Stamp: 2012-01-07 17:44:17.200
################################################################
Alarm Information
Check logs and found below alarm:
Jan  7 2012 17:45:42 NE40E-X3-PE-01 %%01VOSCPU/4/CPU_USAGE_HIGH(l)[79]:Slot=1;The CPU is overloaded, and the tasks with top three CPU occupancy are SOCK, PES, SRM. (CpuUsage=96%, Threshold=80%)

Jan  7 2012 17:42:39 NE40E-X3-PE-01 %%01VOSCPU/4/CPU_USAGE_HIGH(l)[80]:Slot=1;The CPU is overloaded, and the tasks with top three CPU occupancy are SOCK, PES, VIDL. (CpuUsage=90%, Threshold=80%)
Handling Process
According to anti-attack function on NE40E, we can use command “display attack-source-trace”  to locate the attack source.
<NE40E-X3-PE-01>display cpu-defend car protocol arp statistics slot 1

Slot               : 1

Application switch : Open

Default Action     : Min-to-cp

--------------------------------------------

IPV4 ARP packet

Protocol switch: N/A

Packet information:

  Passed packet(s)  : 4783398            

  Dropped packet(s) : 250749283          

Configuration information:

  Configged CIR : 2000    kbps       Actual CIR in NP : 2000    kbps

  Configged CBS : 20000   bytes      Actual CBS in NP : 20000   bytes

  Priority : The index on this board can not be shown . Please see the NP Priority.

  Min-packet-length : NA

<NE40E-X3-PE-01> display attack-source-trace slot 1 original-information
Info: Please waiting............
Slot: 1
Attack-source-trace Capacity:1M
No.1
Packet Info:
Interface Name    : GigabitEthernet1/0/22.900
Vlanid            : 0  
Attack Type       : ARP   
------------ 

From the output, i can see there is many arp request packets dropped.
We also found the attack packets IP 10.125.97.1 and MAC 78-1d-ba-a4-7d-4d.
Check interface g1/0/22.900 as below:
GigabitEthernet1/0/22.900 current state : UP

Line protocol current state : UP

Last line protocol up time : 2011-10-12 12:38:46

Description:TO CX-NMS

Route Port,The Maximum Transmit Unit is 1500

Internet Address is 10.125.97.1/27

IP Sending Frames' Format is PKTFMT_ETHNT_2, Hardware address is 781d-baa4-7d4d

We can decide that there is loop under interface gi1/0/22.900. It makes there is arp storm in Customer Network. It is root cause.
Root Cause
According to alarm information and high CPU process, we doubt there is attack in customer network.
Suggestions
We can confirm attack on NE40E using below two ways:
1.display  cpu-defend  car protocol
It is used to check packet dropped. If there is loop path in network, we can check ARP and VRRP protocol packets usually. The output of this command is different in V300R003 and V600R001 version. We need confirm protocol name with Index in V300R003 version. But on V600R001 and later, the command can show you the protocol directly.
2. Using "display attack-source-trace" is very useful. There is three parameters you can choose. Different output for different parameter.
<40E-R4>display attack-source-trace slot 1 ?
  brief                 Brief
  original-information  Attack data original information
  verbose               Verbose

From the output, we can get which port has attack according to packet's IP and MAC address.

END