Solutions to E1000E Firewall high CPU issue

Publication Date:  2013-10-31 Views:  454 Downloads:  0
Issue Description
HRP_M[Eudemon]dis cpu-usage-for-user                                                                                              
===== Current CPU usage info =====                                                                                                
CPU Usage Stat. Cycle: 37 (Second)                                                                                                
CPU Usage            : 98%                                                                                                         
CPU Usage Stat. Time : 2009-12-28  00:25:20                                                                                       
CPU Usage Stat. Tick : 0x77d(CPU Tick High) 0xf251d663(CPU Tick Low)                                                              
Actual Stat. Cycle   : 0x0(CPU Tick High) 0x44ef06d8(CPU Tick Low) 
Alarm Information
None
Handling Process
1. Find the source of the attack or replace higher-performance equipment to withstand attacks. If the attacker's source or destination is fixed, you can configure packet filtering to prevent packets or add a route to the destination of the black hole.
2.
a) In the case of guaranteed p2p effect, as much as possible the number of small packets detected, turn off the global p2p function only in specific inter-domain restrictions.
2) If the address pool address as a firewall interface ip, then turned off from the external network to the NAT address pool initiated packet filtering;
3) If the non-interface address pool address, you need to configure the address pool to the next hop is NULL 0
3. Configuring routed to NULL 0; configure packet filtering to prohibit the broadcast packets, broadcast packets and find the source, reducing the amount of broadcast packets sent.
4. The network traffic larger, reaching firewall performance, only replace the higher performance firewall.
5. Open acl acceleration, see if cpu goes down.
Root Cause
1. When Firewall receives an attack, and attack traffic much more than a firewall can handle. It will looks like below:
a) Log as follows: SYN-flood attacks, these attacks on the firewall PING source address is the ping fails.
%Aug 31 11:00:40 2009 Firewall SEC/5/ATCKDF:AttackType:Syn flood attack; Receive
Interface: Ethernet0/0/2 ; proto:TCP ; from 83.114.172.2:17420 31.44.12.28:4593
0 155.97.157.53:33398 221.0.78.42:51273 95.37.83.115:57169 149.84.187.55:34919 1
86.68.20.53:22088 80.11.89.63:31504 181.43.195.33:44363 14.122.55.33:6174 126.10
2.83.29:17447 77.107.21.94:44098 ; to 118.123.248.184:25511 58.218.178.62:80 ; b
egin time :2009/08/31 11:00:10; end time: 2009/08/31 11:00:40; total packets: 43
5252; max speed: 14530(packet/s); 
b) Viewing interface errors, Resource errors indicate interface processing performance over packet loss.

   Input: 61560749 packets, 3817466697 bytes                                 
           14 broadcasts (0.00%), 0 multicasts (0.00%)                        
           91573 errors, 8170 runts, 0 giants, 663 CRC,                       
           0 collisions, 0 late collisions, 0 overruns,                       
           0 jabbers, 0 input no buffers, 82740 Resource errors,

2. Open p2p detection function, and the configuration of the NAT outbound
a) View Configuration Discovery Configuring the p2p function, and application of the NAT outbound function
nat address-group 2 X.X225.33 X.X.225.33
interface GigabitEthernet2/0/1                                                                                                    
     ip address X.X.225.33 255.255.255.224
b) View the packets to the address pool a lot, if ip address pool and interface are the same, because it is initiated from outside the network packets, then the time the firewall itself sending packets processed, resulting in busy handling their own firewall packets; if ip address pool and the interface is not the same, there may be a firewall between the external network and the gateway to form a loop.

[Eudemon]display firewall  session table  verbose  source global  X.X.225.33
   Current Total Sessions : 23112
c) View ip statistics, there are a lot of TTL timeout packets and sent their report, and the second is growing rapidly.

[Eudemon]display  ip statistics                                                                                                
  Input:   sum            37267830      local            8992747                                                                  
           bad protocol       0      bad format           0                                                                       
           bad checksum       0      bad options          0                                                                       
           TTL exceeded   27392399                                                                                                
  Output:  forwarding       405      local            2526854905                                                                  
           dropped            0      no route             0                                                                       
  Fragment:input             19      output               0                                                                       
           dropped            0                                                                                                   
           fragmented         0      couldn't fragment    0                                                                       
  Reassembling:sum            5      timeouts             4                                      

3. View interface statistics more, and does so much traffic, interfaces exist overruns or Resource errors, out of the total interface traffic statistics little difference; Network New Connection larger average packet is small, it is because the performance lead over cpu utilization is high.

4. Whether the rule in many configurations, and does not open the acl acceleration, for over 1000 rules would have to open the acl acceleration, otherwise if the existing network created more when consumed on performance is relatively large.
Suggestions
None

END