某广电局点S9300下挂用户无法正常通过DHCP获取ip地址问题处理过程

发布时间:  2016-08-15 浏览次数:  464 下载次数:  0
问题描述
S9300下挂用户无法正常获取(DHCP)IP地址
处理过程

1、查看设备log,提示大量CP-CAR DHCP丢包。

2、查看CPU利用率,DHCP进程利用率较大

<BiShan-9306-127.201>display cpu-usage
CPU Usage Stat. Cycle: 60 (Second)
CPU Usage            : 48% Max: 99%
CPU Usage Stat. Time : 2016-07-28  20:07:48
CPU utilization for five seconds: 48%: one minute: 48%: five minutes: 47%.

TaskName        CPU  Runtime(CPU Tick High/Tick Low)  Task Explanation
BOX              0%         0/  deb165       BOX Output                   
_TIL             0%         0/       0       Infinite loop event task     
_EXC             0%         0/       0       Exception Agent Task         
VIDL            52%         5/1fd304a8       DOPRA IDLE                   
TICK             0%         0/ d49c503           

.......    
DHCP            21%         2/2071dfc6       DHCP Dynamic Host Config Protocol
AAA              0%         0/  179627       AAA  Authen Account Authorize 

...... 
OS              11%         1/135f9afe       Operation System  

3、查看CPU-DEFEND计数,发现某一个(2号)槽位的dhcp-server报文计数很大,dhcp-client 报文正常  

<BiShan-9306-127.201>
<BiShan-9306-127.201>display cpu-defend statistics packet-type dhcp-client all
Statistics on mainboard:
-------------------------------------------------------------------------------
Packet Type         Pass(Bytes)  Drop(Bytes)   Pass(Packets)   Drop(Packets)
-------------------------------------------------------------------------------
dhcp-client                   0            0               0               0
-------------------------------------------------------------------------------
Statistics on slot 1:
-------------------------------------------------------------------------------
Packet Type         Pass(Bytes)  Drop(Bytes)   Pass(Packets)   Drop(Packets)
-------------------------------------------------------------------------------
dhcp-client              658604            0            1761               0
-------------------------------------------------------------------------------
Statistics on slot 2:
-------------------------------------------------------------------------------
Packet Type         Pass(Bytes)  Drop(Bytes)   Pass(Packets)   Drop(Packets)
-------------------------------------------------------------------------------
dhcp-client              377854            0            1019               0
-------------------------------------------------------------------------------
<BiShan-9306-127.201>display cpu-defend statistics packet-type dhcp-client all
Statistics on mainboard:
-------------------------------------------------------------------------------
Packet Type         Pass(Bytes)  Drop(Bytes)   Pass(Packets)   Drop(Packets)
-------------------------------------------------------------------------------
dhcp-client                   0            0               0               0
-------------------------------------------------------------------------------
Statistics on slot 1:
-------------------------------------------------------------------------------
Packet Type         Pass(Bytes)  Drop(Bytes)   Pass(Packets)   Drop(Packets)
-------------------------------------------------------------------------------
dhcp-client              658604            0            1761               0
-------------------------------------------------------------------------------
Statistics on slot 2:
-------------------------------------------------------------------------------
Packet Type         Pass(Bytes)  Drop(Bytes)   Pass(Packets)   Drop(Packets)
-------------------------------------------------------------------------------
dhcp-client              377854            0            1019               0
-------------------------------------------------------------------------------
<BiShan-9306-127.201>
<BiShan-9306-127.201>
<BiShan-9306-127.201>display cpu-defend statistics packet-type dhcp-server all
Statistics on mainboard:
-------------------------------------------------------------------------------
Packet Type         Pass(Bytes)  Drop(Bytes)   Pass(Packets)   Drop(Packets)
-------------------------------------------------------------------------------
dhcp-server                   0            0               0               0
-------------------------------------------------------------------------------
Statistics on slot 1:
-------------------------------------------------------------------------------
Packet Type         Pass(Bytes)  Drop(Bytes)   Pass(Packets)   Drop(Packets)
-------------------------------------------------------------------------------
dhcp-server           315304452      3085262          868244            8915
-------------------------------------------------------------------------------
Statistics on slot 2:
-------------------------------------------------------------------------------
Packet Type         Pass(Bytes)  Drop(Bytes)   Pass(Packets)   Drop(Packets)
-------------------------------------------------------------------------------
dhcp-server          7908354303  1765798682k        15569471      4911923678
-------------------------------------------------------------------------------
<BiShan-9306-127.201>display cpu-defend statistics packet-type dhcp-server all
Statistics on mainboard:
-------------------------------------------------------------------------------
Packet Type         Pass(Bytes)  Drop(Bytes)   Pass(Packets)   Drop(Packets)
-------------------------------------------------------------------------------
dhcp-server                   0            0               0               0
-------------------------------------------------------------------------------
Statistics on slot 1:
-------------------------------------------------------------------------------
Packet Type         Pass(Bytes)  Drop(Bytes)   Pass(Packets)   Drop(Packets)
-------------------------------------------------------------------------------
dhcp-server           315304452      3085262          868244            8915
-------------------------------------------------------------------------------
Statistics on slot 2:
-------------------------------------------------------------------------------
Packet Type         Pass(Bytes)  Drop(Bytes)   Pass(Packets)   Drop(Packets)
-------------------------------------------------------------------------------
dhcp-server          7912039365  1766934991k        15579791      4915109150
-------------------------------------------------------------------------------
<BiShan-9306-127.201>        

4、配置攻击朔源,定位具体端口(2/0/13)收到dhcp-server过多。

#
cpu-defend policy arp-policy
auto-defend enable
auto-defend trace-type source-mac source-ip source-portvlan
auto-defend protocol all

#
cpu-defend-policy arp-policy global
#

[BiShan-9306-127.201]display auto-defend attack-source slot 2
  Attack Source User Table (LPU2):
  -------------------------------------------------------------------------
      MacAddress       InterfaceName      Vlan:Outer/Inner      TOTAL 
  -------------------------------------------------------------------------
  0023-b83c-e215   GigabitEthernet2/0/13        1               9504   
  -------------------------------------------------------------------------
  Total: 1

  Attack Source Port Table (LPU2)
  -----------------------------------------------------
    InterfaceName        Vlan:Outer/Inner       TOTAL  
  -----------------------------------------------------
  GigabitEthernet2/0/13    1                    11520 
  -----------------------------------------------------
  Total: 1

  Attack Source IP Table (LPU2)
  -------------------------------------
   IPAddress        TOTAL Packets 
  -------------------------------------
  -------------------------------------
  Total: 0
[BiShan-9306-127.201-cpu-defend-policy-arp-policy]display auto-defend attack-source slot 2
  Attack Source User Table (LPU2):
  -------------------------------------------------------------------------
      MacAddress       InterfaceName      Vlan:Outer/Inner      TOTAL 
  -------------------------------------------------------------------------
  0023-b83c-e215   GigabitEthernet2/0/13        1               13104  
  -------------------------------------------------------------------------
  Total: 1

  Attack Source Port Table (LPU2)
  -----------------------------------------------------
    InterfaceName        Vlan:Outer/Inner       TOTAL  
  -----------------------------------------------------
  GigabitEthernet2/0/13    1                    15968 
  -----------------------------------------------------
  Total: 1

  Attack Source IP Table (LPU2)
  -------------------------------------
   IPAddress        TOTAL Packets 
  -------------------------------------
  -------------------------------------
  Total: 0

5、根据攻击朔源的报文特征,临时将这部分报文通过黑名单丢弃,然后查看业务正常,CPU利用率恢复正常。

#
acl number 4567
rule 5 permit source-mac 0023-b83c-e215
#

#

cpu-defend policy arp-policy
blacklist 1 acl 4567

#


[BiShan-9306-127.201]display cpu-usage 
CPU Usage Stat. Cycle: 60 (Second)
CPU Usage            : 11% Max: 99%
CPU Usage Stat. Time : 2016-07-28  20:50:09
CPU utilization for five seconds: 11%: one minute: 12%: five minutes: 13%.

TaskName        CPU  Runtime(CPU Tick High/Tick Low)  Task Explanation


根因
1、设备收到大量的dhcp-server报文,导致CPU处理dhcp正常报文异常。
解决方案

1、通过黑名单丢弃攻击报文,防止CPU处理不过来场景。

#
acl number 4567
rule 5 permit source-mac 0023-b83c-e215
#

#

cpu-defend policy arp-policy
blacklist 1 acl 4567

#

建议与总结

1、故障处理,根据日志去进行判断和定位。

2、保持内网环境清洁,根本上解决攻击问题。

END