S12700交换机收到TC报文导致业务丢包.

发布时间:  2016-12-05 浏览次数:  112 下载次数:  1
问题描述

1.1   问题涉及的设备及版本

设备类型

版本

补丁

S12708

v200r008c00spc500

v200r008sph006

1.2        网络拓朴


S12708堆叠为核心,下挂多接入设备,接入设备下挂IP话机和终端、服务器


1.3   现网问题描述

使用过程中出现IP话机不通,同时ping服务器有丢包

处理过程

1)   查看127日志,收到大量的arp报文。

Nov 24 2016 09:51:43+08:00 TJ-DS-OA-S12708-01 %%01DEFD/6/CPCAR_DROP_LPU(l)[138211]:Rate of packets to cpu exceeded the CPCAR limit on the LPU in slot 1/3. (Protocol=arp-request, CIR/CBS=64/12032, ExceededPacketCount=4627)

Nov 24 2016 09:51:43+08:00 TJ-DS-OA-S12708-01 %%01DEFD/6/CPCAR_DROP_LPU(l)[138212]:Rate of packets to cpu exceeded the CPCAR limit on the LPU in slot 1/3. (Protocol=arp-miss, CIR/CBS=64/12032, ExceededPacketCount=27675)

Nov 24 2016 09:51:43+08:00 TJ-DS-OA-S12708-01 %%01DEFD/6/CPCAR_DROP_LPU(l)[138213]:Rate of packets to cpu exceeded the CPCAR limit on the LPU in slot 1/4. (Protocol=arp-request, CIR/CBS=64/12032, ExceededPacketCount=136290)

Nov 24 2016 09:51:43+08:00 TJ-DS-OA-S12708-01 %%01DEFD/6/CPCAR_DROP_LPU(l)[138215]:Rate of packets to cpu exceeded the CPCAR limit on the LPU in slot 1/4. (Protocol=arp-miss, CIR/CBS=64/12032, ExceededPacketCount=1651247)

Nov 24 2016 09:51:43+08:00 TJ-DS-OA-S12708-01 %%01DEFD/6/CPCAR_DROP_LPU(l)[138217]:Rate of packets to cpu exceeded the CPCAR limit on the LPU in slot 2/3. (Protocol=arp-request, CIR/CBS=64/12032, ExceededPacketCount=4353)

Nov 24 2016 09:51:43+08:00 TJ-DS-OA-S12708-01 %%01DEFD/6/CPCAR_DROP_LPU(l)[138218]:Rate of packets to cpu exceeded the CPCAR limit on the LPU in slot 2/3. (Protocol=arp-miss, CIR/CBS=64/12032, ExceededPacketCount=26811)

Nov 24 2016 09:51:43+08:00 TJ-DS-OA-S12708-01 %%01DEFD/6/CPCAR_DROP_LPU(l)[138219]:Rate of packets to cpu exceeded the CPCAR limit on the LPU in slot 2/4. (Protocol=arp-request, CIR/CBS=64/12032, ExceededPacketCount=122811)

Nov 24 2016 09:51:43+08:00 TJ-DS-OA-S12708-01 %%01DEFD/6/CPCAR_DROP_LPU(l)[138221]:Rate of packets to cpu exceeded the CPCAR limit on the LPU in slot 2/4. (Protocol=arp-miss, CIR/CBS=64/12032, ExceededPacketCount=1105371)

2)   查看攻击溯源,有部分MAC发的大量的arp报文。

<TJ-DS-OA-S12708-01>dis  auto-defend attack-source sl  1/4  d

  Attack Source User Table (LPU1/4):

  ----------------------------------------------------

  MAC Address                    fc4d-d4d9-7664     

  Interface                      XGigabitEthernet1/4/0/27

  VLAN: Outer/Inner              2638                

      ARP:                       2385   

  Total                          2385               

  ----------------------------------------------------

  Total: 1

 

  Attack Source IP Table (LPU1/4):

  ----------------------------------------------------

  IP address                     10.137.51.136      

      ARP:                       2385   

  Total                          2385               

  ----------------------------------------------------

  Total: 1

<TJ-DS-OA-S12708-01>dis  auto-defend attack-source sl  2/4  d

  Attack Source User Table (LPU2/4):

  ----------------------------------------------------

  MAC Address                    6c0b-8442-f0ca     

  Interface                      XGigabitEthernet2/4/0/30

  VLAN: Outer/Inner              2640               

      ARP:                       1470   

  Total                          1470               

  ----------------------------------------------------

  ----------------------------------------------------

  MAC Address                    0024-7e03-5c39     

  Interface                      XGigabitEthernet2/4/0/14

  VLAN: Outer/Inner              2632               

      ARP:                       930    

  Total                          930                

  ----------------------------------------------------

  Total: 2

 

  Attack Source IP Table (LPU2/4):

  ----------------------------------------------------

  IP address                     10.137.53.33       

      ARP:                       1470   

  Total                          1470               

  ----------------------------------------------------

  ----------------------------------------------------

  IP address                     10.137.45.10       

      ARP:                       930    

  Total                          930                

  ----------------------------------------------------

  Total: 2

<TJ-DS-OA-S12708-01>dis  auto-defend attack-source sl  2/3  d

  Attack Source User Table (LPU2/3):

  ----------------------------------------------------

  MAC Address                    4437-e65d-f3ae     

  Interface                      XGigabitEthernet2/3/0/34

  VLAN: Outer/Inner              2627               

      ARP:                       3745   

  Total                          3745                

  ----------------------------------------------------

  Total: 1

 

  Attack Source IP Table (LPU2/3):

  ----------------------------------------------------

  IP address                     10.137.230.37      

      ARP:                       3745   

  Total                          3745               

  ----------------------------------------------------

  Total: 1

<TJ-DS-OA-S12708-01>dis  auto-defend attack-source sl  1/3  d

  Attack Source User Table (LPU1/3):

  ----------------------------------------------------

  MAC Address                    4439-c48d-fc57     

  Interface                      XGigabitEthernet1/3/0/26

  VLAN: Outer/Inner              2619               

      ARP:                       945    

  Total                          945                

  ----------------------------------------------------

  ----------------------------------------------------

  MAC Address                    fc4d-d447-334a     

  Interface                      XGigabitEthernet1/3/0/13

  VLAN: Outer/Inner              2606               

      ARP:                       1445   

  Total                          1445               

  ----------------------------------------------------

  ----------------------------------------------------

  MAC Address                    4437-e65d-f36c     

  Interface                      XGigabitEthernet1/3/0/34

  VLAN: Outer/Inner              2627               

      ARP:                       1105   

  Total                          1105                

  ----------------------------------------------------

  Total: 3

 

  Attack Source IP Table (LPU1/3):

  ----------------------------------------------------

  IP address                     10.137.221.14      

      ARP:                       945    

  Total                          945                

  ----------------------------------------------------

  ----------------------------------------------------

  IP address                     10.137.207.35      

      ARP:                       1445   

  Total                          1445               

  ----------------------------------------------------

  ----------------------------------------------------

  IP address                     10.137.230.26      

      ARP:                       1105   

  Total                          1105               

  ----------------------------------------------------

 

3)   查看STP拓扑状态,有拓扑状态变化记录。

<TJ-DS-OA-S12708-01>dis  stp to

<TJ-DS-OA-S12708-01>dis  stp topology-change

 CIST topology change information

   Number of topology changes             :45

   Time since last topology change        :0 days 0h:42m:58s

   Topology change initiator(notified)    :Eth-Trunk27

   Topology change last received from     :84ad-58d9-b030

   Number of generated topologychange traps :   0

   Number of suppressed topologychange traps:   0

4)   查看STP TC统计在出现ping服务器丢包时,有TC统计增加

<TJ-DS-OA-S12708-01>dis stp  tc-b  s

 -------------------------- STP TC/TCN information --------------------------

 MSTID Port                        TC(Send/Receive)      TCN(Send/Receive)

 0     XGigabitEthernet1/3/0/40    120/0                 0/0

 ...

 0     Eth-Trunk22                 134/0                 0/0

 0     Eth-Trunk23                 138/0                 0/0

 0     Eth-Trunk24                 143/0                 0/0

 0     Eth-Trunk25                 135/5                 0/0

 0     Eth-Trunk26                 126/13                0/0

 0     Eth-Trunk27                 29/71                 0/0

 0     Eth-Trunk28                 121/0                 0/0

 

5)   综上,127在收到TC报文时,会清ARPMAC,由于下挂设备发送的ARP报文较多,大量的ARP报文会把正常的ARP丢弃,所以ARP学习比正常时要慢,在网络中部署优化措施后解决。

6)   客户排查网络,发现接入交换机下有接cisco交换机,cisco交换机上没有配置边缘端口,所以有终端接入或开关机时会触发STP TC报文。

解决方案

部署优化措施后解决。

mac-address update arp   

//用来使能MAC刷新ARP功能,即MAC地址的出接口变化时,通知更新ARP表项的出接口。

ip forwarding converge normal

//用来去使能设备进行环网切换时,IP流量走二层转发流程

arp topology-change disable

//用来去使能设备响应TC报文的功能(即当设备收到TC报文时,不对ARP表项进行老化或删除)

END