Two S6700 switches have the Rapid Spanning Tree Protocol (RSTP) protocol enabled globally, and the Open Shortest Path First (OSPF) and Virtual Router Redundancy Protocol (VRRP) are running between the switches. When the master switch receives a large number of TC packets, the status of the OSPF and VRRP protocols flaps on the switch. Both the switches record logs about high CPU usage, and the CPU usage displayed on the network management system (NMS) exceeds 90% multiple times. Logs show that the many ARP packets are dropped because the rate of ARP packets exceeds the CPCAR.
Figure shows the CPU usage data on the NMS.
Figure CPU usage data on the NMS
Log Information:
1. The switches have recorded logs about high CPU usage.
S6700-1 %%01VOSCPU/4/CPU_USAGE_HIGH(l)[31]:The CPU is overloaded(CpuUsage=96%, Threshold=95%), and the tasks with top three CPU occupancy are:
FTS total : 18%
SRMT total : 11%
SOCK total : 8%
S6700-1 %%01VOSCPU/4/CPU_USAGE_HIGH(l)[60]:The CPU is overloaded(CpuUsage=100%, Threshold=95%), and the tasks with top three CPU occupancy are:
PPI total : 41%
SRMT total : 10%
FTS total : 8%
2. There are also logs indicating that a large number of ARP packets have been discarded because of CPCAR exceeding.
S6700-1 %%01DEFD/4/CPCAR_DROP_MPU(l)[56]:Rate of packets to cpu exceeded the CPCAR limit on the MPU. (Protocol=arp-miss, ExceededPacketCount=016956)
S6700-1 %%01DEFD/4/CPCAR_DROP_MPU(l)[57]:Rate of packets to cpu exceeded the CPCAR limit on the MPU. (Protocol=arp-reply, ExceededPacketCount=020699)
S6700-1 %%01DEFD/4/CPCAR_DROP_MPU(l)[58]:Rate of packets to cpu exceeded the CPCAR limit on the MPU. (Protocol=arp-request, ExceededPacketCount=0574)
3. Check statistics about TC packets sent and received on RSTP-enabled interfaces.
The number of received TC packets keeps increasing on all RSTP-enabled interfaces.
<S6700> display stp tc-bpdu statistics
-------------------------- STP TC/TCN information --------------------------
MSTID Port TC(Send/Receive) TCN(Send/Receive)
0 GigabitEthernet0/0/1 19319/3271 0/0
0 GigabitEthernet0/0/2 29761/676 0/0
0 GigabitEthernet0/0/3 128/4 0/0
0 GigabitEthernet0/0/4 24615/1016 0/0
0 GigabitEthernet0/0/5 30697/98 0/0
0 GigabitEthernet0/0/6 25447/317 0/0