Packet loss of voice service because AG management service of S9030 caused network loop

Publication Date:  2012-11-06 Views:  320 Downloads:  0
Issue Description
Version:S9303 V100R002C00SPC200
Network topology:
S9303-1(1/0/32vlan1203)----
|         |--AG-1
|         |--S3328--IAD
S9303-2(1/0/32vlan1202)----
Two S9303 is connected by VRRP, and S9303-1 is the master. The port from AG to S9303-2 is UP. S3328 is connected with two S9303 at their port 1/0/32.
A severe packet loss problem happened from S9303-1 to signaling address of AG, but no problem from S9303-1 to management address of AG.
Alarm Information
#Nov 18 19:30:00 2010 Cssn-s2-a-gansuqy-5 L2IF/4/MAC_FLAPPING_ALARM:OID 1.3.6.1.4.1.2011.5.25.42.2.1.7.12The mac-address has flap value. (BaseTrapSeverity=0, BaseTrapProbableCause=0, BaseTrapEventType=4, L2IfPort=549,entPhysicalIndex=1, MacAdd=0025-9e70-a59c,vlanid=1202, FormerIfDescName=GigabitEthernet1/0/27,CurrentIfDescName=GigabitEthernet1/0/25,DeviceName=Cssn-s2-a-gansuqy-5)
*0.5895075 Cssn-s2-a-gansuqy-5 L2IF_MAC/7/MACFLAPPING:MAC-Flapping Mac: 0025-9e70-a59c, IfName: GigabitEthernet1/0/25, Vlan: 1202, Slot: 1
*0.5901003 Cssn-s2-a-gansuqy-5 L2IF_MAC/7/MACFLAPPING:MAC-Flapping Mac: 0025-9e70-a59c, IfName: GigabitEthernet1/0/32, Vlan: 1203, Slot: 1
Handling Process
1、According to the failure of client reported, test the direct link of two S9303 that did not lose packet, so did S9303 and AG direct connection link. exclude the link problem;
2、There is the same failure on the 3 AG connecting with S9303,AG engineer take the master and backup switching, the problem remianed. exclude the AG device peoblem;
3、Check S9303vVRRP state that is normal, exclude the VRRP poblem;
4、Replace VRRP heartbeat link from GE optical board to FE electrical interface board, the problem remianed. exclude the board peoblem.
5、1. Turn on the mac-flapping check in S9303-1, the statistics increase because of mac-flapping.
command as follows:
[S9303-hidecmd]mac-flapping check enable
[S9303-hidecmd]display mac-flapping  statistics
------------------------
Slot    Total          
------------------------
  1      93     
------------------------
Total Flapping:93
[S9303-hidecmd]display mac-flapping statistics
------------------------
Slot    Total          
------------------------
  1      170    
------------------------
Total Flapping:170
6、 Turn on the mac-flapping alarm , t d, t m switch
 S9300 series switch MAC flapping alarm is not in loggbuff, it need to  Turn on the mac-flapping alarm , t d, t m switch, command is as follows:
[S9303]mac-flapping alarm  enable
#Nov 18 19:30:00 2010 Cssn-s2-a-gansuqy-5 L2IF/4/MAC_FLAPPING_ALARM:OID 1.3.6.1.4.1.2011.5.25.42.2.1.7.12The mac-address has flap value . (BaseTrapSeverity=0, BaseTrapProbableCause=0, BaseTrapEventType=4, L2IfPort=549,entPhysicalIndex=1, MacAdd=0025-9e70-a59c,vlanid=1202, FormerIfDescName=GigabitEthernet1/0/27,CurrentIfDescName=GigabitEthernet1/0/25,DeviceName=Cssn-s2-a-gansuqy-5)
*0.5895075 Cssn-s2-a-gansuqy-5 L2IF_MAC/7/MACFLAPPING:MAC-Flapping Mac: 0025-9e70-a59c, IfName: GigabitEthernet1/0/25, Vlan: 1202, Slot: 1
*0.5901003 Cssn-s2-a-gansuqy-5 L2IF_MAC/7/MACFLAPPING:MAC-Flapping Mac: 0025-9e70-a59c, IfName: GigabitEthernet1/0/32, Vlan: 1203, Slot: 1
AG signal MAC is learnt  from 1/0/25 and 1/0/27 on S9303-1,VLAN is 1202。1/0/25 is  the practical interface of AG,1/0/27 is the interconnection of 2 S9303, 1/0/32 interface learn the mac of AG from VLAN1203.
7、2 S9303 1/0/32 connected the same S3328 switch,and all the interface of switch default by VLAN1。S9303-1 1/0/32 default vlan 1203 ,S9303-2 1/0/32 default vlan 1202。
topology:S9303-1(1/0/32vlan1203)----
            |                        |
            |                        |--S3328--IAD
          S9303-2(1/0/32vlan1202)----
8、VLAN1202 is the signal VLAN,VLAN1203 is media VLAN,for AG using the same MAC address. media packet VLAN1203 MAC through S9303-1(1/0/32)--S3328--S9303-2(1/0/32)--S9303-1(1/0/27) creat signal VLAN1202 MAC table.
9、Negotiation with client to close S9303-1 1/0/32 interface, check mac table that is normal,test signal and media address that do not lose.1/0/32 default VLAN did not refer to  network management VLAN, so there is no MAC flapping problem,and it did not lose packet.
Root Cause
1、Link problem;
2、AG device problem;
3、VRRP state problem;
4、S9300 hardware problem
5、other reason
Suggestions
Layout the network to avoid the loop.

END