No relevant resource is found in the selected language.

This site uses cookies. By continuing to browse the site you are agreeing to our use of cookies. Read our privacy policy>

Reminder

To have a better experience, please upgrade your IE browser.

upgrade

Add new member interface into a Eth-trunk between Router_X and Router_Y cause the packet loss,protocol flapping and OSPF peer reset.

Publication Date:  2012-07-27 Views:  48 Downloads:  0
Issue Description
At the Live network operation, we found OSPF peer reset during adding the new trunk member (GE port) at eth-trunk between X location of Router to Y location of Router.


Alarm Information
OSPF reset alarm was come at U2000.

Handling Process
When there is service traffic on the device, and if we need  the operation of Join and Delete member interface from Eth-trunk interface.
we should follow the below step:
  1. First, We should shutdown the interface,
  2. Then do the operation,
  3. At the end of operation, then undo shutdown the interface.  

     


Root Cause
Reason:

On the interface 8/0/0 of Router_X, the interface joined to the Eth-trunk2 at the situation of not shutdown, and opposite device Y is still not set 8/8/0 interface to join to the Eth-trunk2, so when Eth-trunk calculate, it have hashed the ospf packet to the interface of 8/0/0.

Then it because Router_Y can't receive any ospf hello packet, the timer of ospf hello is 2 seconds, and when it can't received hello packet for 3 times, the ospf is down.  The reason is 1-way.

Analysis:

From the log of Router_X:

Nov  3 2011 01:41:38 Router_X %%01SHELL/5/CMDRECORD(l): task: vt0 ip: 10.254.0.11 user: cno command: un ip add.----------> Delete the ip address
#Nov  3 01:41:38 2011 Router_X IFNET/4/TRAP:1.3.6.1.6.3.1.1.5.3 interface 536871169 turns into DOWN state.
Nov  3 2011 01:41:38 Router_X %%01IFNET/4/LINK_STATE(l): Line protocol on interface GigabitEthernet8/0/0 has turned into DOWN state.---------->
Line protocol down.

Nov  3 2011 01:42:26 Router_X %%01SHELL/5/CMDRECORD(l): task: vt0 ip: 10.254.0.11 user: cno command: eth-tr 2.

Nov  3 2011 01:42:37 Router_X %%01OSPF/6/NBR_CHG_E(l): Neighbor event, neighbor state was changed. (Process ID=1, Neighbor address=192.168.127.73, Neighbor event=1-Way, Neighbor previous state=Full, Neighbor current state=Init)

From the logs of Router_Y:

Y_Router didn't have any delete ip operation, but the link protocol down, it is because the opposite interface down.

Nov  3 2011 01:41:29 Router_Y %%01IFNET/4/LINKNO_STATE(l)[54296]:The line protocol on the interface GigabitEthernet8/0/8 has entered the DOWN state.

At the same time, this router can't received any hello packet from opposite device, then opsf down.

Nov  3 2011 01:42:36 Router_Y %%01OSPF/6/IF_CHG_E(l)[54315]:Interface 192.168.127.73 received event NeighborChange, interface state changed from BackupDR to DR. (ProcessId=1)
Nov  3 2011 01:42:36 Router_Y %%01OSPF/3/NBR_CHG_DOWN(l)[54316]:Neighbor event:neighbor state changed to Down. (ProcessId=1, NeighborAddress=192.168.125.247, NeighborEvent=InactivityTimer, NeighborPreviousState=Full, NeighborCurrentState=Down)

Below is the logs of Nov 3, when customer do operation on the Y_Router, we didn't find undo ip address operation, only find joined trunk operation.

Nov  2 2011 18:18:16 Router_Y %%01SHELL/5/CMDRECORD(l)[53660]:Record command information. (Task=VT0 , Ip=10.254.0.11, User=cno, Command="quit")
Nov  3 2011 00:10:58 Router_Y %%01SHELL/6/DISPLAY_CMDRECORD(l)[54129]:Record command information. (Task=VT0 , Ip=10.254.0.11, User=cno, Command="display temperature")
Nov  3 2011 00:11:52 Router_Y %%01SHELL/6/DISPLAY_CMDRECORD(l)[54132]:Record command information. (Task=VT0 , Ip=10.254.0.11, User=cno, Command="display device")
Nov  3 2011 00:24:48 Router_Y %%01SHELL/6/DISPLAY_CMDRECORD(l)[54150]:Record command information. (Task=VT0 , Ip=10.254.0.11, User=cno, Command="display temperature")
Nov  3 2011 01:36:46 Router_Y %%01SHELL/6/DISPLAY_CMDRECORD(l)[54255]:Record command information. (Task=VT1 , Ip=10.254.0.11, User=cno, Command="display current-configuration interface Eth-Trunk 12")


Analysis the other link, the problem is same. But the customer is first do the operation the NE40E, so the 1-way logs is print on Router_Y.


Suggestions
Follow the upper step, during adding new trunk member at Live network operation. Otherwise, maybe there is packet loss and protocol flapping & as a result ospf will be reset.


END