No relevant resource is found in the selected language.

This site uses cookies. By continuing to browse the site you are agreeing to our use of cookies. Read our privacy policy>

Reminder

To have a better experience, please upgrade your IE browser.

upgrade

Service affected because ISIS protocol has error calculation.

Publication Date:  2012-07-27 Views:  2 Downloads:  0
Issue Description
At the certain time customer's service was affected and huge logs appeared in the logbuffer:

oct  3 2011 15:39:18.294.4 mah-ne80e-1 %%01isis/6/gt_lgcl_subnet_fail(l): isis 1 failed to obtain the logical index for local subnet while matching the nexthop address 10.222.30.1/32. (physicalindex=10)
oct  3 2011 15:39:18.294.3 mah-ne80e-1 %%01isis/6/gt_lgcl_subnet_fail(l): isis 1 failed to obtain the logical index for local subnet while matching the nexthop address 10.222.30.1/32. (physicalindex=10)
oct  3 2011 15:39:18.294.2 mah-ne80e-1 %%01isis/6/gt_lgcl_subnet_fail(l): isis 1 failed to obtain the logical index for local subnet while matching the nexthop address 10.222.30.1/32. (physicalindex=10)
oct  3 2011 15:39:18.294.1 mah-ne80e-1 %%01isis/6/gt_lgcl_subnet_fail(l): isis 1 failed to obtain the logical index for local subnet while matching the nexthop address 10.222.30.1/32. (physicalindex=10)

when the problem was happened, from mah-ne80e-1 and mah-ne80e-2, it can’t ping to asbr (10.222.254.77 and 10.222.254.177) , after reset the isis, the problem was recovered. This problem depends from software version V300R003C02B697


Alarm Information
oct  3 2011 15:39:18.294.4 mah-ne80e-1 %%01isis/6/gt_lgcl_subnet_fail(l): isis 1 failed to obtain the logical index for local subnet while matching the nexthop address 10.222.30.1/32. (physicalindex=10)
oct  3 2011 15:39:18.294.3 mah-ne80e-1 %%01isis/6/gt_lgcl_subnet_fail(l): isis 1 failed to obtain the logical index for local subnet while matching the nexthop address 10.222.30.1/32. (physicalindex=10)
oct  3 2011 15:39:18.294.2 mah-ne80e-1 %%01isis/6/gt_lgcl_subnet_fail(l): isis 1 failed to obtain the logical index for local subnet while matching the nexthop address 10.222.30.1/32. (physicalindex=10)
oct  3 2011 15:39:18.294.1 mah-ne80e-1 %%01isis/6/gt_lgcl_subnet_fail(l): isis 1 failed to obtain the logical index for local subnet while matching the nexthop address 10.222.30.1/32. (physicalindex=10)

Handling Process
from the logs of the router, in the 4 seconds between oct  3 2011 15:39:14 and 15:39:18, there is 3 times isis neighbor flapping, the neighbor is down and up for 6 times.
in that 4 seconds, there is a lot of isis logs, the logs means: when the isis is refreshing the routes, the interface is down, it cause refreshing failure.
 
oct  3 2011 15:39:18.294.4 mah-ne80e-1 %%01isis/6/gt_lgcl_subnet_fail(l): isis 1 failed to obtain the logical index for local subnet while matching the nexthop address 10.222.30.1/32. (physicalindex=10)
oct  3 2011 15:39:18.294.3 mah-ne80e-1 %%01isis/6/gt_lgcl_subnet_fail(l): isis 1 failed to obtain the logical index for local subnet while matching the nexthop address 10.222.30.1/32. (physicalindex=10)
……..
oct  3 2011 15:39:15.657.3 mah-ne80e-1 %%01isis/6/gt_lgcl_subnet_fail(l): isis 1 failed to obtain the logical index for local subnet while matching the nexthop address 10.222.30.1/32. (physicalindex=10)
oct  3 2011 15:39:15.657.2 mah-ne80e-1 %%01isis/6/gt_lgcl_subnet_fail(l): isis 1 failed to obtain the logical index for local subnet while matching the nexthop address 10.222.30.1/32. (physicalindex=10)
 
after that los, the neighbor status is changed many times, and latest status is up.
oct  3 2011 15:39:18.325.1 mah-ne80e-1 %%01isis/4/adj_change(l): the neighbor of isis was changed. (isisprocessid=1, neighbor=0102.2225.5030, interfacename=ge3/0/0, currentstate=up, changetype=3_way_up)
oct  3 2011 15:39:18.324.1 mah-ne80e-1 %%01isis/4/adj_change(l): the neighbor of isis was changed. (isisprocessid=1, neighbor=0102.2225.5030, interfacename=ge3/0/0, currentstate=init, changetype=new_adj_create)
oct  3 2011 15:39:18.314.1 mah-ne80e-1 %%01isis/4/adj_change(l): the neighbor of isis was changed. (isisprocessid=1, neighbor=0102.2225.5030, interfacename=ge3/0/0, currentstate=down, changetype=circuit_down)
oct  3 2011 15:39:18.304.3 mah-ne80e-1 %%01isis/4/adj_change(l): the neighbor of isis was changed. (isisprocessid=1, neighbor=0102.2225.5030, interfacename=ge3/0/0, currentstate=up, changetype=3_way_up)
oct  3 2011 15:39:18.303.4 mah-ne80e-1 %%01isis/4/adj_change(l): the neighbor of isis was changed. (isisprocessid=1, neighbor=0102.2225.5030, interfacename=ge3/0/0, currentstate=init, changetype=new_adj_create)
 
oct  3 2011 15:39:15.655.3 mah-ne80e-1 %%01isis/4/adj_change(l): the neighbor of isis was changed. (isisprocessid=1, neighbor=0102.2225.5030, interfacename=ge3/0/0, currentstate=down, changetype=circuit_down)
oct  3 2011 15:39:15.194.2 mah-ne80e-1 %%01isis/4/adj_change(l): the neighbor of isis was changed. (isisprocessid=1, neighbor=0102.2225.5030, interfacename=ge3/0/0, currentstate=up, changetype=3_way_up)
oct  3 2011 15:39:15.192.1 mah-ne80e-1 %%01isis/4/adj_change(l): the neighbor of isis was changed. (isisprocessid=1, neighbor=0102.2225.5030, interfacename=ge3/0/0, currentstate=init, changetype=new_adj_create)
oct  3 2011 15:39:14.457.2 mah-ne80e-1 %%01isis/4/adj_change(l): the neighbor of isis was changed. (isisprocessid=1, neighbor=0102.2225.5030, interfacename=ge3/0/0, currentstate=down, changetype=circuit_down)
 
 
after that, the isis neighbor is not changed any more.
from the los, we can find the interface down/up issue is match the time of isis flapping .
 
oct  3 2011 15:39:14.454.2 mah-ne80e-1 %%01lspm/6/slototherevent(l): got interface event down and address 0.0.0.0 in interface gigabitethernet3/0/0.
oct  3 2011 15:39:14.455.3 mah-ne80e-1 %%01ifnet/4/linkno_state(l): the line protocol on the interface gigabitethernet3/0/0 has entered the down state.
oct  3 2011 15:39:14.949.1 mah-ne80e-1 %%01lspm/6/slototherevent(l): got interface event up and address 0.0.0.0 in interface gigabitethernet3/0/0.
oct  3 2011 15:39:14.950.1 mah-ne80e-1 %%01ifnet/4/linkno_state(l): the line protocol on the interface gigabitethernet3/0/0 has entered the up state.
. . . . . . . . . . . .
oct  3 2011 15:39:18.311.3 mah-ne80e-1 %%01lspm/6/slototherevent(l): got interface event down and address 0.0.0.0 in interface gigabitethernet3/0/0.
oct  3 2011 15:39:18.312.2 mah-ne80e-1 %%01ifnet/4/linkno_state(l): the line protocol on the interface gigabitethernet3/0/0 has entered the down state.
oct  3 2011 15:39:18.321.2 mah-ne80e-1 %%01lspm/6/slototherevent(l): got interface event up and address 0.0.0.0 in interface gigabitethernet3/0/0.
because the isis neighbor status changing is follow the interface flapping, interface flapping cause the isis calculate error
Root Cause
When one interface is down/up so many times at several seconds, when the isis neighbor is up , and at that time point, the router is doing partial route calculate, route is not start refreshing or is doing refreshing, the interface is down, it cause route refreshing error (it print logs) . if before the route refreshing, the latest isis neighbor status is up, the routes will be refreshing to wrong status.
we have recurred this problem in the lab, recur probability is too low. so this problem is only happened one time in customer’s network.

Suggestions
1  change the configuration of “timer spf 5 50 50” to “timer spf 1 50 50”.
decrease the isis routes calculate timer to 1 second, it will speed up routes calculate, it will not make so many isis neighbor changes between the time of two routes calculate, it will not make this problem happen.
2 we find the isis flapping issue is happen on the interface of 3/0/0, so it is better to make the configuration of “carrier down-hold-time 600”.
when that interface can’t received laser any more, after 600ms, the interface will be down, this command can forbidden interface flapping.
 

END