S12700交换机(v200r008c00spc500版本)VRRP主备地址之间无法ping通

发布时间:  2016-08-01 浏览次数:  106 下载次数:  1
问题描述

客户割接思科交换机,割接完成后,出现VRRP主备地址之间无法ping通的情况。

告警信息


检查日志,发现有路由超规格告警

XA-QYZX-S12708-A告警:
#Jul 30 2016 05:30:43+08:00 XA-QYZX-S12708-A L3MB/4/FWDRESLACK:OID 1.3.6.1.4.1.2011.5.25.227.2.1.5 The layer 3 resource usage has reached or exceeded 85%.(EntPhysicalindex=67371017,EntPhysicalName=LPU Board 1,Slot=1,ResourceType=47)


#Jul 30 2016 05:30:43+08:00 XA-QYZX-S12708-A L3MB/4/FWDRESLACK:OID 1.3.6.1.4.1.2011.5.25.227.2.1.5 The layer 3 resource usage has reached or exceeded 85%.(EntPhysicalindex=67371017,EntPhysicalName=LPU Board 1,Slot=1,ResourceType=28)


#Jul 30 2016 05:30:41+08:00 XA-QYZX-S12708-A FIB/1/OVLDFORWARD:Slot=1;OID 1.3.6.1.4.1.2011.5.25.129.2.9.3 The interface board is in the overload forwarding state because the FIB module is overloaded. (EntityPhysicalIndex=0, HwBaseTrapSeverity=1, HwBaseTrapProbableCause=1, HwBaseTrapEventType=4, HwFibOverloadModule=1, entPhysicalName=slot )


XA-QYZX-S12708-C告警:
#Jul 30 2016 05:38:13+08:00 XA-QYZX-S12708-C L3MB/4/FWDRESLACK:OID 1.3.6.1.4.1.2011.5.25.227.2.1.5 The layer 3 resource usage has reached or exceeded 85%.(EntPhysicalindex=67371017,EntPhysicalName=LPU Board 1,Slot=1,ResourceType=47)


#Jul 30 2016 05:38:13+08:00 XA-QYZX-S12708-C L3MB/4/FWDRESLACK:OID 1.3.6.1.4.1.2011.5.25.227.2.1.5 The layer 3 resource usage has reached or exceeded 85%.(EntPhysicalindex=67371017,EntPhysicalName=LPU Board 1,Slot=1,ResourceType=28)


#Jul 30 2016 05:38:09+08:00 XA-QYZX-S12708-C FIB/1/OVLDFORWARD:Slot=1;OID 1.3.6.1.4.1.2011.5.25.129.2.9.3 The interface board is in the overload forwarding state because the FIB module is overloaded. (EntityPhysicalIndex=0, HwBaseTrapSeverity=1, HwBaseTrapProbableCause=1, HwBaseTrapEventType=4, HwFibOverloadModule=1, entPhysicalName=slot )
 





处理过程

1、检查交换机路由状态,发现两框1号单板均有路由超规格现象

板出现超限转发状态,正常状态为Normal state

====display fib overload state slot 1===============

  Overload mode:

    Overload forward mode.

  Overload state:

    Overload forward state


2、检查单板路由数,发现路由数已经达到规格最大值

===============display fib statistics all===============

IPv4 FIB Total Route Prefix Count : 13105; Entry Count : 13105

 

IPv4 FIB Public Route Prefix Count : 13105; Entry Count : 13105


  ===============display fib 1 statistics all===============

IPv4 FIB Route Prefix Capacity : 12288

IPv4 FIB Total Route Prefix Count : 12268; Entry Count : 12268

 

IPv4 FIB Public Route Prefix Count : 12268; Entry Count : 12268


3、 检查日志,发现有路由超规格告警

XA-QYZX-S12708-A告警:
#Jul 30 2016 05:30:43+08:00 XA-QYZX-S12708-A L3MB/4/FWDRESLACK:OID 1.3.6.1.4.1.2011.5.25.227.2.1.5 The layer 3 resource usage has reached or exceeded 85%.(EntPhysicalindex=67371017,EntPhysicalName=LPU Board 1,Slot=1,ResourceType=47)


#Jul 30 2016 05:30:43+08:00 XA-QYZX-S12708-A L3MB/4/FWDRESLACK:OID 1.3.6.1.4.1.2011.5.25.227.2.1.5 The layer 3 resource usage has reached or exceeded 85%.(EntPhysicalindex=67371017,EntPhysicalName=LPU Board 1,Slot=1,ResourceType=28)


#Jul 30 2016 05:30:41+08:00 XA-QYZX-S12708-A FIB/1/OVLDFORWARD:Slot=1;OID 1.3.6.1.4.1.2011.5.25.129.2.9.3 The interface board is in the overload forwarding state because the FIB module is overloaded. (EntityPhysicalIndex=0, HwBaseTrapSeverity=1, HwBaseTrapProbableCause=1, HwBaseTrapEventType=4, HwFibOverloadModule=1, entPhysicalName=slot )


XA-QYZX-S12708-C告警:
#Jul 30 2016 05:38:13+08:00 XA-QYZX-S12708-C L3MB/4/FWDRESLACK:OID 1.3.6.1.4.1.2011.5.25.227.2.1.5 The layer 3 resource usage has reached or exceeded 85%.(EntPhysicalindex=67371017,EntPhysicalName=LPU Board 1,Slot=1,ResourceType=47)


#Jul 30 2016 05:38:13+08:00 XA-QYZX-S12708-C L3MB/4/FWDRESLACK:OID 1.3.6.1.4.1.2011.5.25.227.2.1.5 The layer 3 resource usage has reached or exceeded 85%.(EntPhysicalindex=67371017,EntPhysicalName=LPU Board 1,Slot=1,ResourceType=28)


#Jul 30 2016 05:38:09+08:00 XA-QYZX-S12708-C FIB/1/OVLDFORWARD:Slot=1;OID 1.3.6.1.4.1.2011.5.25.129.2.9.3 The interface board is in the overload forwarding state because the FIB module is overloaded. (EntityPhysicalIndex=0, HwBaseTrapSeverity=1, HwBaseTrapProbableCause=1, HwBaseTrapEventType=4, HwFibOverloadModule=1, entPhysicalName=slot )
 



4、模拟复现路由超规格现象,在超规格的情况下此时新upvlanif接口可以复现ping不通问题

由于fib超规格,会导致主机路由没有下发,这个时候报文没法命中主机路由上送,从而导致ping不通。



根因


  由于fib超规格,在vlanif up的时候会导致主机路由没有下发,这个时候报文没法命中主机路由上送,从而导致ping不通。由于现网动态路由会存在刷新的情况,如果此时有路由释放, 1号单板就会删除一部分释放的路由,然后通过将vlanif shutdownundo shutdown ,触发路由重新下发,路由重新下发成功后则可重新ping通。



解决方案

1、 通过修改对端发往S12708的路由策略,类似于其它区域中心的策略实现,过滤一部分其它区域的路由,保留本地的明细路由,这样可以缩减路由表至3k以内,不会超出单板的12k规格

2、可以将该单板配置为大路由模式(两个框的该单板均需要配置)  

配置方式:
[XA-QYZX-S12708-A]assign resource-mode slot 1 mode enhanced-ipv4
此时1号单板的ipv4路由规格由13K扩展为128K,后续如果有ipv6业务需要,则可以配置为ipv4-ipv6模式,ipv4-ipv6模式单板可分配64K IPv4 + 10K IPv6。

注意点:
1) 修改为大路由模式后,单板的mac地址表容量会从98K降低到32K,这个需要注意一下,如果现网此单板下有超过32k(3万台)主机的需求,可能会存在风险,如果主机数目小于32k,则无风险;
2) 配置完成后,需要重启对应单板才可以生效,重启之后使用如下命令检查配置是否已经生效
重启命令:reset slot 1
                                                                                                                                   
检查命令,如下状态为生效状态:
[XA-QYZX-S12708-A]display resource-assign configuration                                                                            
Resource assign status:                                                                                                            
Slot  Cur-Resmode     Next-Resmode    Cur-Aclmode          Next-Aclmode                                                            
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -                                                    
  1   enhanced-ipv4   enhanced-ipv4   --                   --    
                                                                 
3)两框的ET1D2X48SEC0单板均已经超规格,因此两框的单板均需要配置
4)插在5、7槽位的ET1D2G48SEA0单板路由规格也已经快达到规格最大值(规格16K,目前也已经达到13K),要考虑后续业务增加带来的风险(此类型单板无法扩充路由表),需要减少对端设备通告过来的BGP路由数量

display fib 5 statistics all===============
==================================================================
IPv4 FIB Route Prefix Capacity : 16384
IPv4 FIB Total Route Prefix Count : 13098; Entry Count : 13098
IPv4 FIB Public Route Prefix Count : 13098; Entry Count : 13098

===============display fib 7 statistics all===============
==================================================================
IPv4 FIB Route Prefix Capacity : 16384
IPv4 FIB Total Route Prefix Count : 13098; Entry Count : 13098
IPv4 FIB Public Route Prefix Count : 13098; Entry Count : 13098







建议与总结

在大路由环境下添加路由策略,从而减少路由条目,也可以精简路由便于管理。

END