USG2200跟Cisco设备通过GRE建立OSPF邻居停留在Exchange状态问题分析

发布时间:  2015-09-28 浏览次数:  361 下载次数:  0
问题描述

组网:

USG2200 (OSPF)----- GRE Tunnel -----(OSPF) Cisco

故障现象:

使USG2200与对端Cisco 2600路由器通过GRE隧道建立OSPF建立邻居,但是OSPF邻居状态一直卡在Exchange 状态,没有达到Full的状态。本端tunnel地址是10.1.1.1,对端是10.1.1.4

<USG2210>display ospf peer 10.1.1.4

19:46:10  2014/11/17

         OSPF Process 1 with Router ID 10.8.1.250

                 Neighbors

Area 0.0.0.0 interface 10.1.1.1(Tunnel1)'s neighbors

Router ID: 10.1.1.4         Address: 10.1.1.4         GR State: Normal    

   State: Exchange  Mode:Nbr is  Slave  Priority: 0

   DR: 10.1.1.1  BDR: None   MTU: 1500

   Dead timer due in 28  sec

   Neighbor is up for 00:00:00    

   Authentication Sequence: [ 0 ]

处理过程

1、  检查USG2200 GRE,包过滤等配置没有发现问题,同时通过ping确认对端可达。

[USG2210]ping 10.1.1.4

11:28:39  2014/11/18

  PING 10.1.1.4: 56  data bytes, press CTRL_C to break

    Reply from 10.1.1.4: bytes=56 Sequence=1 ttl=255 time=10 ms

    Reply from 10.1.1.4: bytes=56 Sequence=2 ttl=255 time=10 ms

    Reply from 10.1.1.4: bytes=56 Sequence=3 ttl=255 time=10 ms

    Reply from 10.1.1.4: bytes=56 Sequence=4 ttl=255 time=10 ms

  --- 10.1.1.4 ping statistics ---

    4 packet(s) transmitted

    4 packet(s) received

    0.00% packet loss

    round-trip min/avg/max = 10/10/10 ms

2、检查OSPF邻居状态,一直处于Exchange,并且本端MTU1500

[USG2210]display ospf peer 10.1.1.4

11:34:52  2014/11/18

          OSPF Process 1 with Router ID 10.8.1.250

                    Neighbors

 Area 0.0.0.0 interface 10.1.1.1(Tunnel1)'s neighbors

 Router ID: 10.1.1.4         Address: 10.1.1.4         GR State: Normal    

   State: Exchange  Mode:Nbr is  Slave  Priority: 0

   DR: 10.1.1.1  BDR: None   MTU: 1500

   Dead timer due in 34  sec

   Neighbor is up for 00:00:00    

   Authentication Sequence: [ 0 ]

3、检查本端和对端的MTU值。本端tunnel接口默认配置,MTU1500,对端tunnel接口配置了ip mtu 1500,从show ip interface tunnel 1来看,MTU也是1500,两端已经保持一致了,按理说就没有问题了。

本端配置:

#

interface Tunnel1

 alias Tunnel1

 ip address 10.1.1.1 255.255.255.248

 tunnel-protocol gre p2mp

 source Vlanif100

 gre key 39021371

 gre checksum

 nhrp authentication plain %$%$g5V.BKq<]E_mR1:N:|j"UXOF%$%$

 nhrp network-id 10

 nhrp entry holdtime second 300

 nhrp entry multicast dynamic

 nhrp server

 nhrp redirect

 ospf network-type broadcast

 ospf dr-priority 255

#

[USG2210] display interface Tunnel  1

14:53:13  2014/11/18

Tunnel1 current state : UP  

Line protocol current state : UP

Tunnel1 current firewall zone : untrust3

Description : Huawei, USG2200 Series, Tunnel1 Interface, Route Port

The Maximum Transmit Unit is 1500 bytes

Internet Address is 10.1.1.1/29

Encapsulation is TUNNEL, loopback not set

Tunnel source Vlanif100

Tunnel protocol/transport P2MP-GRE/IP, key 39021371

Checksumming of packets enabledQoS max-bandwidth : 100000 Kbps

Output queue : (Urgent queue : Size/Length/Discards)  0/50/0

Output queue : (Frag queue : Size/Length/Discards)  0/1000/0

Output queue : (Protocol queue : Size/Length/Discards) 0/1000/0

Output queue : (FIFO queue : Size/Length/Discards)  0/256/0

    Last 300 seconds input rate 0 bits/s, 0 packets/s

    Last 300 seconds output rate 0 bits/s, 0 packets/s

    7414175 packets input, 1533881659 bytes

    0 input error

    7992991 packets output, 3387912302 bytes

    0 output error

对端配置:

!

interface Tunnel1

 bandwidth 1000

 ip address 10.1.1.4 255.255.255.248

 ip broadcast-address 10.1.1.7

 no ip redirects

 ip mtu 1500

 ip nhrp authentication gao@123

 ip nhrp map multicast 124.172.122.97

 ip nhrp map 10.1.1.1 124.172.122.97

 ip nhrp network-id 10

 ip nhrp holdtime 300

 ip nhrp nhs 10.1.1.1

 ip tcp adjust-mss 1360

 ip ospf network broadcast

 ip ospf priority 0

 ip ospf mtu-ignore

 

C2621-2# show ip interface tunnel 1

Tunnel1 is up, line protocol is up

  Internet address is 10.1.1.4/29

  Broadcast address is 10.1.1.7

  Address determined by non-volatile memory

  MTU is 1500 bytes

  Helper address is not set

  Directed broadcast forwarding is disabled

  Multicast reserved groups joined: 224.0.0.5

  Outgoing access list is not set

  Inbound  access list is not set

  Proxy ARP is enabled

  Local Proxy ARP is disabled

  Security level is default

  Split horizon is enabled

  ICMP redirects are never sent

  ICMP unreachables are always sent

  ICMP mask replies are never sent

  IP fast switching is disabled

  IP fast switching on the same interface is disabled

  IP Flow switching is disabled

  IP CEF switching is disabled

  IP Null turbo vector

  IP multicast fast switching is disabled

IP multicast distributed fast switching is disabled

  IP route-cache flags are Fast, CEF

  Router Discovery is disabled

  IP output packet accounting is disabled

  IP access violation accounting is disabled

  TCP/IP header compression is disabled

  RTP/IP header compression is disabled

  Policy routing is disabled

  Network address translation is disabled

  BGP Policy Mapping is disabled

  WCCP Redirect outbound is disabled

  WCCP Redirect inbound is disabled

  WCCP Redirect exclude is disabled

4OSPF错误统计,rest ospf进程后从新观察错误统计,看到有少量的错误报文,但是具体什么

原因导致也看出来。

[USG2210] display ospf error 

11:32:48  2014/11/18

          OSPF Process 1 with Router ID 10.8.1.250

                    OSPF error statistics

General packet errors:

 0     : IP: received my own packet     0     : Bad packet

 0     : Bad version                    0     : Bad checksum

 0     : Bad area id                    0     : Drop on unnumbered interface

 0     : Bad virtual link               0     : Bad authentication type

 0     : Bad authentication key         0     : Packet too small

 0     : Packet size > ip length        0     : Transmit error

 0     : Interface down                 2     : Unknown neighbor

HELLO packet errors:

 0     : Netmask mismatch               0     : Hello timer mismatch

 0     : Dead timer mismatch            0     : Extern option mismatch

 0     : Router id confusion            0     : Virtual neighbor unknown

 0     : NBMA neighbor unknown

DD packet errors:

 1     : Neighbor state low             0     : Router id confusion

 0     : Extern option mismatch         0     : Unknown LSA type

 0     : MTU option mismatch

                                         

LS ACK packet errors:

 1     : Neighbor state low             2     : Bad ack

 15    : Duplicate ack                  0     : Unknown LSA type

LS REQ packet errors:

 0     : Neighbor state low             0     : Empty request

 1     : Bad request

LS UPD packet errors:

 0     : Neighbor state low             0     : Newer self-generate LSA

 0     : LSA checksum bad               1     : Received less recent LSA

 0     : Unknown LSA type

Opaque errors:

 0     : 9-out of flooding scope        0     : 10-out of flooding scope

 0     : 11-out of flooding scope

Retransmission for packet over Limitation errors:

 0     : Number for DD Packet           0     : Number for Update Packet

 0     : Number for Request Packet

Configuration errors:

0                 : Tunnel cost mistake

5、  debug ospf,观察两端ospf邻居建立过程。

本端开debug ospf event看,只有中间几个状态切换,切换到Exchange后就没有任何输出了,看不出具体问题。

<USG2210>debugging ospf event  

<USG2210>t d

<USG2210>t m

<USG2210>reset ospf process

*3.832197092 USG2210 RM/7/RMDEBUG:

         OSPF 1: Nbr 10.1.1.4 Rcv HelloReceived State Down -> Init.  

*3.832197092 USG2210 RM/7/RMDEBUG:

         OSPF 1: Nbr 10.1.1.4 Rcv 2WayReceived State Init -> 2Way.

*3.832233422 USG2210 RM/7/RMDEBUG:

         OSPF 1: Nbr 10.1.1.4 Rcv AdjOk? State 2Way -> ExStart.  

*3.832233522 USG2210 RM/7/RMDEBUG:

  OSPF 1: Nbr 10.1.1.4 Rcv NegotiationDone State ExStart -> Exchange. 

 

对端开debug ip ospf events,有完整的debug输出,但是还是看不出什么原因。

C2621-2#debug  ip ospf events

C2621-2#terminal monitor  

C2621-2#clear ip ospf process                                                 

Reset ALL OSPF processes? [no]: yes                                             

C2621-2#                                                                       

Nov 18 07:50:21.336: OSPF: Send hello to 224.0.0.5 area 0 on Tunnel1 from 10.1.1

.4                                                                             

Nov 18 07:50:21.340: OSPF: Send hello to 224.0.0.5 area 0 on FastEthernet0/0 fro

m 172.16.44.1                                                                  

Nov 18 07:50:22.064: OSPF: Flushing External Links                             

Nov 18 07:50:22.064: OSPF: Flushing Opaque AS Links                            

Nov 18 07:50:22.100: OSPF: Flushing Link states in area 0                      

Nov 18 07:50:22.136: OSPF: Interface Tunnel1 going Down                        

Nov 18 07:50:22.136: OSPF: Neighbor change Event on interface Tunnel1          

Nov 18 07:50:22.136: OSPF: DR/BDR election on Tunnel1                          

Nov 18 07:50:22.136: OSPF: Elect BDR 0.0.0.0                                   

Nov 18 07:50:22.136: OSPF: Elect DR 10.8.1.250                                 

Nov 18 07:50:22.136:        DR: 10.8.1.250 (Id)   BDR: none                    

Nov 18 07:50:22.136: %OSPF-5-ADJCHG: Process 1, Nbr 10.8.1.250 on Tunnel1 from E

XCHANGE to DOWN, Neighbor Down: Interface down or detached                     

Nov 18 07:50:22.140: OSPF: Neighbor change Event on interface Tunnel1          

Nov 18 07:50:22.140: OSPF: DR/BDR election on Tunnel1                          

Nov 18 07:50:22.140: OSPF: Elect BDR 0.0.0.0                                    

Nov 18 07:50:22.140: OSPF: Elect DR 0.0.0.0                                    

Nov 18 07:50:22.140:        DR: none    BDR: none                              

Nov 18 07:50:22.140: OSPF: Remember old DR 10.8.1.250 (id)                     

Nov 18 07:50:22.144: OSPF: Interface FastEthernet0/0 going Down                

Nov 18 07:50:22.144: OSPF: Neighbor change Event on interface FastEthernet0/0  

Nov 18 07:50:22.144: OSPF: DR/BDR election on FastEthernet0/0                  

Nov 18 07:50:22.144: OSPF: Elect BDR 0.0.0.0                                   

Nov 18 07:50:22.144: OSPF: Elect DR 0.0.0.0                                    

Nov 18 07:50:22.144: OSPF: Elect BDR 0.0.0.0                                   

C2621-2#                                                                        

Nov 18 07:50:22.144: OSPF: Elect DR 0.0.0.0                                    

Nov 18 07:50:22.148:        DR: none    BDR: none                              

Nov 18 07:50:22.148: OSPF: Flush network LSA immediately                       

Nov 18 07:50:22.148: OSPF: Remember old DR 10.1.1.4 (id)                       

Nov 18 07:50:22.200: OSPF: Interface Tunnel1 going Up                          

Nov 18 07:50:22.200: OSPF: Send hello to 224.0.0.5 area 0 on Tunnel1 from 10.1.1

.4                                                                             

Nov 18 07:50:22.200: OSPF: Interface FastEthernet0/0 going Up                  

Nov 18 07:50:22.200: OSPF: Send hello to 224.0.0.5 area 0 on FastEthernet0/0 fro

m 172.16.44.1                                                                  

Nov 18 07:50:22.220: OSPF: Rcv pkt from Tunnel1 src 10.1.1.1 dst 224.0.0.5 id 10

.8.1.250 type 4 if_state 2 : ignored due to unknown neighbor                    

Nov 18 07:50:23.568: OSPF: Rcv hello from 10.8.1.250 area 0 from Tunnel1 10.1.1.

1                                                                              

Nov 18 07:50:23.572: OSPF: 2 Way Communication to 10.8.1.250 on Tunnel1, state 2

WAY                                                                             

Nov 18 07:50:23.572: OSPF: Backup seen Event before WAIT timer on Tunnel1      

Nov 18 07:50:23.572: OSPF: DR/BDR election on Tunnel1                          

Nov 18 07:50:23.572: OSPF: Elect BDR 0.0.0.0                                   

Nov 18 07:50:23.572: OSPF: Elect DR 10.8.1.250                                 

Nov 18 07:50:23.572:        DR: 10.8.1.250 (Id)   BDR: none                    

Nov 18 07:50:23.576: OSPF: Send DBD to 10.8.1.250 on Tunnel1 seq 0x1585 opt 0x52

 flag 0x7 len 32                                                               

Nov 18 07:50:23.576: OSPF: Send immediate hello to nbr 10.8.1.250, src address 1

0.1.1.1, on Tunnel1                                                             

C2621-2#                                                                       

Nov 18 07:50:23.576: OSPF: Send hello to 10.1.1.1 area 0 on Tunnel1 from 10.1.1.

4                                                                               

Nov 18 07:50:23.576: OSPF: End of hello processing                             

Nov 18 07:50:23.592: OSPF: Rcv DBD from 10.8.1.250 on Tunnel1 seq 0xD15340 opt 0

x2 flag 0x7 len 32  mtu 0 state EXSTART                                         

Nov 18 07:50:23.592: OSPF: NBR Negotiation Done. We are the SLAVE              

Nov 18 07:50:23.592: OSPF: Send DBD to 10.8.1.250 on Tunnel1 seq 0xD15340 opt 0x

52 flag 0x2 len 52                                                             

C2621-2#                                                                       

Nov 18 07:50:27.908: OSPF: Rcv LS UPD from 10.8.1.250 on Tunnel1 length 112 LSA

count 2                                                                        

Nov 18 07:50:28.400: OSPF: Rcv LS UPD from 10.8.1.250 on Tunnel1 length 76 LSA c

ount 1                                                                         

C2621-2#                                                                       

Nov 18 07:50:29.092: OSPF: Rcv LS UPD from 10.8.1.250 on Tunnel1 length 76 LSA c

ount 1                                                                         

C2621-2#                                                                       

Nov 18 07:50:32.200: OSPF: Send hello to 224.0.0.5 area 0 on Tunnel1 from 10.1.1

.4                                                                             

Nov 18 07:50:32.204: OSPF: Send hello to 224.0.0.5 area 0 on FastEthernet0/0 fro

m 172.16.44.1                                                                   

Nov 18 07:50:32.692: OSPF: Rcv hello from 10.8.1.250 area 0 from Tunnel1 10.1.1.

1                                                                              

Nov 18 07:50:32.692: OSPF: End of hello processing                              

C2621-2#                                                                       

Nov 18 07:50:41.808: OSPF: Rcv hello from 10.8.1.250 area 0 from Tunnel1 10.1.1.

1                                                                              

Nov 18 07:50:41.808: OSPF: End of hello processing                             

Nov 18 07:50:42.200: OSPF: Send hello to 224.0.0.5 area 0 on Tunnel1 from 10.1.1

.4                                                                             

Nov 18 07:50:42.204: OSPF: Send hello to 224.0.0.5 area 0 on FastEthernet0/0 fro

m 172.16.44.1                                                                  

C2621-2#                                                                       

Nov 18 07:50:50.932: OSPF: Rcv hello from 10.8.1.250 area 0 from Tunnel1 10.1.1.

1                                                                              

Nov 18 07:50:50.932: OSPF: End of hello processing                             

C2621-2#                                                                        

Nov 18 07:50:52.200: OSPF: Send hello to 224.0.0.5 area 0 on Tunnel1 from 10.1.1

.4                                                                             

Nov 18 07:50:52.204: OSPF: Send hello to 224.0.0.5 area 0 on FastEthernet0/0 fro

m 172.16.44.1                                                                  

C2621-2#                                                                       

Nov 18 07:51:00.048: OSPF: Rcv hello from 10.8.1.250 area 0 from Tunnel1 10.1.1.

1                                                                              

Nov 18 07:51:00.052: OSPF: End of hello processing                             

C2621-2#                                                                       

Nov 18 07:51:02.200: OSPF: Send hello to 224.0.0.5 area 0 on Tunnel1 from 10.1.1

.4                                                                             

Nov 18 07:51:02.200: OSPF: end of Wait on interface FastEthernet0/0            

Nov 18 07:51:02.200: OSPF: DR/BDR election on FastEthernet0/0                  

Nov 18 07:51:02.200: OSPF: Elect BDR 10.1.1.4                                  

Nov 18 07:51:02.204: OSPF: Elect DR 10.1.1.4                                   

Nov 18 07:51:02.204: OSPF: Elect BDR 0.0.0.0                                   

Nov 18 07:51:02.204: OSPF: Elect DR 10.1.1.4                                   

Nov 18 07:51:02.204:        DR: 10.1.1.4 (Id)   BDR: none                      

Nov 18 07:51:02.204: OSPF: Send hello to 224.0.0.5 area 0 on FastEthernet0/0 fro

m 172.16.44.1                                                                  

C2621-2#                                                                       

Nov 18 07:51:09.168: OSPF: Rcv hello from 10.8.1.250 area 0 from Tunnel1 10.1.1.

1                                                                              

Nov 18 07:51:09.172: OSPF: End of hello processing                             

C2621-2#                                                                        

Nov 18 07:51:12.200: OSPF: Send hello to 224.0.0.5 area 0 on Tunnel1 from 10.1.1

.4                                                                             

Nov 18 07:51:12.204: OSPF: Send hello to 224.0.0.5 area 0 on FastEthernet0/0 fro

m 172.16.44.1                                                                  

C2621-2#  

6、在没有其他思路的情况下,查看了这个tunnel接口下有另外两个邻居状态时Full的,并且MTU1468,跟客户确认了下这两个对端设备同样是Cisco的。寻思是不是还是MTU的问题?

[USG2210]display ospf peer Tunnel  1

15:14:25  2014/11/18

          OSPF Process 1 with Router ID 10.8.1.250

                    Neighbors

 Area 0.0.0.0 interface 10.1.1.1(Tunnel1)'s neighbors

 Router ID: 172.16.33.1      Address: 10.1.1.2         GR State: Normal    

   State: Full  Mode:Nbr is  Master  Priority: 0

   DR: 10.1.1.1  BDR: None   MTU: 1468

   Dead timer due in 30  sec

   Neighbor is up for 00:01:50    

   Authentication Sequence: [ 0 ]

 Router ID: 10.1.1.3         Address: 10.1.1.3         GR State: Normal    

   State: Full  Mode:Nbr is  Slave  Priority: 0

   DR: 10.1.1.1  BDR: None   MTU: 1468

   Dead timer due in 38  sec

   Neighbor is up for 00:01:46    

   Authentication Sequence: [ 0 ]

 Router ID: 10.1.1.4         Address: 10.1.1.4         GR State: Normal    

   State: Exchange  Mode:Nbr is  Slave  Priority: 0

   DR: 10.1.1.1  BDR: None   MTU: 1500

   Dead timer due in 35  sec

   Neighbor is up for 00:00:00    

   Authentication Sequence: [ 0 ]

7、在Cisco设备tunnel接口上去掉了ip mtu的配置,再看它的MTU就变成了1468,在这种情况下,把本端tunnel接口的MTU修改成mtu 1468,OSPF邻居状态就变成Full了。所以问题的根本原因还是MTU有问题。

C2621-2(config-if)#no ip mtu 

C2621-2# show ip interface tunnel 1

Tunnel1 is up, line protocol is up

  Internet address is 10.1.1.4/29

  Broadcast address is 10.1.1.7

  Address determined by non-volatile memory

  MTU is 1468 bytes

  Helper address is not set

  Directed broadcast forwarding is disabled

  Multicast reserved groups joined: 224.0.0.5

  Outgoing access list is not set

  Inbound  access list is not set

  Proxy ARP is enabled

  Local Proxy ARP is disabled

  Security level is default

  Split horizon is enabled

  ICMP redirects are never sent

  ICMP unreachables are always sent

  ICMP mask replies are never sent

  IP fast switching is disabled

  IP fast switching on the same interface is disabled

  IP Flow switching is disabled

  IP CEF switching is disabled

  IP Null turbo vector

  IP multicast fast switching is disabled

  IP multicast distributed fast switching is disabled

  IP route-cache flags are Fast, CEF

  Router Discovery is disabled

  IP output packet accounting is disabled

  IP access violation accounting is disabled

  TCP/IP header compression is disabled

  RTP/IP header compression is disabled

  Policy routing is disabled

  Network address translation is disabled

  BGP Policy Mapping is disabled

  WCCP Redirect outbound is disabled

  WCCP Redirect inbound is disabled

  WCCP Redirect exclude is disabled

C2621-2#

[USG2210-Tunnel1]mtu 1468

[USG2210]display interface Tunnel 1

17:31:42  2014/11/18

Tunnel1 current state : UP  

Line protocol current state : UP

Tunnel1 current firewall zone : untrust3

Description : Huawei, USG2200 Series, Tunnel1 Interface, Route Port

The Maximum Transmit Unit is 1468 bytes

Internet Address is 10.1.1.1/29

Encapsulation is TUNNEL, loopback not set

Tunnel source Vlanif100

Tunnel protocol/transport P2MP-GRE/IP, key 39021371

Checksumming of packets enabledQoS max-bandwidth : 100000 Kbps

Output queue : (Urgent queue : Size/Length/Discards)  0/50/0

Output queue : (Frag queue : Size/Length/Discards)  0/1000/0

Output queue : (Protocol queue : Size/Length/Discards) 0/1000/0

Output queue : (FIFO queue : Size/Length/Discards)  0/256/0

    Last 300 seconds input rate 0 bits/s, 0 packets/s

    Last 300 seconds output rate 0 bits/s, 0 packets/s

    7798259 packets input, 1612082521 bytes

    0 input error

    8384748 packets output, 3594825506 bytes

0 output error

[USG2210]display ospf peer Tunnel  1

17:34:01  2014/11/18

......

 Router ID: 10.1.1.4         Address: 10.1.1.4         GR State: Normal    

   State: Full  Mode:Nbr is  Slave  Priority: 0

   DR: 10.1.1.1  BDR: None   MTU: 1468

   Dead timer due in 35  sec

   Neighbor is up for 00:02:23    

   Authentication Sequence: [ 0 ]

根因
两端MTU不一致,导致OSPF邻居无法建立。
解决方案
将本端tunnel接口的MTU修改成mtu 1468后问题解决。
建议与总结

1、  OSPF邻居状态异常的问题,常规定位步骤就是先看邻居之间是否可达(包括路由,包过滤等),

然后看OSPF错误统计是否有明显的比如Router ID冲突等的错误,再是开debug ospf看能否看出原因。

2、  OSPF邻居卡在Exchange 状态的问题,还是优先检查MTU的情况。

3、  CiscoOSPF debug命令:

C2621-2#debug  ip ospf events

C2621-2#terminal monitor  

C2621-2#clear ip ospf process

END