Service Forwarding Is Interrupted When OSPF Multi-Process and BGP Import Routes from Each Other on an S9700 Switch

Publication Date:  2015-12-22 Views:  437 Downloads:  0
Issue Description
[Version information]

S9700 V200R003C00SPC500 V200R003SPH012

[Networking description]



In the preceding figure, LSW1 (S9700) functions as an MCE, while R1 (NE40E-X3) and R2 (NE40E-X3) function as PEs of the MPLS/IBGP VPN network.

Two firewalls FW1 (USG6650) and FW2 (USG6650) work in transparent mode and transparently transmit all Layer 2 services.

R1, R2, FW1, FW2, and LSW1 are managed through VLANIF 100.

LSW1 has two OSPF processes OSPF 101 and OSPF 102 configured, which are bound to two VPN instances. The two VPN instances communicate through BGP. LSW1 establishes two OSPF neighbor relationships with R1 through VLANIF 4 and VLANIF 6 and establishes two OSPF neighbor relationships with R2 through VLANIF 5 and VLANIF 7.

[Fault description]

ClIENT1 cannot communicate with CLIENT2.

[Configuration script]

LSW1:

#
router id 192.168.100.39
#
ip vpn-instance Both_Internet&Internal_Service
ipv4-family
  route-distinguisher 65013:3
  vpn-target 300:1 export-extcommunity
  vpn-target 100:1 200:1 300:1 import-extcommunity
#
ip vpn-instance Internal_Service
ipv4-family
  route-distinguisher 65013:2
  vpn-target 200:1 export-extcommunity
  vpn-target 200:1 300:1 import-extcommunity
#
interface Vlanif4
ip binding vpn-instance Internal_Service
ip address 10.20.64.9 255.255.255.252 
#
interface Vlanif5
ip binding vpn-instance Internal_Service
ip address 10.20.64.13 255.255.255.252 
#
interface Vlanif6
ip binding vpn-instance Both_Internet&Internal_Service
ip address 10.20.64.17 255.255.255.252 
#
interface Vlanif7
ip binding vpn-instance Both_Internet&Internal_Service
ip address 10.20.64.21 255.255.255.252 
#
interface Vlanif9
ip binding vpn-instance Both_Internet&Internal_Service
ip address 10.20.64.129 255.255.255.192 
#
interface Vlanif100
ip binding vpn-instance Internal_Service
ip address 172.23.16.1 255.255.255.0 
#
interface GigabitEthernet0/0/1
port link-type trunk
undo port trunk allow-pass vlan 1
port trunk allow-pass vlan 4 6 100
#
interface GigabitEthernet0/0/2
port link-type trunk
undo port trunk allow-pass vlan 1
port trunk allow-pass vlan 5 7 100
#
interface GigabitEthernet0/0/23
port link-type access
port default vlan 9
#
interface GigabitEthernet0/0/24
port link-type access
port default vlan 100
#
interface LoopBack0
ip address 192.168.100.39 255.255.255.255 
#
bgp 65013
#
ipv4-family unicast
  undo synchronization
#
ipv4-family vpn-instance Both_Internet&Internal_Service 
  import-route direct
#
ipv4-family vpn-instance Internal_Service 
  import-route direct
#
ospf 101 router-id 192.168.100.39 vpn-instance Internal_Service
import-route direct
import-route static
vpn-instance-capability simple
area 0.0.0.0 
  network 10.20.64.8 0.0.0.3 
  network 10.20.64.12 0.0.0.3 
  network 192.168.200.39 0.0.0.0 
#
ospf 102 router-id 192.168.100.39 vpn-instance Both_Internet&Internal_Service
import-route direct
import-route static
vpn-instance-capability simple
area 0.0.0.0 
  network 10.20.64.16 0.0.0.3 
  network 10.20.64.20 0.0.0.3 
#

R1:

#
router id 192.168.100.37
#
ip vpn-instance Both_Internet&Internal_Service
ipv4-family
  route-distinguisher 65000:75
  vpn-target 300:1 export-extcommunity
  vpn-target 100:1 200:1 300:1 import-extcommunity
#
ip vpn-instance Internal_Service
ipv4-family
  route-distinguisher 65000:74
  vpn-target 200:1 export-extcommunity
  vpn-target 200:1 300:1 import-extcommunity
#
ip vpn-instance Internet
ipv4-family
  route-distinguisher 65000:73
  vpn-target 100:1 export-extcommunity
  vpn-target 100:1 300:1 import-extcommunity
#
mpls lsr-id 192.168.100.37
mpls
#
mpls ldp
#
interface GigabitEthernet0/0/0
ip address 10.20.64.25 255.255.255.252
mpls
mpls ldp
#
interface GigabitEthernet0/0/1.4
vlan-type dot1q 4
ip binding vpn-instance Internal_Service
ip address 10.20.64.10 255.255.255.252
#
interface GigabitEthernet0/0/1.6
vlan-type dot1q 6
ip binding vpn-instance Both_Internet&Internal_Service
ip address 10.20.64.18 255.255.255.252
#
interface GigabitEthernet0/0/1.100
vlan-type dot1q 100
ip binding vpn-instance Internal_Servicef
ip address 172.23.16.2 255.255.255.0
#
interface LoopBack0
ip address 192.168.100.37 255.255.255.255
#
bgp 65000
peer 192.168.100.38 as-number 65000
peer 192.168.100.38 connect-interface LoopBack0
#
ipv4-family unicast
  undo synchronization
  peer 192.168.100.38 enable 

ipv4-family vpnv4
  policy vpn-target
  peer 192.168.100.38 enable 

ipv4-family vpn-instance Both_Internet&Internal_Service 
  import-route static 
  import-route ospf 102 

ipv4-family vpn-instance Internal_Service 
  import-route static 
  import-route ospf 101

ospf 1 router-id 192.168.100.37 
area 0.0.0.0 
  network 10.20.64.24 0.0.0.3 
  network 192.168.100.37 0.0.0.0 

ospf 101 router-id 192.168.100.37 vpn-instance Internal_Service 
import-route static 
import-route bgp 
area 0.0.0.0 
  network 10.20.64.8 0.0.0.3 
  network 192.168.200.37 0.0.0.0 
#
ospf 102 router-id 192.168.100.37 vpn-instance Both_Internet&Internal_Service
import-route static 
import-route bgp 
area 0.0.0.0 
  network 10.20.64.16 0.0.0.3 
#

R2:

#
router id 192.168.100.38
#
ip vpn-instance Both_Internet&Internal_Service
ipv4-family
  route-distinguisher 65000:78
  vpn-target 300:1 export-extcommunity
  vpn-target 100:1 200:1 300:1 import-extcommunity
#
ip vpn-instance Internal_Service
ipv4-family
  route-distinguisher 65000:77
  vpn-target 200:1 export-extcommunity
  vpn-target 200:1 300:1 import-extcommunity
#
ip vpn-instance Internet
ipv4-family
  route-distinguisher 65000:76
  vpn-target 100:1 export-extcommunity
  vpn-target 100:1 300:1 import-extcommunity
#
mpls lsr-id 192.168.100.38
mpls
#
mpls ldp
#
interface GigabitEthernet0/0/0
ip address 10.20.64.26 255.255.255.252
mpls
mpls ldp
#
interface GigabitEthernet0/0/2.5
vlan-type dot1q 5
ip binding vpn-instance Internal_Service
ip address 10.20.64.14 255.255.255.252
#
interface GigabitEthernet0/0/2.7
vlan-type dot1q 7
ip binding vpn-instance Both_Internet&Internal_Service
ip address 10.20.64.22 255.255.255.252
#
interface GigabitEthernet0/0/2.100
vlan-type dot1q 100
ip binding vpn-instance Internal_Service
ip address 172.23.16.3 255.255.255.0
#
interface LoopBack0
ip address 192.168.100.38 255.255.255.255
#
bgp 65000
peer 192.168.100.37 as-number 65000
peer 192.168.100.37 connect-interface LoopBack0
#
ipv4-family unicast
  undo synchronization
  peer 192.168.100.37 enable
#
ipv4-family vpnv4
  policy vpn-target
  peer 192.168.100.37 enable
#
ipv4-family vpn-instance Both_Internet&Internal_Service
  import-route static
  import-route ospf 102
#
ipv4-family vpn-instance Internal_Service
  import-route static
  import-route ospf 101
#
#
ospf 1 router-id 192.168.100.38
area 0.0.0.0
  network 10.20.64.24 0.0.0.3
  network 192.168.100.38 0.0.0.0
#
ospf 101 router-id 192.168.100.38 vpn-instance Internal_Service
import-route static
import-route bgp
area 0.0.0.0
  network 10.20.64.12 0.0.0.3
  network 192.168.100.38 0.0.0.0
#
ospf 102 router-id 192.168.100.38 vpn-instance Both_Internet&Internal_Service
import-route static
import-route bgp
area 0.0.0.0 
network 10.20.64.20 0.0.0.3
#
Handling Process
1. Analyze routes.

Figure 1 shows that VLANIF 9 and VLANIF 100 communicate across a VPN, and LSW1 receives packets sent to network segment 172.23.16.0 through VLANIF 9.

Figure 1 Inter-VPN communication between VLANIF 9 and VLANIF 10



Figure 2 shows the IP routing table of LSW1. According to the next hop of the route, the packets should be directly forwarded to CLIENT2.



Figure 3 Directly forwarding packets to CLIENT2



However, ICMP response packets are load balanced through R1 and R2, as shown in Figure4.



Figure 5Next hops of ICMP response packets



FW1 and FW2 are stateful firewalls and will discard received ICMP response packets if they do not receive any ICMP request packets, regardless of whether interzone policies allow the packets to pass through, as shown in Figure 6.

Figure 6Discarding ICMP response packets



This situation is a typical example of incorrect packet discarding by firewalls because of inconsistent forward and reverse paths.

2. Analyze the reason for inconsistent forward and reverse paths.

Observe the line in red in Figure 6.Packets sent from CLIENT2 to CLIENT1 should be directly forwarded to CLIENT1 by LSW1; however, LSW1 forwards the packets to R1 and R2 according to a different route.

If you examine the IP routing table of LSW1, you will find that this route is a Type 5 route learned by OSPF 102.



During intra-VPN communication, local route crossing is implemented through BGP. The BGP route preference is 255, while the OSPF Type 5 route has a higher preference of 150. LSW1 incorrectly learned the OSPF Type 5 route and so used this route to replace the BGP local cross route.

To verify this situation, analyze why LSW1 obtains the OSPF Type 5 route.

a. LSW1 sends the direct route of VLANIF 9 to R1 through OSPF 102, as indicated by the line in blue in Figure 7.




b. Check the route import configuration on R1. Here, only the problem-involved configuration is displayed.

R1:

#
BGP 65000
ipv4-family vpn-instance Both_Internet&Internal_Service
  import-route static
  import-route ospf 102
#
ip vpn-instance Both_Internet&Internal_Service
ipv4-family
  route-distinguisher 65000:75
  vpn-target 300:1 export-extcommunity
  vpn-target 100:1 200:1 300:1 import-extcommunity
#
ip vpn-instance Internal_Service
ipv4-family
  route-distinguisher 65000:74
  vpn-target 200:1 export-extcommunity
  vpn-target 200:1 300:1 import-extcommunity
#
ospf 101 router-id 192.168.100.37 vpn-instance Internal_Service
import-route bgp
#

Figure 8shows packet processing on R1.

Phase 1: R1 imports the route learned by OSPF 102 into the BGP instance Both_Internet&Internal_Service.
Phase 2: The learned route is delivered to the BGP VPN instance Internal_Service when BGP VPN local route crossing occurs on R1, because the two VPN instances Both_Internet&Internal_Service and Internal_Service communicate through BGP.
Phase 3: The BGP routing table is imported into OSPF 101 VPN-Instance Internal_Service, and the BGP VPN route is delivered to the OSPF 101 routing table and marked as a Type 5 LSA external route.

Figure8Packet processing on R1



To verify the processing in phase 3, check the OSPF 101 LSDB on R1.



In the preceding figure, R1 has advertised the route 192.168.100.37, and R2 has advertised 192.168.100.38. R2 packet processing is the same as R1 packet processing and is not mentioned here.

c. In Figure 9 R1 has advertised the BGP route to LSW1 through OSPF 101. Therefore, a Type 5 LSA external route learned by OSPF 101 is generated on LSW1. This route has a higher priority than the BGP VPN local cross route. Subsequently, only this route is delivered to the FIB table. As a result, the problem occurs.

Figure 9Advertising routes from R1 to LSW1 through OSPF 101



3. Analyze why LSW1 sends packets received from Client1 to Client2 directly, as shown in Figure 10.

Figure 10 Sending packets by LSW1 to Client2



a. Check whether R1 does not import the OSPF-learned route into BGP.

As shown in Figure 11, the problem occurs in phase 1. VLAN 100 is used for network device management, and so each network device has a logical interface corresponding to this network segment.

Figure 11 Phase in which the problem occurs



interface GigabitEthernet0/0/1.100
vlan-type dot1q 100
ip binding vpn-instance Internal_Servicef
ip address 172.23.16.2 255.255.255.0
#

The following rule applies: Routes imported between routing protocols are preferred routes.

On R1, the preferred route for VPN instance Internal_Service is direct route, as shown in the following IP routing table.



In this situation, the OSPF-learned route is not the preferred route and is not imported into BGP. You can run the following command on R1 to confirm this.



b. Test whether the OSPF-learned route is imported into BGP.

Shut down the logical interface 0/0/1.100 on R1.



Run the following command on R1 to check whether the OSPF-learned route is imported. The information in the red rectangle shows that this route has been imported into BGP.



On LSW1, check the route for traffic from Client1 to Client2. The problem occurs on traffic from Client2 to Client1.

Root Cause
When BGP multi-VPN instance and OSPF multi-process import routes from each other, the forward and reverse paths are inconsistent because OSPF routes have higher priorities than BGP routes. Consequently, packets are discarded by firewalls.
Solution
1. When BGP imports an OSPF route, a community name is recorded, as shown in the following red rectangle.



Actually, OSPF is only expected to import the L3VPN routes learned through MP-IBGP but not local routes. Therefore, OSPF processes need to be prevented from learning routes from each other.

To meet this requirement, use routing policies to filter routes when OSPF imports BGP routes. Perform the following configuration on R1.



Perform the same configuration on R2.

2. After the preceding configurations are complete, check whether the problem on LSW1 is solved.





The preceding command output shows that OSPF 101 does not import the route to 10.20.64.128/26. This problem has been solved.
Suggestions
Devices on a network often have a large number of routes. When routing problems occur, analyze device forwarding behaviors, route delivery policies, and route priorities to solve these problems.

END