Example for Configuring E2E VXLAN to Implement DCI
End-to-end VXLAN tunnels can be configured to enable communication between VMs in different DCs.
Applicable Products and Versions
- This example applies to CE16800, CE12800, CE12800E, CE8800, CE7800, CE6800 (excluding the CE6820, CE6850EI, CE6810EI, and CE6810LI), and CE5880EI series switches running V200R002C50 or later versions.
- For details about the mapping between software versions and switch models, see the Hardware Query Tool.
Networking Requirements
As shown in Figure 2-45, an enterprise has VMs deployed in different data centers. VMa1 on Server 1 belongs to VLAN 10, and VMb2 on Server 2 belongs to VLAN 20. VMa1 on Server 1 and VMb2 on Server 2 reside on different network segments. To allow VMs in different data centers to communicate with each other, configure distributed VXLAN gateways. BGP EVPN is configured on Leaf 1 in DC A and Leaf 4 in DC B to create a VXLAN tunnel, so that VMa1 in DC A and VMb2 in DC B can communicate.
Device |
Interface |
IP Address |
Device |
Interface |
IP Address |
|---|---|---|---|---|---|
Device1 |
10GE1/0/1 |
192.168.50.1/24 |
Device2 |
10GE1/0/1 |
192.168.60.1/24 |
10GE1/0/2 |
192.168.1.1/24 |
10GE1/0/2 |
192.168.1.2/24 |
||
LoopBack0 |
1.1.1.1/32 |
LoopBack0 |
2.2.2.2/32 |
||
Spine1 |
10GE1/0/1 |
192.168.10.1/24 |
Spine2 |
10GE1/0/1 |
192.168.30.1/24 |
10GE1/0/2 |
192.168.20.1/24 |
10GE1/0/2 |
192.168.40.1/24 |
||
LoopBack0 |
3.3.3.3/32 |
LoopBack0 |
4.4.4.4/32 |
||
Leaf1 |
10GE1/0/1 |
192.168.10.2/24 |
Leaf4 |
10GE1/0/1 |
192.168.40.2/24 |
10GE1/0/2 |
- |
10GE1/0/2 |
- |
||
LoopBack0 |
5.5.5.5/32 |
LoopBack0 |
8.8.8.8/32 |
||
Leaf2 |
10GE1/0/1 |
192.168.20.2/24 |
Leaf3 |
10GE1/0/1 |
192.168.30.2/24 |
10GE1/0/3 |
192.168.50.2/24 |
10GE1/0/3 |
192.168.60.2/24 |
||
LoopBack0 |
6.6.6.6/32 |
LoopBack0 |
7.7.7.7/32 |
Configuration Roadmap
The configuration roadmap is as follows:
Configure IP addresses of nodes.
Configure a routing protocol to ensure route reachability between nodes.
Configure BGP EVPN in DC A and DC B to create VXLAN tunnels between distributed gateways and establish IBGP peer relationships in DC A and DC B.
Establish an EBGP peer relationship between Leaf 2 and Leaf 3.
Configure BGP EVPN on Leaf 1 and Leaf 4 to create a VXLAN tunnel.
Data Preparation
To complete the configuration, you need the following data:
VLAN IDs of VMs
BD IDs
VXLAN network identifiers (VNIs) in BDs and VNIs in VPN instances
Procedure
- Assign an IP address to each node interface, including the loopback interface.
For configuration details, see Configuration File in this section.
- Configure a routing protocol.
Configure an IGP within a data center or configure BGP between data centers. OSPF is used in this example.
For configuration details, see Configuration File in this section.
- Configure the VXLAN tunnel mode and enable the VXLAN ACL extension function. (This step only needs to be performed on the CE12800, CE16800 equipped with A series cards, CE6870EI, and CE6875EI.)
# Configure Leaf1. The configurations on Leaf4 are similar to that on Leaf1, and is not mentioned here.
[~Leaf1] ip tunnel mode vxlan [*Leaf1] assign forward nvo3 acl extend enable [*Leaf1] commit
After configuring the VXLAN tunnel mode and enabling the VXLAN ACL extension function, you need to save the configuration and restart the device to make the configuration take effect. You can restart the device immediately or after completing all the configurations.
- Configure BGP peers.
- Enable EVPN on leaf nodes to configure EVPN peers.
- Create a VXLAN tunnel between Leaf 1 and Leaf 4.
- Configure distributed gateway and bind VBDIF interfaces to VPN instances.# Configure a service loopback interface on Leaf 1. The configurations on Leaf4 are similar to that on Leaf1, and is not mentioned here. (This step only needs to be performed on the CE6850HI, CE6850U-HI, CE6851HI, CE6860EI, CE7850EI, CE8850EI, and CE8860EI.)
[~Leaf1] interface eth-trunk 2 [*Leaf1-Eth-Trunk2] service type tunnel [*Leaf1-Eth-Trunk2] quit [*Leaf1] interface 10ge 1/0/5 [*Leaf1-10GE1/0/5] eth-trunk 2 [*Leaf1-10GE1/0/5] quit [*Leaf1] commit
- The member interfaces must be idle physical interfaces that do not transmit services. There is no requirement on the interface status.
- Ensure that the Eth-Trunk bandwidth is at least twice the bandwidth required for transmitting Layer 3 VXLAN gateway traffic. For example, if traffic is sent from users to the gateway across the VXLAN network at a rate of 10 Gbit/s, add two 10GE interface to the Eth-Trunk that you want to use as the service loopback interface.
# Configure Leaf 1.
[~Leaf1] interface vbdif 10[*Leaf1-Vbdif10] ip binding vpn-instance vpn1[*Leaf1-Vbdif10] ip address 10.1.1.1 24[*Leaf1-Vbdif10] arp collect host enable[*Leaf1-Vbdif10] vxlan anycast-gateway enable[*Leaf1-Vbdif10] quit[*Leaf1] commitRepeat this step for Leaf4. For configuration details, see Configuration File in this section.
- Configure distributed gateway and bind VBDIF interfaces to VPN instances.
Verifying the Configuration
[~Leaf1] display vxlan tunnel
Number of vxlan tunnel : 1 Tunnel ID Source Destination State Type Uptime ----------------------------------------------------------------------------------- 4026531842 5.5.5.5 8.8.8.8 up dynamic 00:10:16
After configurations are complete, VMa1 and VMb2 can communicate with each other.
Configuration File
The following example is the CE12800 configuration file.
Spine1 configuration file
# sysname Spine1 # interface 10GE1/0/1 undo portswitch ip address 192.168.10.1 255.255.255.0 # interface 10GE1/0/2 undo portswitch ip address 192.168.20.1 255.255.255.0 # interface LoopBack0 ip address 3.3.3.3 255.255.255.255 # ospf 1 area 0.0.0.0 network 3.3.3.3 0.0.0.0 network 192.168.10.0 0.0.0.255 network 192.168.20.0 0.0.0.255 # return
Leaf1 configuration file
# sysname Leaf1 # assign forward nvo3 acl extend enable # evpn-overlay enable # ip vpn-instance vpn1 ipv4-family route-distinguisher 11:11 vpn-target 1:1 export-extcommunity vpn-target 11:1 export-extcommunity evpn vpn-target 1:1 import-extcommunity vpn-target 11:1 import-extcommunity evpn vxlan vni 5010 # bridge-domain 10 vxlan vni 10 evpn route-distinguisher 10:1 vpn-target 10:1 export-extcommunity vpn-target 11:1 export-extcommunity vpn-target 10:1 import-extcommunity # interface Vbdif10 ip binding vpn-instance vpn1 ip address 10.1.1.1 255.255.255.0 vxlan anycast-gateway enable arp collect host enable # interface 10GE1/0/1 undo portswitch ip address 192.168.10.2 255.255.255.0 # interface 10GE1/0/2.1 mode l2 encapsulation dot1q vid 10 bridge-domain 10 # interface LoopBack0 ip address 5.5.5.5 255.255.255.255 # interface Nve1 source 5.5.5.5 vni 10 head-end peer-list protocol bgp # bgp 100 instance evpn1 peer 6.6.6.6 as-number 100 peer 6.6.6.6 connect-interface LoopBack0 # l2vpn-family evpn policy vpn-target peer 6.6.6.6 enable peer 6.6.6.6 advertise irb # ospf 1 area 0.0.0.0 network 5.5.5.5 0.0.0.0 network 192.168.10.0 0.0.0.255 # return
Leaf2 configuration file
# sysname Leaf2 # evpn-overlay enable # interface 10GE1/0/1 undo portswitch ip address 192.168.20.2 255.255.255.0 # interface 10GE1/0/3 undo portswitch ip address 192.168.50.2 255.255.255.0 # interface LoopBack0 ip address 6.6.6.6 255.255.255.255 # bgp 20 peer 192.168.50.1 as-number 10 # ipv4-family unicast network 5.5.5.5 255.255.255.255 network 6.6.6.6 255.255.255.255 peer 192.168.50.1 enable # bgp 100 instance evpn1 peer 5.5.5.5 as-number 100 peer 5.5.5.5 connect-interface LoopBack0 peer 7.7.7.7 as-number 200 peer 7.7.7.7 ebgp-max-hop 255 peer 7.7.7.7 connect-interface LoopBack0 # l2vpn-family evpn undo policy vpn-target peer 5.5.5.5 enable peer 5.5.5.5 advertise irb peer 5.5.5.5 next-hop-invariable peer 7.7.7.7 enable peer 7.7.7.7 advertise irb peer 7.7.7.7 next-hop-invariable # ospf 1 import-route bgp area 0.0.0.0 network 6.6.6.6 0.0.0.0 network 192.168.20.0 0.0.0.255 # return
Spine2 configuration file
# sysname Spine2 # interface 10GE1/0/1 undo portswitch ip address 192.168.30.1 255.255.255.0 # interface 10GE1/0/2 undo portswitch ip address 192.168.40.1 255.255.255.0 # interface LoopBack0 ip address 4.4.4.4 255.255.255.255 # ospf 1 area 0.0.0.0 network 4.4.4.4 0.0.0.0 network 192.168.30.0 0.0.0.255 network 192.168.40.0 0.0.0.255 # return
Leaf3 configuration file
# sysname Leaf3 # evpn-overlay enable # interface 10GE1/0/1 undo portswitch ip address 192.168.30.2 255.255.255.0 # interface 10GE1/0/3 undo portswitch ip address 192.168.60.2 255.255.255.0 # interface LoopBack0 ip address 7.7.7.7 255.255.255.255 # bgp 30 peer 192.168.60.1 as-number 10 # ipv4-family unicast network 7.7.7.7 255.255.255.255 network 8.8.8.8 255.255.255.255 peer 192.168.60.1 enable # bgp 200 instance evpn1 peer 6.6.6.6 as-number 100 peer 6.6.6.6 ebgp-max-hop 255 peer 6.6.6.6 connect-interface LoopBack0 peer 8.8.8.8 as-number 200 peer 8.8.8.8 connect-interface LoopBack0 # l2vpn-family evpn undo policy vpn-target peer 6.6.6.6 enable peer 6.6.6.6 advertise irb peer 6.6.6.6 next-hop-invariable peer 8.8.8.8 enable peer 8.8.8.8 advertise irb peer 8.8.8.8 next-hop-invariable # ospf 1 import-route bgp area 0.0.0.0 network 7.7.7.7 0.0.0.0 network 192.168.30.0 0.0.0.255 # return
Leaf4 configuration file
# sysname Leaf4 # assign forward nvo3 acl extend enable # evpn-overlay enable # ip vpn-instance vpn1 ipv4-family route-distinguisher 11:14 vpn-target 4:4 export-extcommunity vpn-target 11:1 export-extcommunity evpn vpn-target 4:4 import-extcommunity vpn-target 11:1 import-extcommunity evpn vxlan vni 5010 # bridge-domain 20 vxlan vni 20 evpn route-distinguisher 40:1 vpn-target 40:1 export-extcommunity vpn-target 11:1 export-extcommunity vpn-target 40:1 import-extcommunity # interface Vbdif20 ip binding vpn-instance vpn1 ip address 10.2.1.1 255.255.255.0 vxlan anycast-gateway enable arp collect host enable # interface 10GE1/0/1 undo portswitch ip address 192.168.40.2 255.255.255.0 # interface 10GE1/0/2.1 mode l2 encapsulation dot1q vid 20 bridge-domain 20 # interface LoopBack0 ip address 8.8.8.8 255.255.255.255 # interface Nve1 source 8.8.8.8 vni 20 head-end peer-list protocol bgp # bgp 200 instance evpn1 peer 7.7.7.7 as-number 200 peer 7.7.7.7 connect-interface LoopBack0 # l2vpn-family evpn policy vpn-target peer 7.7.7.7 enable peer 7.7.7.7 advertise irb # ospf 1 area 0.0.0.0 network 8.8.8.8 0.0.0.0 network 192.168.40.0 0.0.0.255 # return
Device1 configuration file
# sysname Device1 # interface 10GE1/0/1 undo portswitch ip address 192.168.50.1 255.255.255.0 # interface 10GE1/0/2 undo portswitch ip address 192.168.1.1 255.255.255.0 # interface LoopBack0 ip address 1.1.1.1 255.255.255.255 # bgp 10 peer 192.168.1.2 as-number 10 peer 192.168.50.2 as-number 20 # ipv4-family unicast peer 192.168.1.2 enable peer 192.168.1.2 next-hop-local peer 192.168.50.2 enable # return
Device2 configuration file
# sysname Device2 # interface 10GE1/0/1 undo portswitch ip address 192.168.60.1 255.255.255.0 # interface 10GE1/0/2 undo portswitch ip address 192.168.1.2 255.255.255.0 # interface LoopBack0 ip address 2.2.2.2 255.255.255.255 # bgp 10 peer 192.168.1.1 as-number 10 peer 192.168.60.2 as-number 30 # ipv4-family unicast peer 192.168.1.1 enable peer 192.168.1.1 next-hop-local peer 192.168.60.2 enable # return