CloudEngine 12800 V200R020C10 Configuration Guide - VXLAN
This document describes the configurations of VXLAN.
This document describes the configurations of VXLAN.
Example for Configuring VXLAN in Distributed Gateway Mode Using BGP EVPN
Networking Requirements
Distributed VXLAN gateways can be configured to address problems that occur in legacy centralized VXLAN gateway networking, for example, forwarding paths are not optimal, and the ARP entry specification is a bottleneck.
On the network shown in Figure 15-2, an enterprise has VMs deployed in different data centers. VM1 on Server1 belongs to VLAN 10, and VM1 on Server2 belongs to VLAN 20. VM1 on Server1 and VM1 on Server2 reside on different network segments. Server1 connects to the VXLAN through Device2 and Device4. To allow VM1s in different data centers to communicate with each other, configure distributed VXLAN gateways. Device1 is deployed in AS 100, Device2 and Device4 are deployed in AS 200, and Device3 is deployed in AS 300. Device1, Device2, Device3, and Device4 use AS 100 for BGP EVPN.
Device |
Interface |
IP Address |
|---|---|---|
Device1 |
10GE1/0/1 |
192.168.3.2/24 |
10GE1/0/2 |
192.168.2.2/24 |
|
10GE1/0/3 |
192.168.4.2/24 |
|
LoopBack0 |
1.1.1.1/32 |
|
Device2 |
10GE1/0/1 |
192.168.2.1/24 |
LoopBack0 |
2.2.2.2/32 |
|
LoopBack1 |
2.2.2.210/32 |
|
Device3 |
10GE1/0/1 |
192.168.3.1/24 |
LoopBack0 |
3.3.3.3/32 |
|
Device4 |
10GE1/0/1 |
192.168.4.1/24 |
LoopBack0 |
4.4.4.4/32 |
|
LoopBack1 |
2.2.2.210/32 |
Configuration Roadmap
- Configure EBGP to run between Device1 and Device2 and between Device1 and Device3 and between Device1 and Device4.
- Configure Device2 and Device4 as the root bridge, and configure the same bridge ID for them.
- Configure M-LAG between Device2 and Device4.
- Configure a service access point on Device2, Device3, and Device4 to differentiate service traffic.
- Configure EVPN as the VXLAN control plane.
- Specify Device1 as an IBGP EVPN peer for Device2, Device3, and Device4.
- Specify Device2, Device3, and Device4 as BGP EVPN peers for Device1 and configure Device2, Device3, and Device4 as RR clients.
- Configure VPN and EVPN instances on Device2, Device3, and Device4.
- Configure an ingress replication list on Device2, Device3, and Device4.
- Configure Device2, Device3, and Device4 as Layer 3 VXLAN gateways.
- Configure BGP between Device1 and Device2, Device3, and Device4 respectively to advertise IRB routes.
Data Preparation
To complete the configuration, you need the following data:
- VMs' VLAN IDs (10 and 20)
- IP addresses of interfaces connecting devices
- BD IDs (10 and 20)
- VNI IDs (10 and 20)
- VNI ID (5010) in a VPN instance
- RDs and RTs of EVPN and VPN instances, as described in Table 15-2
Device |
RD |
RT |
|---|---|---|
Device2 |
EVPN instance: 10:2 VPN instance: 20:2 |
EVPN instance:
VPN instance:
|
Device3 |
EVPN instance: 10:3 VPN instance: 20:3 |
EVPN instance:
VPN instance:
|
Device4 |
EVPN instance: 10:4 VPN instance: 20:4 |
EVPN instance:
VPN instance:
|
- For a VPN instance, you need to configure ERT Y and IRT Y with the EVPN parameter to enable route leaking into peer EVPN instances for host route generation. If route leaking into common L3VPN instances is required, configure common ERTs and IRTs.
- For an EVPN instance, apart from ERT A, ERT B, IRT A, and IRT B for two BDs, you also need to configure ERT Y for route iteration with a VPN instance. Generally, you do not need to configure IRT Y. If IRT Y is configured, the EVPN instances in different BDs flood MAC addresses to each other.
Precautions
Run the assign forward nvo3 f-linecard compatibility enable command in the system view to ensure that VXLAN traffic can be forwarded when the card interoperability mode is non-enhanced mode.
![]()
If VXLAN traffic is forwarded between cards when the card interoperability mode is non-enhanced mode, the VXLAN traffic may fail to be forwarded. To use the VXLAN function, you must configure the assign forward nvo3 f-linecard compatibility enable command when the card interoperability mode is non-enhanced mode.
VXLAN-related constraints are described in the procedure. To obtain more constraint information, see Licensing Requirements and Limitations for VXLANs .
Procedure
- Configure an EBGP routing protocol.
# Configure Device1. Repeat this step for Device2, Device3, and Device4.
<HUAWEI> system-view [~HUAWEI] sysname Device1 [*HUAWEI] commit [~Device1] interface loopback 0 [*Device1-LoopBack0] ip address 1.1.1.1 32 [*Device1-LoopBack0] quit [*Device1] interface 10ge 1/0/1 [*Device1-10GE1/0/1] undo portswitch [*Device1-10GE1/0/1] ip address 192.168.3.2 24 [*Device1-10GE1/0/1] quit [*Device1] interface 10ge 1/0/2 [*Device1-10GE1/0/2] undo portswitch [*Device1-10GE1/0/2] ip address 192.168.2.2 24 [*Device1-10GE1/0/2] quit [*Device1] interface 10ge 1/0/3 [*Device1-10GE1/0/3] undo portswitch [*Device1-10GE1/0/3] ip address 192.168.4.2 24 [*Device1-10GE1/0/3] quit [*Device1] bgp 100 [*Device1-bgp] peer 192.168.2.1 as-number 200 [*Device1-bgp] peer 192.168.3.1 as-number 300 [*Device1-bgp] peer 192.168.4.1 as-number 200 [*Device1-bgp] network 1.1.1.1 32 [*Device1-bgp] quit [*Device1] commit
As Device 2 and Device 4 both belong to AS 200, to ensure that the two devices can notify each other of BGP routes, run the peer 192.168.2.2 allow-as-loop command on Device 2 and the peer 192.168.4.2 allow-as-loop command on Device 4.
- Configure the VXLAN tunnel mode and enable the VXLAN ACL extension function. (This step only needs to be performed on the CE12800.)
# Configure Device2. Repeat this step for Device3 and Device4.
[~Device2] ip tunnel mode vxlan [*Device2] assign forward nvo3 acl extend enable [*Device2] commit
![]()
After modifying the VXLAN tunnel mode or enabling the VXLAN ACL extension function, you need to save the configuration and restart the device to make the configuration take effect. You can restart the device immediately or after completing all the configurations.
- Configure V-STP-based M-LAG between Device2 and Device4.
![]()
If the link through which Device2 is uplink connected to the VXLAN network fails, Device2 discards all received user traffic because no uplink outbound interface is available. You can configure a Monitor Link group to associate the uplink and downlink interfaces of Device2. When the uplink outbound interface of Device2 becomes Down, the downlink interface also becomes Down. Then user traffic will not be forwarded or discarded by Device2. For details about how to configure a Monitor Link group, see Configuring the Uplink and Downlink Interfaces in a Monitor Link Group.
- Configure a service access point on Device2, Device3, and Device4.
# Configure Device2. Repeat this step for Device3 and Device4.
[~Device2] bridge-domain 10 [*Device2-bd10] quit [*Device2] interface eth-trunk 10.1 mode l2 [*Device2-Eth-Trunk10.1] encapsulation dot1q vid 10 [*Device2-Eth-Trunk10.1] bridge-domain 10 [*Device2-Eth-Trunk10.1] quit [*Device2] commit
- Configure EVPN as the VXLAN control plane.
# Configure Device1. Repeat this step for Device2, Device3, and Device4.
[~Device1] evpn-overlay enable [*Device1] commit
- Specify Device2, Device3, and Device4 as BGP EVPN peers for Device1 and configure them as RR clients.# Specify BGP EVPN peers for Device1.
[~Device1] bgp 100 instance evpn1 [*Device1-bgp-instance-evpn1] peer 2.2.2.2 as-number 100 [*Device1-bgp-instance-evpn1] peer 2.2.2.2 connect-interface LoopBack0 [*Device1-bgp-instance-evpn1] peer 3.3.3.3 as-number 100 [*Device1-bgp-instance-evpn1] peer 3.3.3.3 connect-interface LoopBack0 [*Device1-bgp-instance-evpn1] peer 4.4.4.4 as-number 100 [*Device1-bgp-instance-evpn1] peer 4.4.4.4 connect-interface LoopBack0 [*Device1-bgp-instance-evpn1] l2vpn-family evpn [*Device1-bgp-instance-evpn1-af-evpn] peer 2.2.2.2 enable Warning: This operation will reset the peer session. Continue? [Y/N]: y [*Device1-bgp-instance-evpn1-af-evpn] peer 2.2.2.2 reflect-client [*Device1-bgp-instance-evpn1-af-evpn] peer 3.3.3.3 enable Warning: This operation will reset the peer session. Continue? [Y/N]: y [*Device1-bgp-instance-evpn1-af-evpn] peer 3.3.3.3 reflect-client [*Device1-bgp-instance-evpn1-af-evpn] peer 4.4.4.4 enable Warning: This operation will reset the peer session. Continue? [Y/N]: y [*Device1-bgp-instance-evpn1-af-evpn] peer 4.4.4.4 reflect-client [*Device1-bgp-instance-evpn1-af-evpn] undo policy vpn-target [*Device1-bgp-instance-evpn1-af-evpn] quit [*Device1-bgp-instance-evpn1] quit [*Device1] commit
- Specify Device1 as an IBGP EVPN peer for Device2, Device3, and Device4.# Specify Device1 as a BGP EVPN peer for Device2. Repeat this step for Device3 and Device4.
[~Device2] bgp 100 instance evpn1 [*Device2-bgp-instance-evpn1] peer 1.1.1.1 as-number 100 [*Device2-bgp-instance-evpn1] peer 1.1.1.1 connect-interface LoopBack0 [*Device2-bgp-instance-evpn1] l2vpn-family evpn [*Device2-bgp-instance-evpn1-af-evpn] peer 1.1.1.1 enable Warning: This operation will reset the peer session. Continue? [Y/N]: y [*Device2-bgp-instance-evpn1-af-evpn] quit [*Device2-bgp-instance-evpn1] quit [*Device2] commit
- Configure VPN and EVPN instances on Device2, Device3, and Device4.
# Configure VPN and EVPN instances on Device2. Repeat this step for Device3 and Device4.
[~Device2] ip vpn-instance vpn1 [*Device2-vpn-instance-vpn1] vxlan vni 5010 [*Device2-vpn-instance-vpn1] ipv4-family [*Device2-vpn-instance-vpn1-af-ipv4] route-distinguisher 20:2 [*Device2-vpn-instance-vpn1-af-ipv4] vpn-target 100:5010 evpn [*Device2-vpn-instance-vpn1-af-ipv4] quit [*Device2-vpn-instance-vpn1] quit [*Device2] bridge-domain 10 [*Device2-bd10] vxlan vni 10 [*Device2-bd10] evpn [*Device2-bd10-evpn] route-distinguisher 10:2 [*Device2-bd10-evpn] vpn-target 100:10 [*Device2-bd10-evpn] vpn-target 100:5010 export-extcommunity [*Device2-bd10-evpn] quit [*Device2-bd10] quit [*Device2] commit
- Configure an ingress replication list on Device2, Device3, and Device4.
# Configure Device2. Repeat this step for Device3 and Device4. You do not need to configure a MAC address for the NVE interface of Device3.
[~Device2] interface nve 1 [*Device2-Nve1] source 2.2.2.210 [*Device2-Nve1] mac-address 0000-5e00-0101 [*Device2-Nve1] vni 10 head-end peer-list protocol bgp [*Device2-Nve1] quit [*Device2] commit
![]()
Ensure that the IP addresses and MAC addresses of the NVE interfaces on Device2 and Device4 are the same, as they are dual-active gateways.
- Configure Device2, Device3, and Device4 as Layer 3 VXLAN gateways.# Configure a Layer 3 VXLAN gateway on Device2. The configurations on Device3 and Device4 are similar to that on Device2, and are not mentioned here. The IP addresses of VBDIF interfaces on Device2 and Device3 must be on different network segments. Configure the same IP address and MAC address for the VBDIF interface of Device2 and Device4. You do not need to configure a MAC address for the VBDIF interface of Device3.
[~Device2] interface vbdif10 [*Device2-Vbdif10] ip binding vpn-instance vpn1 [*Device2-Vbdif10] ip address 10.1.1.1 255.255.255.0 [*Device2-Vbdif10] mac-address 0000-5e00-0102 [*Device2-Vbdif10] vxlan anycast-gateway enable [*Device2-Vbdif10] arp collect host enable [*Device2-Vbdif10] quit [*Device2] commit
![]()
Ensure that the IP addresses and MAC addresses of the VBDIF interfaces on Device2 and Device4 are the same, as they are dual-active gateways.
- Configure BGP between Device1 and Device2, Device3, and Device4 respectively to advertise IRB routes.
# Configure Device1. The configurations of Device2, Device3, and Device4 are similar to the configuration of Device1, and are not mentioned here.
[~Device1] bgp 100 instance evpn1 [~Device1-bgp-instance-evpn1] l2vpn-family evpn [~Device1-bgp-instance-evpn1-af-evpn] peer 2.2.2.2 advertise irb [*Device1-bgp-instance-evpn1-af-evpn] peer 3.3.3.3 advertise irb [*Device1-bgp-instance-evpn1-af-evpn] peer 4.4.4.4 advertise irb [*Device1-bgp-instance-evpn1-af-evpn] quit [*Device1-bgp-instance-evpn1] quit [*Device1] commit
- Configure BGP between Device2 and Device3 and between Device2 and Device4 to advertise IP prefix routes to peers.
# Configure Device2. Repeat this step for Device3 and Device4.
[~Device2] bgp 100 instance evpn1 [~Device2-bgp-instance-evpn1] ipv4-family vpn-instance vpn1 [*Device2-bgp-instance-evpn1-vpn1] import-route direct [*Device2-bgp-instance-evpn1-vpn1] advertise l2vpn evpn [*Device2-bgp-instance-evpn1-vpn1] quit [*Device2-bgp-instance-evpn1] quit [*Device2] commit
- Verify the configuration.
After completing the configurations, run the display vxlan tunnel command on Device2, Device3, and Device4 to check VXLAN tunnel information. The following example shows the command output on Device2.
[~Device2] display vxlan tunnel Number of vxlan tunnel : 1 Tunnel ID Source Destination State Type Uptime ----------------------------------------------------------------------------------- 4026531841 2.2.2.210 3.3.3.3 up dynamic 0032h21mVM1s on different servers can communicate.
![]()
The tunnel comes Up only after a Layer 2 sub-interface on Device2, Device3, or Device4 is connected to a server. When there is no server connected to the device, the VXLAN tunnel state is not displayed because no IRB route is advertised.
Configuration Files
Device1 configuration file
# sysname Device1 # evpn-overlay enable # interface 10GE1/0/1 undo portswitch ip address 192.168.3.2 255.255.255.0 # interface 10GE1/0/2 undo portswitch ip address 192.168.2.2 255.255.255.0 # interface 10GE1/0/3 undo portswitch ip address 192.168.4.2 255.255.255.0 # interface LoopBack0 ip address 1.1.1.1 255.255.255.255 # bgp 100 peer 192.168.2.1 as-number 200 peer 192.168.3.1 as-number 300 peer 192.168.4.1 as-number 200 # ipv4-family unicast network 1.1.1.1 255.255.255.255 peer 192.168.2.1 enable peer 192.168.3.1 enable peer 192.168.4.1 enable # bgp 100 instance evpn1 peer 2.2.2.2 as-number 100 peer 2.2.2.2 connect-interface LoopBack0 peer 3.3.3.3 as-number 100 peer 3.3.3.3 connect-interface LoopBack0 peer 4.4.4.4 as-number 100 peer 4.4.4.4 connect-interface LoopBack0 # l2vpn-family evpn undo policy vpn-target peer 2.2.2.2 enable peer 2.2.2.2 advertise irb peer 2.2.2.2 reflect-client peer 3.3.3.3 enable peer 3.3.3.3 advertise irb peer 3.3.3.3 reflect-client peer 4.4.4.4 enable peer 4.4.4.4 advertise irb peer 4.4.4.4 reflect-client # return
Device2 configuration file
# sysname Device2 # assign forward nvo3 acl extend enable //This step only needs to be performed on the CE12800. # dfs-group 1 source ip 2.2.2.2 # stp mode rstp stp v-stp enable # evpn-overlay enable # ip vpn-instance vpn1 ipv4-family route-distinguisher 20:2 vpn-target 100:5010 export-extcommunity evpn vpn-target 100:5010 import-extcommunity evpn vxlan vni 5010 # bridge-domain 10 vxlan vni 10 evpn route-distinguisher 10:2 vpn-target 100:10 export-extcommunity vpn-target 100:5010 export-extcommunity vpn-target 100:10 import-extcommunity # interface Vbdif10 ip binding vpn-instance vpn1 ip address 10.1.1.1 255.255.255.0 mac-address 0000-5e00-0102 vxlan anycast-gateway enable arp collect host enable # interface Eth-Trunk1 mode lacp-static peer-link 1 # interface Eth-Trunk10 stp edged-port enable mode lacp-static dfs-group 1 m-lag 1 # interface Eth-Trunk10.1 mode l2 encapsulation dot1q vid 10 bridge-domain 10 # interface 10GE1/0/1 undo portswitch ip address 192.168.2.1 255.255.255.0 # interface 10GE1/0/2 eth-trunk 10 # interface 10GE1/0/3 eth-trunk 1 # interface 10GE1/0/4 eth-trunk 1 # interface LoopBack0 ip address 2.2.2.2 255.255.255.255 # interface LoopBack1 ip address 2.2.2.210 255.255.255.255 # interface Nve1 source 2.2.2.210 vni 10 head-end peer-list protocol bgp mac-address 0000-5e00-0101 # bgp 200 peer 192.168.2.2 as-number 100 # ipv4-family unicast network 2.2.2.2 255.255.255.255 network 2.2.2.210 255.255.255.255 peer 192.168.2.2 enable peer 192.168.2.2 allow-as-loop # bgp 100 instance evpn1 peer 1.1.1.1 as-number 100 peer 1.1.1.1 connect-interface LoopBack0 # ipv4-family vpn-instance vpn1 import-route direct advertise l2vpn evpn # l2vpn-family evpn policy vpn-target peer 1.1.1.1 enable peer 1.1.1.1 advertise irb # returnDevice3 configuration file
# sysname Device3 # assign forward nvo3 acl extend enable //This step only needs to be performed on the CE12800. # evpn-overlay enable # ip vpn-instance vpn1 ipv4-family route-distinguisher 20:3 vpn-target 100:5010 export-extcommunity evpn vpn-target 100:5010 import-extcommunity evpn vxlan vni 5010 # bridge-domain 20 vxlan vni 20 evpn route-distinguisher 10:3 vpn-target 100:20 export-extcommunity vpn-target 100:5010 export-extcommunity vpn-target 100:20 import-extcommunity # interface Vbdif20 ip binding vpn-instance vpn1 ip address 20.1.1.1 255.255.255.0 vxlan anycast-gateway enable arp collect host enable # interface 10GE1/0/1 undo portswitch ip address 192.168.3.1 255.255.255.0 # interface 10GE1/0/2.1 mode l2 encapsulation dot1q vid 20 bridge-domain 20 # interface LoopBack0 ip address 3.3.3.3 255.255.255.255 # interface Nve1 source 3.3.3.3 vni 20 head-end peer-list protocol bgp # bgp 300 peer 192.168.3.2 as-number 100 # ipv4-family unicast network 3.3.3.3 255.255.255.255 peer 192.168.3.2 enable # bgp 100 instance evpn1 peer 1.1.1.1 as-number 100 peer 1.1.1.1 connect-interface LoopBack0 # ipv4-family vpn-instance vpn1 import-route direct advertise l2vpn evpn # l2vpn-family evpn policy vpn-target peer 1.1.1.1 enable peer 1.1.1.1 advertise irb # returnDevice4 configuration file
# sysname Device4 # assign forward nvo3 acl extend enable //This step only needs to be performed on the CE12800. # dfs-group 1 source ip 4.4.4.4 # stp mode rstp stp v-stp enable # evpn-overlay enable # ip vpn-instance vpn1 ipv4-family route-distinguisher 20:4 vpn-target 100:5010 export-extcommunity evpn vpn-target 100:5010 import-extcommunity evpn vxlan vni 5010 # bridge-domain 10 vxlan vni 10 evpn route-distinguisher 10:4 vpn-target 100:10 export-extcommunity vpn-target 100:5010 export-extcommunity vpn-target 100:10 import-extcommunity # interface Vbdif10 ip binding vpn-instance vpn1 ip address 10.1.1.1 255.255.255.0 mac-address 0000-5e00-0102 vxlan anycast-gateway enable arp collect host enable # interface Eth-Trunk1 mode lacp-static peer-link 1 # interface Eth-Trunk10 stp edged-port enable mode lacp-static dfs-group 1 m-lag 1 # interface Eth-Trunk10.1 mode l2 encapsulation dot1q vid 10 bridge-domain 10 # interface 10GE1/0/1 undo portswitch ip address 192.168.4.1 255.255.255.0 # interface 10GE1/0/2 eth-trunk 10 # interface 10GE1/0/3 eth-trunk 1 # interface 10GE1/0/4 eth-trunk 1 # interface LoopBack0 ip address 4.4.4.4 255.255.255.255 # interface LoopBack1 ip address 2.2.2.210 255.255.255.255 # interface Nve1 source 2.2.2.210 vni 10 head-end peer-list protocol bgp mac-address 0000-5e00-0101 # bgp 200 peer 192.168.4.2 as-number 100 # ipv4-family unicast network 2.2.2.210 255.255.255.255 network 4.4.4.4 255.255.255.255 peer 192.168.4.2 enable peer 192.168.4.2 allow-as-loop # bgp 100 instance evpn1 peer 1.1.1.1 as-number 100 peer 1.1.1.1 connect-interface LoopBack0 # ipv4-family vpn-instance vpn1 import-route direct advertise l2vpn evpn # l2vpn-family evpn policy vpn-target peer 1.1.1.1 enable peer 1.1.1.1 advertise irb # return