Hardware Distributed VXLAN Using the Spine/Leaf Two-Layer Architecture
This section describes the configurations that are different from the configurations of the gateway/spine/leaf three-layer architecture.
The configuration differences are as follows:
- Configuration of connected interfaces between the gateway and leaf node
- BGP route configuration on the leaf node
- BGP route configuration on the gateway
- BGP EVPN configuration on the gateway and leaf node
Networking Requirements
Figure 1 shows the hardware distributed VXLAN using the spine/leaf two-layer architecture.
- Server layer: Servers are connected to the VXLAN network through Layer 2 sub-interfaces.
- Leaf nodes (distributed gateways, also called east-west gateways): Servers are connected to leaf nodes through stacking, Multi-Chassis Link Aggregation Group (M-LAG), or super virtual fabric (SVF). Leaf nodes and spine nodes communicate at Layer 3. A stack, M-LAG, or SVF system consisting of leaf nodes functions as a virtual tunnel end point (VTEP) to allow server traffic to access the VXLAN.
- Spine nodes/GWs: Two gateways (GWs) constitute an M-LAG, and serve as dual-active gateways. The gateways and leaf nodes communicate at Layer 3, and connect to external routers Router-1 and Router-2.
- FWs: Two firewalls (FWs) are configured to work in active/standby mirroring mode and connected to the two gateways in bypass mode.
- LBs: Load balancers (LBs) are deployed by manufacturers.
Port Connection Planning
Deploy Leaf-CE6851HI-1 and Leaf-CE6851HI-2 as a stack. The port planning is as follows.
Planning Description |
Local Device |
Port |
Remote Device |
Port |
|---|---|---|---|---|
Configure leaf node interconnection ports as stack ports and at least two stack links. |
Leaf-CE6851HI-1 |
40GE1/0/1-2 |
Leaf-CE6851HI-2 |
40GE1/0/1-2 |
Leaf-CE6851HI-2 |
40GE1/0/1-2 |
Leaf-CE6851HI-1 |
40GE1/0/1-2 |
|
Configure ports on the leaf nodes and spine nodes for interconnection. |
Leaf-CE6851HI-1 |
40GE1/0/3 |
Exit-Gateway-CE12808-1 |
40GE1/0/0 |
Leaf-CE6851HI-1 |
40GE1/0/4 |
Exit-Gateway-CE12808-2 |
40GE1/0/1 |
|
Leaf-CE6851HI-2 |
40GE1/0/3 |
Exit-Gateway-CE12808-2 |
40GE1/0/0 |
|
Leaf-CE6851HI-2 |
40GE1/0/4 |
Exit-Gateway-CE12808-1 |
40GE1/0/1 |
|
Configure ports on the leaf nodes and servers for interconnection. Connect the servers with dual NICs to the leaf nodes in load balancing mode to ensure link reliability. |
Leaf-CE6851HI-1 |
10GE1/0/1-2 |
Server |
Eth0 |
Leaf-CE6851HI-2 |
10GE1/0/1-2 |
Server |
Eth1 |
Deploy Leaf-CE6851HI-3 and Leaf-CE6851HI-4 as an M-LAG. The port planning is as follows.
Planning Description |
Local Device |
Port |
Remote Device |
Port |
|---|---|---|---|---|
Configure peer-link ports of the M-LAG to transmit protocol packets, as well as data packets when faults occur. Configure at least two member links for the peer-link to ensure reliability. |
Leaf-CE6851HI-3 |
40GE1/0/1-2 |
Leaf-CE6851HI-4 |
40GE1/0/1-2 |
Leaf-CE6851HI-4 |
40GE1/0/1-2 |
Leaf-CE6851HI-3 |
40GE1/0/1-2 |
|
Configure ports on the leaf nodes and spine nodes for interconnection. |
Leaf-CE6851HI-3 |
40GE1/0/3 |
Exit-Gateway-CE12808-1 |
40GE1/0/2 |
Leaf-CE6851HI-3 |
40GE1/0/4 |
Exit-Gateway-CE12808-2 |
40GE1/0/3 |
|
Leaf-CE6851HI-4 |
40GE1/0/3 |
Exit-Gateway-CE12808-2 |
40GE1/0/2 |
|
Leaf-CE6851HI-4 |
40GE1/0/4 |
Exit-Gateway-CE12808-1 |
40GE1/0/3 |
|
Configure ports on the leaf nodes and servers for interconnection. Connect the servers with dual NICs to the leaf nodes in load balancing mode to ensure link reliability and improve link utilization. |
Leaf-CE6851HI-3 |
10GE1/0/1 |
Server |
Eth0 |
Leaf-CE6851HI-4 |
10GE1/0/1 |
Server |
Eth1 |
Connect Leaf-CE6851HI-5 and Leaf-CE6851HI-6 to three CE5810 switches and configure the five devices to form an SVF system. The port planning is as follows.
Planning Description |
Local Device |
Port |
Remote Device |
Port |
|---|---|---|---|---|
Configure leaf node interconnection ports as stack ports and at least two stack links. |
Leaf-CE6851HI-5 |
40GE1/0/1-2 |
Leaf-CE6851HI-6 |
40GE1/0/1-2 |
Leaf-CE6851HI-6 |
40GE1/0/1-2 |
Leaf-CE6851HI-5 |
40GE1/0/1-2 |
|
Configure ports on the leaf nodes and spine nodes for interconnection. |
Leaf-CE6851HI-5 |
40GE1/0/3 |
Exit-Gateway-CE12808-1 |
40GE1/0/6 |
Leaf-CE6851HI-5 |
40GE1/0/4 |
Exit-Gateway-CE12808-2 |
40GE1/0/7 |
|
Leaf-CE6851HI-6 |
40GE1/0/3 |
Exit-Gateway-CE12808-1 |
40GE1/0/6 |
|
Leaf-CE6851HI-6 |
40GE1/0/4 |
Exit-Gateway-CE12808-2 |
40GE1/0/7 |
|
Configure ports on SVF parent nodes and leaf nodes for interconnection. |
Leaf-CE6851HI-5 |
10GE1/0/1 |
CE5800-1 |
GE1/0/1 |
10GE1/0/2 |
CE5800-2 |
GE1/0/1 |
||
10GE1/0/3 |
CE5800-3 |
GE1/0/1 |
||
Leaf-CE6851HI-6 |
10GE1/0/1 |
CE5800-1 |
GE1/0/2 |
|
10GE1/0/2 |
CE5800-2 |
GE1/0/2 |
||
10GE1/0/3 |
CE5800-3 |
GE1/0/2 |
||
Configure ports on the SVF leaf nodes and servers for interconnection. Connect the servers with dual NICs to the SVF leaf nodes in load balancing mode to ensure link reliability. |
CE5810-1 |
GE1/0/1 |
Server |
Eth0 |
CE5810-2 |
GE1/0/1 |
Server |
Eth1 |
Deploy Exit-Gateway-CE12808-1 and Exit-Gateway-CE12808-2. The port planning is as follows.
Planning Description |
Local Device |
Port |
Remote Device |
Port |
|---|---|---|---|---|
Configure ports on the egress gateways and leaf nodes for interconnection. |
Exit-Gateway-CE12808-1 |
40GE1/0/0-3 40GE1/0/6-7 |
Leaf-CE6851HI-1 Leaf-CE6851HI-2 Leaf-CE6851HI-3 Leaf-CE6851HI-4 Leaf-CE6851HI-5 Leaf-CE6851HI-6 |
40GE1/0/3-4 |
Exit-Gateway-CE12808-2 |
40GE1/0/0-3 40GE1/0/6-7 |
Leaf-CE6851HI-1 Leaf-CE6851HI-2 Leaf-CE6851HI-3 Leaf-CE6851HI-4 Leaf-CE6851HI-5 Leaf-CE6851HI-6 |
40GE1/0/3-4 |
|
Configure peer-link ports of the M-LAG to transmit protocol packets, as well as data packets when faults occur. Configure at least two member links across cards for the peer-link to ensure reliability. |
Exit-Gateway-CE12808-1 |
40GE1/0/23 40GE2/0/23 |
Exit-Gateway-CE12808-2 |
40GE1/0/23 40GE2/0/23 |
Exit-Gateway-CE12808-2 |
40GE1/0/23 40GE2/0/23 |
Exit-Gateway-CE12808-1 |
40GE1/0/23 40GE2/0/23 |
|
NOTE:
Configure ports on the egress gateways and firewalls for interconnection. NOTE:
Two firewalls in active/standby mirroring mode must use the same port to connect to a gateway. For example, if FW-1 connects to GW-1 through GE 1/0/1, GE 1/0/1 must be set for FW-2 to connect to GW-1. |
Exit-Gateway-CE12808-1 |
10GE3/0/0-1 |
FW-USG9560-1 FW-USG9560-2 |
GE1/0/1 |
Exit-Gateway-CE12808-1 |
10GE3/0/2-3 |
FW-USG9560-1 FW-USG9560-2 |
GE1/0/3 |
|
Exit-Gateway-CE12808-2 |
10GE3/0/0-1 |
FW-USG9560-1 FW-USG9560-2 |
GE1/0/2 |
|
Exit-Gateway-CE12808-2 |
10GE3/0/2-3 |
FW-USG9560-1 FW-USG9560-2 |
GE1/0/4 |
|
Configure ports on the egress gateways and egress routers for interconnection. |
Exit-Gateway-CE12808-1 |
10GE3/0/4 |
Router-1 |
GE1/0/0 |
Exit-Gateway-CE12808-1 |
10GE3/0/5 |
Router-2 |
GE1/0/0 |
|
Exit-Gateway-CE12808-2 |
10GE3/0/4 |
Router-1 |
GE1/0/1 |
|
Exit-Gateway-CE12808-2 |
10GE3/0/5 |
Router-2 |
GE1/0/1 |
|
Configure Layer 3 interconnection interfaces between egress gateways, which are used to connect to egress routers in mesh mode. |
Exit-Gateway-CE12808-1 |
10GE3/0/6 |
Exit-Gateway-CE12808-2 |
10GE3/0/6 |
VLAN Planning
The following table describes VLAN planning of the solution.
Planning Description |
Suggestion |
VLAN ID Example |
|---|---|---|
VLAN for firewall interconnection |
You are advised to use Layer 3 main interfaces for Layer 3 interconnection. Firewall interconnection involves interconnection of multiple network segments. Therefore, plan VLANs to share physical links. |
11 12 (Create VLANIF 12 and bind it to a VPN.) |
Tenant VLAN |
Service capacity expansion needs to be considered as many tenants or services use VXLAN. |
10 |
Before switching an interface on the CE6855HI or CE7855EI to Layer 3 mode, run the vlan reserved for main-interface startvlanid to endvlanid command to configure a dedicated reserved VLAN for the Layer 3 main interface.
BD and VNI Planning
The following table describes BD and VNI planning of the solution.
Planning Description |
Suggestion |
BD and VNI ID Example |
|---|---|---|
BD |
Plan the same number of BDs as the number of VLANs. You are advised to use the same BD ID as the VLAN ID. |
BD 10 is used here. |
VNI |
Plan the same number of Layer 2 VNIs as the number of BDs. You are advised to plan the VNI ID as the BD ID plus 10000. A BD corresponds to a VNI. The number of Layer 3 VNIs is the same as the number of L3VPNs. |
Layer 2 VNI 10010 and Layer 3 VNI 10 are used here. |
RD and RT Planning
The following table describes the RD planning of the solution.
Planning Description |
Suggestion |
Example |
|---|---|---|
RD of CE6851HI-1 and CE6851HI-2 |
Distributed gateways must be configured with RDs of EVPN and VPN instances, and RDs must be unique. |
11:1 12:1 (VPN instance) |
RD of CE6851HI-3 |
Distributed gateways must be configured with RDs of EVPN and VPN instances, and RDs must be unique. |
13:1 12:2 (VPN instance) |
RD of CE6851HI-4 |
Distributed gateways must be configured with RDs of EVPN and VPN instances, and RDs must be unique. |
14:1 12:3 (VPN instance) |
RD of CE6851HI-5 and CE6851HI-6 |
Distributed gateways must be configured with RDs of EVPN and VPN instances, and RDs must be unique. |
15:1 12:4 (VPN instance) |
RD of Exit-Gateway-CE12808-1 |
Distributed gateways must be configured with RDs of EVPN and VPN instances, and RDs must be unique. |
16:1 |
RD of Exit-Gateway-CE12808-2 |
Distributed gateways must be configured with RDs of EVPN and VPN instances, and RDs must be unique. |
17:1 |
The following table describes the RT planning of the solution.
Device |
Suggestion |
Example |
|---|---|---|
RT of CE6851HI-1 and CE6851HI-2 |
Distributed gateways must be configured with RTs of EVPN and VPN instances. |
1:1 11:1 (outbound RT of an EVPN instance, which is used for interworking with a VPN instance) 11:1 (RT of a VPN instance, which is used for interworking with an EVPN instance) |
RT of CE6851HI-3 |
Distributed gateways must be configured with RTs of EVPN and VPN instances. |
1:1 11:1 (outbound RT of an EVPN instance, which is used for interworking with a VPN instance) 11:1 (RT of a VPN instance, which is used for interworking with an EVPN instance) |
RT of CE6851HI-4 |
Distributed gateways must be configured with RTs of EVPN and VPN instances. |
1:1 11:1 (outbound RT of an EVPN instance, which is used for interworking with a VPN instance) 11:1 (RT of a VPN instance, which is used for interworking with an EVPN instance) |
RT of CE6851HI-5 and CE6851HI-6 |
Distributed gateways must be configured with RTs of EVPN and VPN instances. |
1:1 11:1 (outbound RT of an EVPN instance, which is used for interworking with a VPN instance) 11:1 (RT of a VPN instance, which is used for interworking with an EVPN instance) |
RT of Exit-Gateway-CE12808-1 |
Distributed gateways must be configured with RTs of EVPN and VPN instances. |
1:1 11:1 (outbound RT of an EVPN instance, which is used for interworking with a VPN instance) 11:1 (RT of a VPN instance, which is used for interworking with an EVPN instance) |
RT of Exit-Gateway-CE12808-2 |
Distributed gateways must be configured with RTs of EVPN and VPN instances. |
1:1 11:1 (outbound RT of an EVPN instance, which is used for interworking with a VPN instance) 11:1 (RT of a VPN instance, which is used for interworking with an EVPN instance) |
Figure 2 shows the RT configuration of VPN and EVPN instances.
- In the VPN instance, in addition to local VPN instances of ERT X and IRT X, you need to configure ERT Y and IRT Y with EVPN. They are used with EVPN instances to generate host routes.
- In an EVPN instance, in addition ERT A, ERT B, IRT A, and IRT B for different BDs, you need to configure ERT Y that is used with a VPN instance. Generally, IRT Y does not need to be configured. Otherwise, MAC addresses will be advertised in EVPN instances of different BDs.
IP Address Planning
The following table lists NE interface address planning, including planning of interconnection network segment addresses, VTEP addresses, BGP Router-IDs, loopback addresses for BGP peer setup, M-LAG heartbeat detection addresses, and service network segment addresses.
The following table describes IP address planning for interface interconnection.
Planning Description |
Local Device |
Remote Device |
Network Segment Address |
|---|---|---|---|
Configure interface addresses for connecting the leaf nodes (Leaf-CE6851HI-1 and Leaf-CE6851HI-2) and egress gateways. |
Leaf-CE6851HI-1& CE6851HI-2 |
Exit-Gateway-CE12808-1 |
192.168.40.156/30 192.168.40.168/30 |
Exit-Gateway-CE12808-2 |
192.168.40.164/30 192.168.40.160/30 |
||
Configure interface addresses for connecting the leaf nodes (Leaf-CE6851HI-3 and Leaf-CE6851HI-4) and egress gateways. |
Leaf-CE6851HI-3& CE6851HI-4 |
Exit-Gateway-CE12808-1 |
192.168.41.156/30 192.168.41.168/30 |
Exit-Gateway-CE12808-2 |
192.168.41.164/30 192.168.41.160/30 |
||
Configure interface addresses for connecting the leaf nodes (Leaf-CE6851HI-5 and Leaf-CE6851HI-6) and egress gateways. |
Leaf-CE6851HI-5& CE6851HI-6 |
Exit-Gateway-CE12808-1 |
192.168.46.156/30 192.168.46.168/30 |
Exit-Gateway-CE12808-2 |
192.168.46.164/30 192.168.46.160/30 |
||
Configure interface addresses for connecting Exit-Gateway-CE12808-1 and Router-1. |
Exit-Gateway-CE12808-1 |
Router-1 |
192.168.44.156/30 |
Configure interface addresses for connecting Exit-Gateway-CE12808-2 and Router-2. |
Exit-Gateway-CE12808-2 |
Router-2 |
192.168.44.160/30 |
Configure interface addresses for connecting Exit-Gateway-CE12808-1 and Exit-Gateway-CE12808-2. |
Exit-Gateway-CE12808-1 |
Exit-Gateway-CE12808-2 |
192.168.44.164/30 |
Configure interface addresses for connecting the gateways (Exit-Gateway-CE12808-1 and Exit-Gateway-CE12808-2) and firewalls. |
Exit-Gateway-CE12808-1 Exit-Gateway-CE12808-2 |
FW-USG9560-1 FW-USG9560-2 |
192.168.45.152/29 Virtual IP address: 192.168.45.153 192.168.45.160/29 Virtual IP address: 192.168.45.161 |
The following table describes planning for loopback addresses.
Planning Description |
Device |
IP Address/Mask |
|---|---|---|
NOTE:
Configure Loopback0 addresses as VTEP IP addresses. NOTE:
Leaf-CE6851HI-3 and Leaf-CE6851HI-4 set up an M-LAG to dual-home servers. Therefore, set the VTEP IP addresses of the two leaf nodes to the same. Exit-Gateway-CE12808-1 and Exit-Gateway-CE12808-2 set up an M-LAG to dual-home firewalls. Therefore, set the VTEP IP addresses of the two gateways to the same. |
Leaf-CE6851HI-1& CE6851HI-2 |
11.11.11.11/32 |
Leaf-CE6851HI-3 |
11.11.11.12/32 |
|
Leaf-CE6851HI-4 |
Same as that of Leaf-CE6851HI-3: 11.11.11.12/32 |
|
Leaf-CE6851HI-5& CE6851HI-6 |
11.11.11.17/32 |
|
Exit-Gateway-CE12808-1 |
11.11.11.16/32 |
|
Exit-Gateway-CE12808-2 |
Same as that of Exit-Gateway-CE12808-1: 11.11.11.16/32 |
|
Configure Loopback1 addresses as M-LAG heartbeat detection addresses. |
Leaf-CE6851HI-3 |
13.13.13.13/32 |
Leaf-CE6851HI-4 |
14.14.14.14/32 |
|
Exit-Gateway-CE12808-1 |
18.18.18.18/32 |
|
Exit-Gateway-CE12808-2 |
19.19.19.19/32 |
|
Configure Loopback2 addresses as the EBGP peer addresses for connecting to the remote routers. |
Exit-Gateway-CE12808-1 |
21.21.21.21/32 |
Exit-Gateway-CE12808-2 |
22.22.22.22/32 |
The following table describes planning for service addresses.
Tenant |
IP Address/Mask of the Tenant Network Segment |
Address of the Gateway VBDIF Interface |
VRF |
|---|---|---|---|
Server 1 |
192.168.10.2/24 |
192.168.10.1/24 |
VPN 1 |
Server 2 |
192.168.10.3/24 |
192.168.10.1/24 |
VPN 1 |
Server 3 |
192.168.10.4/24 |
192.168.10.1/24 |
VPN 1 |
Route Planning
EBGP and OSPF are common routing protocols of underlay networks. BGP makes networks secure, flexible, stable, reliable, and efficient from the following aspects:
- Uses authentication and the Generalized TTL Security Mechanism (GTSM) to ensure network security. TTL refers to time to live.
- Provides various routing policies, enabling flexible routing.
- Offers route aggregation and route dampening functions to prevent route flapping, enhancing network stability.
- Uses the Transmission Control Protocol (TCP) with the port number 179 as the transport layer protocol and supports association with Bidirectional Forwarding Detection (BFD), as well as Auto Fast Reroute (FRR), Graceful Restart (GR), and Non-Stop Routing (NSR), enhancing network reliability.
In network evolution, EBGP applies to large-sized networks, and OSPF applies to middle- and small-sized networks. In this document, EBGP is used for the network, and OSPF routes are used for connecting the border leaf nodes to routers through IGP. In practical operations, Intermediate System-Intermediate System (IS-IS) or IBGP can be used.
The following table describes route planning.
NE |
AS Domain Number |
Router-ID |
|---|---|---|
Leaf-CE6851HI-1&CE6851-2 |
65021 |
Loopback0 address |
Leaf-CE6851HI-3 |
65022 |
Loopback1 address |
Leaf-CE6851HI-4 |
65022 |
Loopback1 address |
Leaf-CE6851HI-5&CE6851HI-6 |
65024 |
Loopback0 address |
Exit-Gateway-CE12808-1 |
65000 |
Loopback1 address |
Exit-Gateway-CE12808-2 |
65000 |
Loopback1 address |
Router-1 |
65047 |
Loopback0 address |
Router-2 |
65048 |
Loopback0 address |
Configuring Interface Addresses for Connecting Gateways and Leaf Nodes
In this solution, spine nodes and gateways are deployed on the same devices. Therefore, connect the gateways to leaf nodes directly.
- Configure IP addresses for interfaces on Exit-Gateway-CE12808-1.
[~Huawei] sysname Exit-Gateway-CE12808-1 [*Huawei] commit [~Exit-Gateway-CE12808-1] interface 40ge 1/0/0 [*Exit-Gateway-CE12808-1-40GE1/0/0] description "to-Leaf-CE6851HI-1&CE6851HI-2" [~Exit-Gateway-CE12808-1-40GE1/0/0] undo portswitch [*Exit-Gateway-CE12808-1-40GE1/0/0] ip address 192.168.40.158 30 [*Exit-Gateway-CE12808-1-40GE1/0/0] commit [~Exit-Gateway-CE12808-1-40GE1/0/0] quit [~Exit-Gateway-CE12808-1] interface 40ge 1/0/1 [*Exit-Gateway-CE12808-1-40GE1/0/1] description "to-Leaf-CE6851HI-1&CE6851HI-2" [*Exit-Gateway-CE12808-1-40GE1/0/1] undo portswitch [*Exit-Gateway-CE12808-1-40GE1/0/1] ip address 192.168.40.170 30 [*Exit-Gateway-CE12808-1-40GE1/0/1] commit [~Exit-Gateway-CE12808-1-40GE1/0/1] quit [~Exit-Gateway-CE12808-1] interface 40ge 1/0/2 [~Exit-Gateway-CE12808-1-40GE1/0/2] description "to-Leaf-CE6851-3-40GE1/0/3" [*Exit-Gateway-CE12808-1-40GE1/0/2] undo portswitch [*Exit-Gateway-CE12808-1-40GE1/0/2] ip address 192.168.41.158 30 [*Exit-Gateway-CE12808-1-40GE1/0/2] commit [~Exit-Gateway-CE12808-1-40GE1/0/2] quit [~Exit-Gateway-CE12808-1] interface 40ge 1/0/3 [*Exit-Gateway-CE12808-1-40GE1/0/3] description "to-Leaf-CE6851-4-40GE1/0/4" [*Exit-Gateway-CE12808-1-40GE1/0/3] undo portswitch [*Exit-Gateway-CE12808-1-40GE1/0/3] ip address 192.168.41.170 30 [*Exit-Gateway-CE12808-1-40GE1/0/3] commit [~Exit-Gateway-CE12808-1-40GE1/0/3] quit [~Exit-Gateway-CE12808-1] interface 40ge 1/0/6 [~Exit-Gateway-CE12808-1-40GE1/0/6] undo portswitch [*Exit-Gateway-CE12808-1-40GE1/0/6] ip address 192.168.46.158 30 [*Exit-Gateway-CE12808-1-40GE1/0/6] commit [~Exit-Gateway-CE12808-1-40GE1/0/6] quit [~Exit-Gateway-CE12808-1] interface 40ge 1/0/7 [~Exit-Gateway-CE12808-1-40GE1/0/7] undo portswitch [*Exit-Gateway-CE12808-1-40GE1/0/7] ip address 192.168.46.170 30 [*Exit-Gateway-CE12808-1-40GE1/0/7] commit [~Exit-Gateway-CE12808-1-40GE1/0/7] quit
- Configure IP addresses for interfaces on Exit-Gateway-CE12808-2.
[~Huawei] sysname Exit-Gateway-CE12808-2 [*Huawei] commit [~Exit-Gateway-CE12808-2] interface 40ge 1/0/0 [~Exit-Gateway-CE12808-2-40GE1/0/0] undo portswitch [*Exit-Gateway-CE12808-2-40GE1/0/0] ip address 192.168.40.162 30 [*Exit-Gateway-CE12808-2-40GE1/0/0] commit [~Exit-Gateway-CE12808-2-40GE1/0/0] quit [~Exit-Gateway-CE12808-2] interface 40ge 1/0/1 [~Exit-Gateway-CE12808-2-40GE1/0/1] undo portswitch [*Exit-Gateway-CE12808-2-40GE1/0/1] ip address 192.168.40.166 30 [*Exit-Gateway-CE12808-2-40GE1/0/1] commit [~Exit-Gateway-CE12808-2-40GE1/0/1] quit [~Exit-Gateway-CE12808-2] interface 40ge 1/0/2 [~Exit-Gateway-CE12808-2-40GE1/0/2] description "to-Leaf-CE6851-4-40GE1/0/3" [*Exit-Gateway-CE12808-2-40GE1/0/2] undo portswitch [*Exit-Gateway-CE12808-2-40GE1/0/2] ip address 192.168.41.162 30 [*Exit-Gateway-CE12808-2-40GE1/0/2] commit [~Exit-Gateway-CE12808-2-40GE1/0/2] quit [~Exit-Gateway-CE12808-2] interface 40ge 1/0/3 [~Exit-Gateway-CE12808-2-40GE1/0/3] description "to-Leaf-CE6851-3-40GE1/0/4" [*Exit-Gateway-CE12808-2-40GE1/0/3] undo portswitch [*Exit-Gateway-CE12808-2-40GE1/0/3] ip address 192.168.41.166 30 [*Exit-Gateway-CE12808-2-40GE1/0/3] commit [~Exit-Gateway-CE12808-2-40GE1/0/3] quit [~Exit-Gateway-CE12808-2] interface 40ge 1/0/6 [~Exit-Gateway-CE12808-2-40GE1/0/6] undo portswitch [*Exit-Gateway-CE12808-2-40GE1/0/6] ip address 192.168.46.166 30 [*Exit-Gateway-CE12808-2-40GE1/0/6] commit [~Exit-Gateway-CE12808-2-40GE1/0/6] quit [~Exit-Gateway-CE12808-2] interface 40ge 1/0/7 [~Exit-Gateway-CE12808-2-40GE1/0/7] undo portswitch [*Exit-Gateway-CE12808-2-40GE1/0/7] ip address 192.168.46.162 30 [*Exit-Gateway-CE12808-2-40GE1/0/7] commit [~Exit-Gateway-CE12808-2-40GE1/0/7] quit
Configuring Routes on Leaf Nodes
In this solution, spine nodes and gateways are deployed on the same devices. Therefore, the BGP route configuration on leaf nodes is different from that in the three-layer architecture.
- Configure BGP routes on the stack to connect it to gateways.
[~Leaf-CE6851HI-1&CE6851HI-2] bgp 65021 [*Leaf-CE6851HI-1&CE6851HI-2-bgp] router-id 11.11.11.11 [*Leaf-CE6851HI-1&CE6851HI-2-bgp] timer keepalive 10 hold 30 [*Leaf-CE6851HI-1&CE6851HI-2-bgp] group Gateway-CE12808 external [*Leaf-CE6851HI-1&CE6851HI-2-bgp] peer Gateway-CE12808 as-number 65000 [*Leaf-CE6851HI-1&CE6851HI-2-bgp] peer 192.168.40.158 as-number 65000 [*Leaf-CE6851HI-1&CE6851HI-2-bgp] peer 192.168.40.158 group Gateway-CE12808 [*Leaf-CE6851HI-1&CE6851HI-2-bgp] peer 192.168.40.170 as-number 65000 [*Leaf-CE6851HI-1&CE6851HI-2-bgp] peer 192.168.40.170 group Gateway-CE12808 [*Leaf-CE6851HI-1&CE6851HI-2-bgp] peer 192.168.40.166 as-number 65000 [*Leaf-CE6851HI-1&CE6851HI-2-bgp] peer 192.168.40.166 group Gateway-CE12808 [*Leaf-CE6851HI-1&CE6851HI-2-bgp] peer 192.168.40.162 as-number 65000 [*Leaf-CE6851HI-1&CE6851HI-2-bgp] peer 192.168.40.162 group Gateway-CE12808 [*Leaf-CE6851HI-1&CE6851HI-2-bgp] ipv4-family unicast [*Leaf-CE6851HI-1&CE6851HI-2-bgp-af-ipv4] preference 20 200 10 [*Leaf-CE6851HI-1&CE6851HI-2-bgp-af-ipv4] network 11.11.11.11 255.255.255.255 [*Leaf-CE6851HI-1&CE6851HI-2-bgp-af-ipv4] maximum load-balancing 32 [*Leaf-CE6851HI-1&CE6851HI-2-bgp-af-ipv4] quit [*Leaf-CE6851HI-1&CE6851HI-2-bgp] quit [*Leaf-CE6851HI-1&CE6851HI-2] commit
- Configure BGP routes on Leaf-CE6851HI-3 to connect it to gateways.
[~Leaf-CE6851HI-3] bgp 65022 [*Leaf-CE6851HI-3-bgp] router-id 13.13.13.13 [*Leaf-CE6851HI-3-bgp] timer keepalive 10 hold 30 [*Leaf-CE6851HI-3-bgp] group Gateway-CE12808 external [*Leaf-CE6851HI-3-bgp] peer Gateway-CE12808 as-number 65000 [*Leaf-CE6851HI-3-bgp] peer 192.168.41.158 as-number 65000 [*Leaf-CE6851HI-3-bgp] peer 192.168.41.158 group Gateway-CE12808 [*Leaf-CE6851HI-3-bgp] peer 192.168.41.166 as-number 65000 [*Leaf-CE6851HI-3-bgp] peer 192.168.41.166 group Gateway-CE12808 [*Leaf-CE6851HI-3-bgp] ipv4-family unicast [*Leaf-CE6851HI-3-bgp-af-ipv4] preference 20 200 10 [*Leaf-CE6851HI-3-bgp-af-ipv4] network 11.11.11.12 255.255.255.255 [*Leaf-CE6851HI-3-bgp-af-ipv4] network 10.10.100.0 255.255.255.0 [*Leaf-CE6851HI-3-bgp-af-ipv4] network 13.13.13.13 255.255.255.255 [*Leaf-CE6851HI-3-bgp-af-ipv4] maximum load-balancing 32 [*Leaf-CE6851HI-3-bgp-af-ipv4] quit [*Leaf-CE6851HI-3-bgp] quit [*Leaf-CE6851HI-3] commit
- Configure BGP routes on Leaf-CE6851HI-4 to connect it to gateways.
[~Leaf-CE6851HI-4] bgp 65022 [*Leaf-CE6851HI-4-bgp] router-id 14.14.14.14 [*Leaf-CE6851HI-4-bgp] timer keepalive 10 hold 30 [*Leaf-CE6851HI-4-bgp] group Gateway-CE12808 external [*Leaf-CE6851HI-4-bgp] peer Gateway-CE12808 as-number 65000 [*Leaf-CE6851HI-4-bgp] peer 192.168.41.170 as-number 65000 [*Leaf-CE6851HI-4-bgp] peer 192.168.41.170 group Gateway-CE12808 [*Leaf-CE6851HI-4-bgp] peer 192.168.41.162 as-number 65000 [*Leaf-CE6851HI-4-bgp] peer 192.168.41.162 group Gateway-CE12808 [*Leaf-CE6851HI-4-bgp] ipv4-family unicast [*Leaf-CE6851HI-4-bgp-af-ipv4] preference 20 200 10 [*Leaf-CE6851HI-4-bgp-af-ipv4] network 11.11.11.12 255.255.255.255 [*Leaf-CE6851HI-4-bgp-af-ipv4] network 10.10.100.0 255.255.255.0 [*Leaf-CE6851HI-4-bgp-af-ipv4] network 14.14.14.14 255.255.255.255 [*Leaf-CE6851HI-4-bgp-af-ipv4] maximum load-balancing 32 [*Leaf-CE6851HI-4-bgp-af-ipv4] quit [*Leaf-CE6851HI-4-bgp] quit [*Leaf-CE6851HI-4] commit
- Configure BGP routes on the SVF system to connect the system to gateways.
[~Leaf-CE6851HI-5&CE6851HI-6] bgp 65024 [*Leaf-CE6851HI-5&CE6851HI-6-bgp] router-id 11.11.11.17 [*Leaf-CE6851HI-5&CE6851HI-6-bgp] timer keepalive 10 hold 30 [*Leaf-CE6851HI-5&CE6851HI-6-bgp] group Gateway-CE12808 external [*Leaf-CE6851HI-5&CE6851HI-6-bgp] peer Gateway-CE12808 as-number 65000 [*Leaf-CE6851HI-5&CE6851HI-6-bgp] peer 192.168.46.158 as-number 65000 [*Leaf-CE6851HI-5&CE6851HI-6-bgp] peer 192.168.46.158 group_Gateway-CE12808 [*Leaf-CE6851HI-5&CE6851HI-6-bgp] peer 192.168.46.170 as-number 65000 [*Leaf-CE6851HI-5&CE6851HI-6-bgp] peer 192.168.46.170 group Gateway-CE12808 [*Leaf-CE6851HI-5&CE6851HI-6-bgp] peer 192.168.46.166 as-number 65000 [*Leaf-CE6851HI-5&CE6851HI-6-bgp] peer 192.168.46.166 group Gateway-CE12808 [*Leaf-CE6851HI-5&CE6851HI-6-bgp] peer 192.168.46.162 as-number 65000 [*Leaf-CE6851HI-5&CE6851HI-6-bgp] peer 192.168.46.162 group Gateway-CE12808 [*Leaf-CE6851HI-5&CE6851HI-6-bgp] ipv4-family unicast [*Leaf-CE6851HI-5&CE6851HI-6-bgp-af-ipv4] preference 20 200 10 [*Leaf-CE6851HI-5&CE6851HI-6-bgp-af-ipv4] network 11.11.11.17 255.255.255.255 [*Leaf-CE6851HI-5&CE6851HI-6-bgp-af-ipv4] maximum load-balancing 32 [*Leaf-CE6851HI-5&CE6851HI-6-bgp-af-ipv4] quit [*Leaf-CE6851HI-5&CE6851HI-6-bgp] quit [*Leaf-CE6851HI-5&CE6851HI-6] commit
Configuring Routes on Gateways
In this solution, spine nodes and gateways are deployed on the same devices. Therefore, the BGP route configuration on gateways is different from that in the three-layer architecture.
- Configure BGP routes on Exit-Gateway-CE12808-1.
[~Exit-Gateway-CE12808-1] bgp 65000 [*Exit-Gateway-CE12808-1-bgp] router-id 18.18.18.18 [*Exit-Gateway-CE12808-1-bgp] timer keepalive 10 hold 30 [*Exit-Gateway-CE12808-1-bgp] group Router-1 external //Configure a route to the egress router. [*Exit-Gateway-CE12808-1-bgp] peer Router-1 as-number 65047 [*Exit-Gateway-CE12808-1-bgp] peer Router-1 ebgp-max-hop 10 [*Exit-Gateway-CE12808-1-bgp] peer Router-1 connect-interface loopback 2 [*Exit-Gateway-CE12808-1-bgp] peer 21.21.21.22 as-number 65047 [*Exit-Gateway-CE12808-1-bgp] peer 21.21.21.22 group Router-1 [*Exit-Gateway-CE12808-1-bgp] group Leaf-CE6851HI-1&CE6851HI-2 external //Configure routes to the leaf nodes. [*Exit-Gateway-CE12808-1-bgp] peer Leaf-CE6851HI-1&CE6851HI-2 as-number 65021 [*Exit-Gateway-CE12808-1-bgp] peer 192.168.40.157 as-number 65021 [*Exit-Gateway-CE12808-1-bgp] peer 192.168.40.157 group Leaf-CE6851HI-1&CE6851HI-2 [*Exit-Gateway-CE12808-1-bgp] peer 192.168.40.169 as-number 65021 [*Exit-Gateway-CE12808-1-bgp] peer 192.168.40.169 group Leaf-CE6851HI-1&CE6851HI-2 [*Exit-Gateway-CE12808-1-bgp] group Leaf-CE6851HI-3&4 external [*Exit-Gateway-CE12808-1-bgp] peer Leaf-CE6851HI-3&4 as-number 65022 [*Exit-Gateway-CE12808-1-bgp] peer 192.168.41.157 as-number 65022 [*Exit-Gateway-CE12808-1-bgp] peer 192.168.41.157 group Leaf-CE6851HI-3&4 [*Exit-Gateway-CE12808-1-bgp] peer 192.168.41.169 as-number 65022 [*Exit-Gateway-CE12808-1-bgp] peer 192.168.41.169 group Leaf-CE6851HI-3&4 [*Exit-Gateway-CE12808-1-bgp] group Leaf-CE6851HI-5&CE6851HI-6 external [*Exit-Gateway-CE12808-1-bgp] peer Leaf-CE6851HI-5&CE6851HI-6 as-number 65024 [*Exit-Gateway-CE12808-1-bgp] peer 192.168.46.157 as-number 65024 [*Exit-Gateway-CE12808-1-bgp] peer 192.168.46.157 group Leaf-CE6851HI-5&CE6851HI-6 [*Exit-Gateway-CE12808-1-bgp] peer 192.168.46.169 as-number 65024 [*Exit-Gateway-CE12808-1-bgp] peer 192.168.46.169 group Leaf-CE6851HI-5&CE6851HI-6 [*Exit-Gateway-CE12808-1-bgp] ipv4-family unicast [*Exit-Gateway-CE12808-1-bgp-af-ipv4] preference 20 200 10 [*Exit-Gateway-CE12808-1-bgp-af-ipv4] network 11.11.11.16 255.255.255.255 [*Exit-Gateway-CE12808-1-bgp-af-ipv4] network 18.18.18.18 255.255.255.255 [*Exit-Gateway-CE12808-1-bgp-af-ipv4] network 21.21.21.21 255.255.255.255 [*Exit-Gateway-CE12808-1-bgp-af-ipv4] network 192.168.44.156 255.255.255.252 [*Exit-Gateway-CE12808-1-bgp-af-ipv4] network 192.168.45.152 255.255.255.248 [*Exit-Gateway-CE12808-1-bgp-af-ipv4] maximum load-balancing 32 [*Exit-Gateway-CE12808-1-bgp-af-ipv4] quit [*Exit-Gateway-CE12808-1-bgp] quit [*Exit-Gateway-CE12808-1] commit
- Configure BGP routes on Exit-Gateway-CE12808-2.
[~Exit-Gateway-CE12808-2] bgp 65000 [*Exit-Gateway-CE12808-2-bgp] router-id 19.19.19.19 [*Exit-Gateway-CE12808-2-bgp] timer keepalive 10 hold 30 [*Exit-Gateway-CE12808-2-bgp] group Router-2 external //Configure a route to the egress router. [*Exit-Gateway-CE12808-2-bgp] peer Router-2 as-number 65048 [*Exit-Gateway-CE12808-2-bgp] peer Router-2 ebgp-max-hop 10 [*Exit-Gateway-CE12808-2-bgp] peer Router-2 connect-interface loopback 2 [*Exit-Gateway-CE12808-2-bgp] peer 22.22.22.23 as-number 65048 [*Exit-Gateway-CE12808-2-bgp] peer 22.22.22.23 group Router-2 [*Exit-Gateway-CE12808-2-bgp] group Leaf-CE6851HI-1&CE6851HI-2 external [*Exit-Gateway-CE12808-2-bgp] peer Leaf-CE6851HI-1&CE6851HI-2 as-number 65021 [*Exit-Gateway-CE12808-2-bgp] peer 192.168.40.165 as-number 65021 [*Exit-Gateway-CE12808-2-bgp] peer 192.168.40.165 group Leaf-CE6851HI-1&CE6851HI-2 [*Exit-Gateway-CE12808-2-bgp] peer 192.168.40.161 as-number 65021 [*Exit-Gateway-CE12808-2-bgp] peer 192.168.40.161 group Leaf-CE6851HI-1&CE6851HI-2 [*Exit-Gateway-CE12808-2-bgp] group Leaf-CE6851HI-3&4 external [*Exit-Gateway-CE12808-2-bgp] peer Leaf-CE6851HI-3&4 as-number 65022 [*Exit-Gateway-CE12808-2-bgp] peer 192.168.41.165 as-number 65022 [*Exit-Gateway-CE12808-2-bgp] peer 192.168.41.165 group Leaf-CE6851HI-3&4 [*Exit-Gateway-CE12808-2-bgp] peer 192.168.41.161 as-number 65022 [*Exit-Gateway-CE12808-2-bgp] peer 192.168.41.161 group Leaf-CE6851HI-3&4 [*Exit-Gateway-CE12808-2-bgp] group Leaf-CE6851HI-5&CE6851HI-6 external [*Exit-Gateway-CE12808-2-bgp] peer Leaf-CE6851HI-5&CE6851HI-6 as-number 65024 [*Exit-Gateway-CE12808-2-bgp] peer 192.168.46.165 as-number 65024 [*Exit-Gateway-CE12808-2-bgp] peer 192.168.46.165 group Leaf-CE6851HI-5&CE6851HI-6 [*Exit-Gateway-CE12808-2-bgp] peer 192.168.46.161 as-number 65024 [*Exit-Gateway-CE12808-2-bgp] peer 192.168.46.161 group Leaf-CE6851HI-5&CE6851HI-6 [*Exit-Gateway-CE12808-2-bgp] ipv4-family unicast [*Exit-Gateway-CE12808-2-bgp-af-ipv4] preference 20 200 10 [*Exit-Gateway-CE12808-2-bgp-af-ipv4] network 11.11.11.16 255.255.255.255 [*Exit-Gateway-CE12808-2-bgp-af-ipv4] network 19.19.19.19 255.255.255.255 [*Exit-Gateway-CE12808-2-bgp-af-ipv4] network 22.22.22.22 255.255.255.255 [*Exit-Gateway-CE12808-2-bgp-af-ipv4] network 192.168.44.160 255.255.255.252 [*Exit-Gateway-CE12808-2-bgp-af-ipv4] network 192.168.45.152 255.255.255.248 [*Exit-Gateway-CE12808-2-bgp-af-ipv4] maximum load-balancing 32 [*Exit-Gateway-CE12808-2-bgp-af-ipv4] quit [*Exit-Gateway-CE12808-2-bgp] quit [*Exit-Gateway-CE12808-2] commit
Configuring BGP EVPN
In this solution, the spine node and gateway are converged. When BGP EVPN is deployed in the VXLAN control plane, the egress gateway and leaf node need to establish a peer relationship. However, the peer relationship between the egress gateway and spine node does not need to be established.
- On exit gateways, configure EVPN as the VXLAN control panel. The configurations of other devices are similar and not mentioned here.
[~Exit-Gateway-CE12808-1] evpn-overlay enable [~Exit-Gateway-CE12808-1] commit
- Establish the BGP EVPN peer relationship between exit gateways and leaf nodes.
# Establish the IBGP EVPN peer relationship between the Exit-Gateway-CE12808-1 and leaf nodes. The configurations of other devices are similar to the Exit-Gateway-CE12808-1, and are not mentioned here.
[~Exit-Gateway-CE12808-1] bgp 100 instance evpn1 [*Exit-Gateway-CE12808-1-bgp-instance-evpn1] router-id 18.18.18.18 [*Exit-Gateway-CE12808-1-bgp-instance-evpn1] peer 11.11.11.11 as-number 100 //Establish a BGP EVPN peer relationship with switches in the stack. [*Exit-Gateway-CE12808-1-bgp-instance-evpn1] peer 11.11.11.11 connect-interface loopback 0 [*Exit-Gateway-CE12808-1-bgp-instance-evpn1] peer 13.13.13.13 as-number 100 //Establish a BGP EVPN peer relationship with Leaf-CE6851HI-3. [*Exit-Gateway-CE12808-1-bgp-instance-evpn1] peer 13.13.13.13 connect-interface loopback 0 [*Exit-Gateway-CE12808-1-bgp-instance-evpn1] peer 14.14.14.14 as-number 100 //Establish a BGP EVPN peer relationship with Leaf-CE6851HI-4. [*Exit-Gateway-CE12808-1-bgp-instance-evpn1] peer 14.14.14.14 connect-interface loopback 0 [*Exit-Gateway-CE12808-1-bgp-instance-evpn1] peer 11.11.11.17 as-number 100 //Establish the BGP EVPN peer relationship with the SVF. [*Exit-Gateway-CE12808-1-bgp-instance-evpn1] peer 11.11.11.17 connect-interface loopback 0 [*Exit-Gateway-CE12808-1-bgp-instance-evpn1] l2vpn-family evpn [*Exit-Gateway-CE12808-1-bgp-instance-evpn1-af-evpn] peer 11.11.11.11 enable [*Exit-Gateway-CE12808-1-bgp-instance-evpn1-af-evpn] peer 13.13.13.13 enable [*Exit-Gateway-CE12808-1-bgp-instance-evpn1-af-evpn] peer 14.14.14.14 enable [*Exit-Gateway-CE12808-1-bgp-instance-evpn1-af-evpn] peer 11.11.11.17 enable [*Exit-Gateway-CE12808-1-bgp-instance-evpn1-af-evpn] quit [*Exit-Gateway-CE12808-1-bgp-instance-evpn1] quit [*Exit-Gateway-CE12808-1] commit