No relevant resource is found in the selected language.

This site uses cookies. By continuing to browse the site you are agreeing to our use of cookies. Read our privacy policy>Search

Reminder

To have a better experience, please upgrade your IE browser.

upgrade

CloudEngine 12800, 12800E, 8800, 7800, 6800, and 5800 Series Switches VXLAN Best Practices

Rate and give feedback :
Huawei uses machine translation combined with human proofreading to translate this document to different languages in order to help you better understand the content of this document. Note: Even the most advanced machine translation cannot match the quality of professional translators. Huawei shall not bear any responsibility for translation accuracy and it is recommended that you refer to the English document (a link for which has been provided).
Hardware Distributed VXLAN Using the Gateway/Spine/Leaf Three-Layer Architecture

Hardware Distributed VXLAN Using the Gateway/Spine/Leaf Three-Layer Architecture

Networking Requirements

Figure 1 shows the hardware distributed VXLAN using the gateway/spine/leaf three-layer architecture.

Figure 3-3 Hardware distributed VXLAN using the gateway/spine/leaf three-layer architecture

  • Server layer: Servers are connected to the VXLAN network through Layer 2 sub-interfaces.
  • Leaf nodes (distributed gateways, also called east-west gateways): Servers are connected to leaf nodes through stacking, Multi-Chassis Link Aggregation Group (M-LAG), or super virtual fabric (SVF). Leaf nodes and spine nodes communicate at Layer 3. A stack, M-LAG, or SVF system consisting of leaf nodes functions as a virtual tunnel end point (VTEP) to allow server traffic to access the VXLAN.
  • Spine nodes: A spine node connects to leaf nodes and gateways. Routing protocols are used to ensure that underlay routes are reachable at Layer 3. The spine node does not function as a VTEP.
  • Border leaf nodes (Layer 3 egress gateways, also called north-south gateways): Two switches set up an M-LAG and serve dual-active gateways. Border leaf nodes and spine nodes communicate at Layer 3 and connect to external routers Router-1 and Router-2.
  • Firewalls: Two hardware firewalls are connected to the Layer 3 egress gateways in bypass mode.
  • LB: Load balancers (LBs) are deployed by manufacturers.

Port Connection Planning

Deploy Leaf-CE6851HI-1 and Leaf-CE6851HI-2 as a stack. The port planning is as follows.

Planning Description

Local Device

Port

Remote Device

Port

Configure leaf node interconnection ports as stack ports and at least two stack links.

Leaf-CE6851HI-1

40GE1/0/1-2

Leaf-CE6851HI-2

40GE1/0/1-2

Leaf-CE6851HI-2

40GE1/0/1-2

Leaf-CE6851HI-1

40GE1/0/1-2

Configure ports on the leaf nodes and spine nodes for interconnection.

Leaf-CE6851HI-1

40GE1/0/3

Spine-CE12804-1

40GE1/0/0

Leaf-CE6851HI-1

40GE1/0/4

Spine-CE12804-2

40GE1/0/1

Leaf-CE6851HI-2

40GE1/0/3

Spine-CE12804-2

40GE1/0/0

Leaf-CE6851HI-2

40GE1/0/4

Spine-CE12804-1

40GE1/0/1

Configure ports on the leaf nodes and servers for interconnection. Connect the servers with dual NICs to the leaf nodes in load balancing mode to ensure link reliability.

Leaf-CE6851HI-1

10GE1/0/1-2

Server

Eth0

Leaf-CE6851HI-2

10GE1/0/1-2

Server

Eth1

Deploy Leaf-CE6851HI-3 and Leaf-CE6851HI-4 as an M-LAG. The port planning is as follows.

Planning Description

Local Device

Port

Remote Device

Port

Configure peer-link ports of the M-LAG to transmit protocol packets, as well as data packets when faults occur. Configure at least two member links for the peer-link to ensure reliability.

Leaf-CE6851HI-3

40GE1/0/1-2

Leaf-CE6851HI-4

40GE1/0/1-2

Leaf-CE6851HI-4

40GE1/0/1-2

Leaf-CE6851HI-3

40GE1/0/1-2

Configure ports on the leaf nodes and spine nodes for interconnection.

Leaf-CE6851HI-3

40GE1/0/3

Spine-CE12804-1

40GE1/0/2

Leaf-CE6851HI-3

40GE1/0/4

Spine-CE12804-2

40GE1/0/3

Leaf-CE6851HI-4

40GE1/0/3

Spine-CE12804-2

40GE1/0/2

Leaf-CE6851HI-4

40GE1/0/4

Spine-CE12804-1

40GE1/0/3

Configure ports on the leaf nodes and servers for interconnection. Connect the servers with dual NICs to the leaf nodes in load balancing mode to ensure link reliability and improve link utilization.

Leaf-CE6851HI-3

10GE1/0/1

Server

Eth0

Leaf-CE6851HI-4

10GE1/0/1

Server

Eth1

Deploy Leaf-CE6851HI-5 and Leaf-CE6851HI-5 as an M-LAG. The port planning is as follows.

Planning Description

Local Device

Port

Remote Device

Port

Configure leaf node interconnection ports as stack ports and at least two stack links.

Leaf-CE6851HI-5

40GE1/0/1-2

Leaf-CE6851HI-6

40GE1/0/1-2

Leaf-CE6851HI-6

40GE1/0/1-2

Leaf-CE6851HI-5

40GE1/0/1-2

Configure ports on the leaf nodes and spine nodes for interconnection.

Leaf-CE6851HI-5

40GE1/0/3

Spine-CE12804-1

40GE1/0/6

Leaf-CE6851HI-5

40GE1/0/4

Spine-CE12804-2

40GE1/0/7

Leaf-CE6851HI-6

40GE1/0/3

Spine-CE12804-1

40GE1/0/6

Leaf-CE6851HI-6

40GE1/0/4

Spine-CE12804-2

40GE1/0/7

Configure ports on SVF parent nodes and leaf nodes for interconnection.

Leaf-CE6851HI-5

10GE1/0/1

CE5800-1

GE1/0/1

10GE1/0/2

CE5800-2

GE1/0/1

10GE1/0/3

CE5800-3

GE1/0/1

Leaf-CE6851HI-6

10GE1/0/1

CE5800-1

GE1/0/2

10GE1/0/2

CE5800-2

GE1/0/2

10GE1/0/3

CE5800-3

GE1/0/2

Configure ports on the SVF leaf nodes and servers for interconnection. Connect the servers with dual NICs to the SVF leaf nodes in load balancing mode to ensure link reliability.

CE5810-1

GE1/0/1

Server

Eth0

CE5810-2

GE1/0/1

Server

Eth1

Deploy Spine-CE12804-1 and Spine-CE12804-2. The port planning is as follows.

Planning Description

Local Device

Port

Remote Device

Port

Configure ports on the spine nodes and leaf nodes for interconnection.

Spine-CE12804-1

40GE1/0/0-3

40GE1/0/6-7

Leaf-CE6851HI-1

Leaf-CE6851HI-2

Leaf-CE6851HI-3

Leaf-CE6851HI-4

Leaf-CE6851HI-5

Leaf-CE6851HI-6

40GE1/0/3-4

Spine-CE12804-2

40GE1/0/0-3

40GE1/0/6-7

Leaf-CE6851HI-1

Leaf-CE6851HI-2

Leaf-CE6851HI-3

Leaf-CE6851HI-4

Leaf-CE6851HI-5

Leaf-CE6851HI-6

40GE1/0/3-4

Configure ports on the spine nodes and egress gateways for interconnection.

Spine-CE12804-1

40GE1/0/4-5

Exit-Gateway-CE12808-1

Exit-Gateway-CE12808-2

40GE1/0/0

Spine-CE12804-2

40GE1/0/4-5

Exit-Gateway-CE12808-1

Exit-Gateway-CE12808-2

40GE1/0/1

Deploy Exit-Gateway-CE12808-1 and Exit-Gateway-CE12808-2. The port planning is as follows.

Planning Description

Local Device

Port

Remote Device

Port

Configure ports on the egress gateways and spine nodes for interconnection.

Exit-Gateway-CE12808-1

40GE1/0/0-1

Spine-CE12804-1

Spine-CE12804-2

40GE1/0/4

Exit-Gateway-CE12808-2

40GE1/0/0-1

Spine-CE12804-1

Spine-CE12804-2

40GE1/0/5

Configure peer-link ports of the M-LAG to transmit protocol packets, as well as data packets when faults occur. Configure at least two member links across cards for the peer-link to ensure reliability.

Exit-Gateway-CE12808-1

40GE1/0/23

40GE2/0/23

Exit-Gateway-CE12808-2

40GE1/0/23

40GE2/0/23

Exit-Gateway-CE12808-2

40GE1/0/23

40GE2/0/23

Exit-Gateway-CE12808-1

40GE1/0/23

40GE2/0/23

Configure ports on the egress gateways and firewalls for interconnection.

NOTE:

Two firewalls in active/standby mirroring mode must use the same port to connect to a gateway. For example, if FW-1 connects to GW-1 through GE 1/0/1, GE 1/0/1 must be set for FW-2 to connect to GW-1.

Exit-Gateway-CE12808-1

10GE3/0/0-1

FW-USG9560-1

FW-USG9560-2

GE1/0/1

Exit-Gateway-CE12808-1

10GE3/0/2-3

FW-USG9560-1

FW-USG9560-2

GE1/0/3

Exit-Gateway-CE12808-2

10GE3/0/0-1

FW-USG9560-1

FW-USG9560-2

GE1/0/2

Exit-Gateway-CE12808-2

10GE3/0/2-3

FW-USG9560-1

FW-USG9560-2

GE1/0/4

Configure ports on the egress gateways and egress routers for interconnection.

Exit-Gateway-CE12808-1

10GE3/0/4

Router-1

GE1/0/0

Exit-Gateway-CE12808-1

10GE3/0/5

Router-2

GE1/0/0

Exit-Gateway-CE12808-2

10GE3/0/4

Router-1

GE1/0/1

Exit-Gateway-CE12808-2

10GE3/0/5

Router-2

GE1/0/1

Configure Layer 3 interconnection interfaces between egress gateways, which are used to connect to egress routers in mesh mode.

Exit-Gateway-CE12808-1

10GE3/0/6

Exit-Gateway-CE12808-2

10GE3/0/6

VLAN Planning

The following table describes VLAN planning of the solution.

Planning Description

Suggestion

VLAN ID Example

VLAN for firewall interconnection

You are advised to use Layer 3 main interfaces for Layer 3 interconnection. Firewall interconnection involves interconnection of multiple network segments. Therefore, plan VLANs to share physical links.

11

12 (Create VLANIF 12 and bind it to a VPN.)

Tenant VLAN

Service capacity expansion needs to be considered as many tenants or services use VXLAN.

10

NOTE:

Before switching an interface on the CE6855HI or CE7855EI to Layer 3 mode, run the vlan reserved for main-interface startvlanid to endvlanid command to configure a dedicated reserved VLAN for the Layer 3 main interface.

BD and VNI Planning

The following table describes BD and VNI planning of the solution.

Planning Description

Suggestion

BD and VNI ID Example

BD

Plan the same number of BDs as the number of VLANs. You are advised to use the same BD ID as the VLAN ID.

BD 10 is used here.

VNI

Plan the same number of Layer 2 VNIs as the number of BDs. You are advised to plan the VNI ID as the BD ID plus 10000. A BD corresponds to a VNI.

The number of Layer 3 VNIs is the same as the number of L3VPNs.

Layer 2 VNI 10010 and Layer 3 VNI 10 are used here.

RD and RT Planning

The following table describes the RD planning of the solution.

Planning Description

Suggestion

Example

RD of CE6851HI-1 and CE6851HI-2

Distributed gateways must be configured with RDs of EVPN and VPN instances, and RDs must be unique.

11:1

12:1 (VPN instance)

RD of CE6851HI-3

Distributed gateways must be configured with RDs of EVPN and VPN instances, and RDs must be unique.

13:1

12:2 (VPN instance)

RD of CE6851HI-4

Distributed gateways must be configured with RDs of EVPN and VPN instances, and RDs must be unique.

14: 1

12:3 (VPN instance)

RD of CE6851HI-5 and CE6851HI-6

Distributed gateways must be configured with RDs of EVPN and VPN instances, and RDs must be unique.

15: 1

12:4 (VPN instance)

RD of Exit-Gateway-CE12808-1

Distributed gateways must be configured with RDs of EVPN and VPN instances, and RDs must be unique.

16: 1

RD of Exit-Gateway-CE12808-2

Distributed gateways must be configured with RDs of EVPN and VPN instances, and RDs must be unique.

17: 1

The following table describes the RT planning of the solution.

Device

Suggestion

Example

RT of CE6851HI-1 and CE6851HI-2

Distributed gateways must be configured with RTs of EVPN and VPN instances.

1: 1

10: 1

11:1 (outbound RT of an EVPN instance)

11:1 (RT of a VPN instance, which is used for interworking with an EVPN instance)

RT of CE6851HI-3

Distributed gateways must be configured with RTs of EVPN and VPN instances.

1:1

10:1

11:1 (outbound RT of an EVPN instance)

11:1 (RT of a VPN instance, which is used for interworking with an EVPN instance)

RT of CE6851HI-4

Distributed gateways must be configured with RTs of EVPN and VPN instances.

1:1

10:1

11:1 (outbound RT of an EVPN instance)

11:1 (RT of a VPN instance, which is used for interworking with an EVPN instance)

RT of CE6851HI-5 and CE6851HI-6

Distributed gateways must be configured with RTs of EVPN and VPN instances.

1:1

10:1

11:1 (outbound RT of an EVPN instance)

11:1 (RT of a VPN instance, which is used for interworking with an EVPN instance)

RT of Exit-Gateway-CE12808-1

Distributed gateways must be configured with RTs of EVPN and VPN instances.

1:1

11:1 (outbound RT of an EVPN instance)

11:1 (RT of a VPN instance, which is used for interworking with an EVPN instance)

RT of Exit-Gateway-CE12808-2

Distributed gateways must be configured with RTs of EVPN and VPN instances.

1:1

11:1 (outbound RT of an EVPN instance)

11:1 (RT of a VPN instance, which is used for interworking with an EVPN instance)

Figure 2 shows the RT configuration of VPN and EVPN instances.

  • In the VPN instance, in addition to local VPN instances of ERT X and IRT X, you need to configure ERT Y and IRT Y with EVPN. They are used with EVPN instances to generate host routes.
  • In an EVPN instance, in addition ERT A, ERT B, IRT A, and IRT B for different BDs, you need to configure ERT Y that is used with a VPN instance. Generally, IRT Y does not need to be configured. Otherwise, MAC addresses will be advertised in EVPN instances of different BDs.
Figure 3-4 Configuring RTs

IP Address Planning

The following table lists NE interface address planning, including planning of interconnection network segment addresses, VTEP addresses, BGP Router-IDs, loopback addresses for BGP peer setup, M-LAG heartbeat detection addresses, and service network segment addresses.

The following table describes IP address planning for interface interconnection.

Planning Description

Local Device

Remote Device

Network Segment Address

Configure interface addresses for connecting the leaf nodes (Leaf-CE6851HI-1 and Leaf-CE6851HI-2) and spine nodes.

Leaf-CE6851HI-1&

CE6851HI-2

Spine-CE12804-1

11.254.40.156/30

11.254.40.168/30

Spine-CE12804-2

11.254.40.164/30

11.254.40.160/30

Configure interface addresses for connecting the leaf nodes (Leaf-CE6851HI-3 and Leaf-CE6851HI-4) and spine nodes.

Leaf-CE6851HI-3&

CE6851HI-4

Spine-CE12804-1

11.254.41.156/30

11.254.41.168/30

Spine-CE12804-2

11.254.41.164/30

11.254.41.160/30

Configure interface addresses for connecting the leaf nodes (Leaf-CE6851HI-5 and Leaf-CE6851HI-6) and spine nodes.

Leaf-CE6851HI-5&

CE6851HI-6

Spine-CE12804-1

11.254.46.156/30

11.254.46.168/30

Spine-CE12804-2

11.254.46.164/30

11.254.46.160/30

Configure interface addresses for connecting Spine-CE12804-1 and the egress gateways (Exit-Gateway-CE12808-1 and Exit-Gateway-CE12808-2).

Spine-CE12804-1

Exit-Gateway-CE12808-1

11.254.42.156/30

Exit-Gateway-CE12808-2

11.254.42.160/30

Configure interface addresses for connecting Spine-CE12804-2 and the egress gateways (Exit-Gateway-CE12808-1 and Exit-Gateway-CE12808-2).

Spine-CE12804-2

Exit-Gateway-CE12808-1

11.254.43.156/30

Exit-Gateway-CE12808-2

11.254.43.160/30

Configure interface addresses for connecting Exit-Gateway-CE12808-1 and Router-1.

Exit-Gateway-CE12808-1

Router-1

11.254.44.156/30

Configure interface addresses for connecting Exit-Gateway-CE12808-2 and Router-2.

Exit-Gateway-CE12808-2

Router-2

11.254.44.160/30

Configure interface addresses for connecting Exit-Gateway-CE12808-1 and Exit-Gateway-CE12808-2.

Exit-Gateway-CE12808-1

Exit-Gateway-CE12808-2

11.254.44.164/30

Configure interface addresses for connecting the egress gateways (Exit-Gateway-CE12808-1 and Exit-Gateway-CE12808-2) and firewalls.

Exit-Gateway-CE12808-1

Exit-Gateway-CE12808-2

FW-USG9560-1

FW-USG9560-2

11.254.45.152/29

Virtual IP: 11.254.45.153

11.254.45.160/29

Virtual IP: 11.254.45.161

The following table describes planning for loopback addresses.

Planning Description

Device

IP Address/Mask

Configure Loopback0 addresses as VTEP IP addresses.

NOTE:

Leaf-CE6851HI-3 and Leaf-CE6851HI-4 set up an M-LAG to dual-home servers. Therefore, set the VTEP IP addresses of the two leaf nodes to the same.

Exit-Gateway-CE12808-1 and Exit-Gateway-CE12808-2 set up an M-LAG to dual-home firewalls. Therefore, set the VTEP IP addresses of the two gateways to the same.

Leaf-CE6851HI-1&

CE6851HI-2

11.11.11.11/32

Leaf-CE6851HI-3

11.11.11.12/32

Leaf-CE6851HI-4

Same as that of Leaf-CE6851HI-3: 11.11.11.12/32

Leaf-CE6851HI-5&

CE6851HI-6

11.11.11.17/32

Exit-Gateway-CE12808-1

11.11.11.16/32

Exit-Gateway-CE12808-2

Same as that of Exit-Gateway-CE12808-1: 11.11.11.16/32

Configure Loopback0 addresses of the spine nodes as Router-IDs.

Spine-CE12804-1

11.11.11.14/32

Spine-CE12804-2

11.11.11.15/32

Configure Loopback1 addresses as M-LAG heartbeat detection addresses.

Leaf-CE6851HI-3

13.13.13.13/32

Leaf-CE6851HI-4

14.14.14.14/32

Exit-Gateway-CE12808-1

18.18.18.18/32

Exit-Gateway-CE12808-2

19.19.19.19/32

Configure Loopback2 addresses as the EBGP peer addresses for connecting to the remote routers.

Exit-Gateway-CE12808-1

21.21.21.21/32

Exit-Gateway-CE12808-2

22.22.22.22/32

The following table describes planning for service addresses.

Tenant

IP Address/Mask of the Tenant Network Segment

Address of the Gateway BDIF Interface

VRF

Remarks

Server 1

11.254.10.2/24

11.254.10.1/24

VPN 1

Server 2

11.254.10.3/24

11.254.10.1/24

VPN 1

Server 3

11.254.10.4/24

11.254.10.1/24

VPN 1

Route Planning

EBGP and OSPF are common routing protocols of underlay networks. BGP makes networks secure, flexible, stable, reliable, and efficient from the following aspects:

  • Uses authentication and the Generalized TTL Security Mechanism (GTSM) to ensure network security. TTL refers to time to live.
  • Provides various routing policies, enabling flexible routing.
  • Offers route aggregation and route dampening functions to prevent route flapping, enhancing network stability.
  • Uses the Transmission Control Protocol (TCP) with the port number 179 as the transport layer protocol and supports association with Bidirectional Forwarding Detection (BFD), as well as Auto Fast Reroute (FRR), Graceful Restart (GR), and Non-Stop Routing (NSR), enhancing network reliability.

In network evolution, EBGP applies to large-sized networks, and OSPF applies to middle- and small-sized networks. In this document, EBGP is used for the network, and OSPF routes are used for connecting the border leaf nodes to routers through IGP. In practical operations, Intermediate System-Intermediate System (IS-IS) or IBGP can be used.

The following table describes route planning.

NE

AS Domain Number

Router-ID

Leaf-CE6851HI-1&CE6851-2

65021

Loopback0 address

Leaf-CE6851HI-3

65022

Loopback1 address

Leaf-CE6851HI-4

65022

Loopback1 address

Leaf-CE6851HI-5&CE6851HI-6

65024

Loopback0 address

Spine-CE12804-1

65010

Loopback0 address

Spine-CE12804-2

65010

Loopback0 address

Exit-Gateway-CE12808-1

65000

Loopback1 address

Exit-Gateway-CE12808-2

65000

Loopback1 address

Router-1

65047

Loopback0 address

Router-2

65048

Loopback0 address

Configuring a Stack of Leaf Switches

Configure Leaf-CE6851HI-1 and Leaf-CE6851HI-2 to set up a stack. Then connect the stack to uplink Spine-1 and Spine-2 and downlink servers.

Configuration Roadmap
  1. Establishing a stack: Configure a stack and dual-active detection (DAD), restart the devices, and connect cables to make the stack take effect.
  2. Configuring IP addresses: Configure IP addresses for Layer 3 interconnection interfaces between the leaf and spine nodes, and configure an IP address for Loopback0 (used as the router ID and VTEP IP address).
  3. Configuring server access: Configure switches in a stack to which a server connects.
  4. Configuring routes: Configure the dynamic routing protocol BGP on the stack to ensure Layer 3 reachability between the stack and two neighboring spine nodes.
  5. Configuring BGP EVPN: Configure BGP EVPN as the VXLAN control plane and configure BGP EVPN peers, EVPN instances, ingress replication, and Layer 3 gateways.
Establishing a Stack
  1. On Leaf-CE6851HI-1, set the stack member ID to 1, priority to 150, and domain ID to 10.

    <HUAWEI> system-view
    [~HUAWEI] sysname Leaf-CE6851HI-1
    [*HUAWEI] commit
    [~Leaf-CE6851HI-1] stack
    [~Leaf-CE6851HI-1-stack] stack member 1 priority 150
    [*Leaf-CE6851HI-1-stack] stack member 1 domain 10
    [*Leaf-CE6851HI-1-stack] quit
    [*Leaf-CE6851HI-1] commit

  2. On Leaf-CE6851HI-2, set the stack domain ID to 10.

    <HUAWEI> system-view
    [~HUAWEI] sysname Leaf-CE6851HI-2
    [*HUAWEI] commit
    [~Leaf-CE6851HI-2] stack
    [~Leaf-CE6851HI-2-stack] stack member 1 renumber 2 inherit-config
    [*Leaf-CE6851HI-2-stack] stack member 1 domain 10
    [*Leaf-CE6851HI-2-stack] quit
    [*Leaf-CE6851HI-2] commit

  3. Configure stack ports.

    # On Leaf-CE6851HI-1, add 40GE1/0/1 through 40GE1/0/2 to Stack-Port1/1.

    [~Leaf-CE6851HI-1] interface stack-port 1/1
    [*Leaf-CE6851HI-1-Stack-Port1/1] port member-group interface 40ge 1/0/1 to 1/0/2
    Warning: After the configuration is complete,1.The interface(s) (40GE1/0/1-1/0/2) will be converted to stack mode and be configured with the port crc-statistics trigger error-down command if the configuration does not exist.2.The interface(s) may go Error-Down (crc-statistics) because there is no shutdown configuration on the interfaces.Continue? [Y/N]: y
    [*Leaf-CE6851HI-1-Stack-Port1/1] quit
    [*Leaf-CE6851HI-1] commit
    [~Leaf-CE6851HI-1] quit

    # On Leaf-CE6851HI-2, add 40GE1/0/1 through 40GE1/0/2 to Stack-Port1/1.

    [~Leaf-CE6851HI-2] interface stack-port 1/1
    [*Leaf-CE6851HI-2-Stack-Port1/1] port member-group interface 40ge 1/0/1 to 1/0/2
    Warning: After the configuration is complete,1.The interface(s) (40GE1/0/1-1/0/2) will be converted to stack mode and be configured with the port crc-statistics trigger error-down command if the configuration does not exist.2.The interface(s) may go Error-Down (crc-statistics) because there is no shutdown configuration on the interfaces.Continue? [Y/N]: y
    [*Leaf-CE6851HI-2-Stack-Port1/1] quit
    [*Leaf-CE6851HI-2] commit
    [~Leaf-CE6851HI-2] quit

  4. Save the configuration, and restart the switches.

    # Save the configuration of Leaf-CE6851HI-1, and restart Leaf-CE6851HI-1. Perform the similar configurations for Leaf-CE6851HI-2.

    <Leaf-CE6851HI-1> save
    Warning: The current configuration will be written to the device. Continue? [Y/N]: y
    <Leaf-CE6851HI-1> reboot
    Warning: The system will reboot. Continue? [Y/N]:y

  5. Connect stacking cables to set up a stack.
  6. After the stack is set up, run the save command to save the configuration.
  7. Configure DAD on Leaf-CE6851HI-1 and Leaf-CE6851HI-2 to avoid that the network has two network devices with conflicting configurations in case the stack splits.

    # You can configure DAD on service interfaces in direct mode. You are advised to deploy at least two direct links to ensure reliability. The following is an example:

    [~Leaf-CE6851HI-1] sysname Leaf-CE6851HI-1&CE6851HI-2
    [*Leaf-CE6851HI-1] commit
    [~Leaf-CE6851HI-1&CE6851HI-2] interface 10ge 1/0/30
    [~Leaf-CE6851HI-1&CE6851HI-2-10GE1/0/30] description "for DAD"
    [*Leaf-CE6851HI-1&CE6851HI-2-10GE1/0/30] dual-active detect mode direct
    Warning: The interface will block common data packets, except BPDU packets. Continue? [Y/N]: y
    [*Leaf-CE6851HI-1&CE6851HI-2-10GE1/0/30] quit
    [*Leaf-CE6851HI-1&CE6851HI-2] interface 10ge 1/0/31
    [*Leaf-CE6851HI-1&CE6851HI-2-10GE1/0/31] description "for DAD"
    [*Leaf-CE6851HI-1&CE6851HI-2-10GE1/0/31] dual-active detect mode direct
    Warning: The interface will block common data packets, except BPDU packets. Continue? [Y/N]: y
    [*Leaf-CE6851HI-1&CE6851HI-2-10GE1/0/31] quit
    [*Leaf-CE6851HI-1&CE6851HI-2] interface 10ge 2/0/30
    [*Leaf-CE6851HI-1&CE6851HI-2-10GE2/0/30] description "for DAD"
    [*Leaf-CE6851HI-1&CE6851HI-2-10GE2/0/30] dual-active detect mode direct
    Warning: The interface will block common data packets, except BPDU packets. Continue? [Y/N]: y
    [*Leaf-CE6851HI-1&CE6851HI-2-10GE2/0/30] quit
    [*Leaf-CE6851HI-1&CE6851HI-2] interface 10ge 2/0/31
    [*Leaf-CE6851HI-1&CE6851HI-2-10GE2/0/31] description "for DAD"
    [*Leaf-CE6851HI-1&CE6851HI-2-10GE2/0/31] dual-active detect mode direct
    Warning: The interface will block common data packets, except BPDU packets. Continue? [Y/N]: y
    [*Leaf-CE6851HI-1&CE6851HI-2-10GE2/0/31] commit
    [~Leaf-CE6851HI-1&CE6851HI-2-10GE2/0/31] quit

  8. Set the MAC address of the stack system that functions as a server leaf node to the MAC address of member device 1.

    [~Leaf-CE6851HI-1&CE6851HI-2] stack
    [~Leaf-CE6851HI-1&CE6851HI-2-stack] set system mac-address slot 1     
    //If a CE12800 switch functions as the server leaf node, run the set system mac-address chassis 1 command here.

Configuring IP Addresses
  1. Configure IP addresses for interconnection interfaces.

    NOTE:

    Before switching an interface on the CE6855HI or CE7855EI to Layer 3 mode, run the vlan reserved for main-interface startvlanid to endvlanid command to configure a dedicated reserved VLAN for the Layer 3 main interface.

    [~Leaf-CE6851HI-1&CE6851HI-2] interface 40ge 1/0/3
    [~Leaf-CE6851HI-1&CE6851HI-2-40GE1/0/3] description "to_Spine-CE12804-1-40GE1/0/0"
    [*Leaf-CE6851HI-1&CE6851HI-2-40GE1/0/3] undo portswitch
    [*Leaf-CE6851HI-1&CE6851HI-2-40GE1/0/3] ip address 11.254.40.157 30
    [*Leaf-CE6851HI-1&CE6851HI-2-40GE1/0/3] commit
    [~Leaf-CE6851HI-1&CE6851HI-2-40GE1/0/3] quit
    [~Leaf-CE6851HI-1&CE6851HI-2] interface 40ge 1/0/4
    [~Leaf-CE6851HI-1&CE6851HI-2-40GE1/0/4] description "to_Spine-CE12804-2-40GE1/0/1"
    [*Leaf-CE6851HI-1&CE6851HI-2-40GE1/0/4] undo portswitch
    [*Leaf-CE6851HI-1&CE6851HI-2-40GE1/0/4] ip address 11.254.40.165 30
    [*Leaf-CE6851HI-1&CE6851HI-2-40GE1/0/4] commit
    [~Leaf-CE6851HI-1&CE6851HI-2-40GE1/0/4] quit
    [~Leaf-CE6851HI-1&CE6851HI-2] interface 40ge 2/0/3
    [~Leaf-CE6851HI-1&CE6851HI-2-40GE2/0/3] description "to_Spine-CE12804-2-40GE1/0/0"
    [*Leaf-CE6851HI-1&CE6851HI-2-40GE2/0/3] undo portswitch
    [*Leaf-CE6851HI-1&CE6851HI-2-40GE2/0/3] ip address 11.254.40.161 30
    [*Leaf-CE6851HI-1&CE6851HI-2-40GE2/0/3] commit
    [~Leaf-CE6851HI-1&CE6851HI-2-40GE2/0/3] quit
    [~Leaf-CE6851HI-1&CE6851HI-2] interface 40ge 2/0/4
    [~Leaf-CE6851HI-1&CE6851HI-2-40GE2/0/4] description "to_Spine-CE12804-1-40GE1/0/1"
    [*Leaf-CE6851HI-1&CE6851HI-2-40GE2/0/4] undo portswitch
    [*Leaf-CE6851HI-1&CE6851HI-2-40GE2/0/4] ip address 11.254.40.169 30
    [*Leaf-CE6851HI-1&CE6851HI-2-40GE2/0/4] commit
    [~Leaf-CE6851HI-1&CE6851HI-2-40GE2/0/4] quit

  2. Configure an IP address for the loopback interface.

    [~Leaf-CE6851HI-1&CE6851HI-2] interface loopback 0
    [*Leaf-CE6851HI-1&CE6851HI-2-LoopBack0] description VTEP&Router-ID
    [*Leaf-CE6851HI-1&CE6851HI-2-LoopBack0] ip address 11.11.11.11 32
    [*Leaf-CE6851HI-1&CE6851HI-2-LoopBack0] commit
    [~Leaf-CE6851HI-1&CE6851HI-2-LoopBack0] quit

Configuring Server Access

The following describes the configuration on the switches in a stack to which a server connects.

  1. Set Leaf-CE6851HI-1 and Leaf-CE6851HI-2 in a stack working in load balancing mode.

    [~Leaf-CE6851HI-1&CE6851HI-2] interface eth-trunk 1
    [*Leaf-CE6851HI-1&CE6851HI-2-Eth-Trunk1] mode lacp-static
    [*Leaf-CE6851HI-1&CE6851HI-2-Eth-Trunk1] port link-type trunk
    [*Leaf-CE6851HI-1&CE6851HI-2-Eth-Trunk1] undo port trunk allow-pass vlan 1
    [*Leaf-CE6851HI-1&CE6851HI-2-Eth-Trunk1] trunkport 10ge 1/0/1
    [*Leaf-CE6851HI-1&CE6851HI-2-Eth-Trunk1] trunkport 10ge 2/0/1
    [*Leaf-CE6851HI-1&CE6851HI-2-Eth-Trunk1] quit
    [*Leaf-CE6851HI-1&CE6851HI-2] commit

  2. Configure VXLAN service access points on Leaf-CE6851HI-1 and Leaf-CE6851HI-2.

    [~Leaf-CE6851HI-1&CE6851HI-2] bridge-domain 10
    [~Leaf-CE6851HI-1&CE6851HI-2-bd10] vxlan vni 10010
    [*Leaf-CE6851HI-1&CE6851HI-2-bd10] quit
    [*Leaf-CE6851HI-1&CE6851HI-2] commit
    
    [~Leaf-CE6851HI-1&CE6851HI-2] interface eth-trunk 1.1 mode l2
    [*Leaf-CE6851HI-1&CE6851HI-2-Eth-Trunk1.1] encapsulation dot1q vid 30
    [*Leaf-CE6851HI-1&CE6851HI-2-Eth-Trunk1.1] bridge-domain 10
    [*Leaf-CE6851HI-1&CE6851HI-2-Eth-Trunk1.1] quit
    [*Leaf-CE6851HI-1&CE6851HI-2] commit

Configuring Routes

Configure BGP routes on the underlay network as follows (in practical operations, OSPF can be used).

  1. Configure BGP routes on the stack to connect the stack to spine nodes.

    [~Leaf-CE6851HI-1&CE6851HI-2] bgp 65021
    [*Leaf-CE6851HI-1&CE6851HI-2-bgp] router-id 11.11.11.11
    [*Leaf-CE6851HI-1&CE6851HI-2-bgp] timer keepalive 10 hold 30
    [*Leaf-CE6851HI-1&CE6851HI-2-bgp] group Spine-CE12804 external
    [*Leaf-CE6851HI-1&CE6851HI-2-bgp] peer Spine-CE12804 as-number 65010
    [*Leaf-CE6851HI-1&CE6851HI-2-bgp] peer 11.254.40.158 as-number 65010
    [*Leaf-CE6851HI-1&CE6851HI-2-bgp] peer 11.254.40.158 group Spine-CE12804 
    [*Leaf-CE6851HI-1&CE6851HI-2-bgp] peer 11.254.40.170 as-number 65010
    [*Leaf-CE6851HI-1&CE6851HI-2-bgp] peer 11.254.40.170 group Spine-CE12804
    [*Leaf-CE6851HI-1&CE6851HI-2-bgp] peer 11.254.40.166 as-number 65010
    [*Leaf-CE6851HI-1&CE6851HI-2-bgp] peer 11.254.40.166 group Spine-CE12804     
    [*Leaf-CE6851HI-1&CE6851HI-2-bgp] peer 11.254.40.162 as-number 65010
    [*Leaf-CE6851HI-1&CE6851HI-2-bgp] peer 11.254.40.162 group Spine-CE12804
    [*Leaf-CE6851HI-1&CE6851HI-2-bgp] ipv4-family unicast
    [*Leaf-CE6851HI-1&CE6851HI-2-bgp-af-ipv4] preference 20 200 10
    [*Leaf-CE6851HI-1&CE6851HI-2-bgp-af-ipv4] network 11.11.11.11 255.255.255.255
    [*Leaf-CE6851HI-1&CE6851HI-2-bgp-af-ipv4] maximum load-balancing 32
    [*Leaf-CE6851HI-1&CE6851HI-2-bgp-af-ipv4] quit
    [*Leaf-CE6851HI-1&CE6851HI-2-bgp] quit
    [*Leaf-CE6851HI-1&CE6851HI-2] commit

Configuring BGP EVPN

When BGP EVPN is configured as the VXLAN control plane, perform the following steps to configure the leaf nodes as the RR clients and spine nodes as the RRs, establish IBGP EVPN peer relationship between the spine and leaf nodes, and configure EVPN instances, ingress replication, and Layer 3 gateways.

  1. Enable EVPN as the VXLAN control plane on the stack.

    [~Leaf-CE6851HI-1&CE6851HI-2] evpn-overlay enable
    [*Leaf-CE6851HI-1&CE6851HI-2] commit

  2. Establish IBGP EVPN peer relationships between the stack and spine nodes and use the spine nodes as the RRs.

    [~Leaf-CE6851HI-1&CE6851HI-2] bgp 100 instance evpn1
    [*Leaf-CE6851HI-1&CE6851HI-2-bgp-instance-evpn1] router-id 11.11.11.11
    [*Leaf-CE6851HI-1&CE6851HI-2-bgp-instance-evpn1] peer 11.11.11.14 as-number 100
    //Establish BGP EVPN peer relationships with Spine-CE12804-1.
    [*Leaf-CE6851HI-1&CE6851HI-2-bgp-instance-evpn1] peer 11.11.11.14 connect-interface loopback 0
    [*Leaf-CE6851HI-1&CE6851HI-2-bgp-instance-evpn1] peer 11.11.11.15 as-number 100
    //Establish BGP EVPN peer relationships with Spine-CE12804-2.
    [*Leaf-CE6851HI-1&CE6851HI-2-bgp-instance-evpn1] peer 11.11.11.15 connect-interface loopback 0
    [*Leaf-CE6851HI-1&CE6851HI-2-bgp-instance-evpn1] l2vpn-family evpn
    [*Leaf-CE6851HI-1&CE6851HI-2-bgp-instance-evpn1-af-evpn] peer 11.11.11.14 enable
    [*Leaf-CE6851HI-1&CE6851HI-2-bgp-instance-evpn1-af-evpn] peer 11.11.11.14 advertise irb
    [*Leaf-CE6851HI-1&CE6851HI-2-bgp-instance-evpn1-af-evpn] peer 11.11.11.15 enable
    [*Leaf-CE6851HI-1&CE6851HI-2-bgp-instance-evpn1-af-evpn] peer 11.11.11.15 advertise irb
    [*Leaf-CE6851HI-1&CE6851HI-2-bgp-instance-evpn1-af-evpn] quit
    [*Leaf-CE6851HI-1&CE6851HI-2-bgp-instance-evpn1] quit
    [*Leaf-CE6851HI-1&CE6851HI-2] commit

  3. Configure an EVPN instance on the stack.

    # Configure an EVPN instance on the stack.
    [~Leaf-CE6851HI-1&CE6851HI-2] bridge-domain 10 
    [~Leaf-CE6851HI-1&CE6851HI-2-bd10] evpn 
    [*Leaf-CE6851HI-1&CE6851HI-2-bd10-evpn] route-distinguisher 11:1
    [*Leaf-CE6851HI-1&CE6851HI-2-bd10-evpn] vpn-target 10:1 
    [*Leaf-CE6851HI-1&CE6851HI-2-bd10-evpn] vpn-target 11:1 export-extcommunity    //Configure an RT for interworking with a VPN instance.
    [*Leaf-CE6851HI-1&CE6851HI-2-bd10-evpn] quit
    [*Leaf-CE6851HI-1&CE6851HI-2-bd10] quit 
    [*Leaf-CE6851HI-1&CE6851HI-2] commit 

    # Configure a VPN instance on the stack.

    [~Leaf-CE6851HI-1&CE6851HI-2] ip vpn-instance vpn2 
    [*Leaf-CE6851HI-1&CE6851HI-2-vpn-instance-vpn2] vxlan vni 10 
    [*Leaf-CE6851HI-1&CE6851HI-2-vpn-instance-vpn2] ipv4-family
    [*Leaf-CE6851HI-1&CE6851HI-2-vpn-instance-vpn2-af-ipv4] route-distinguisher 12:1 
    [*Leaf-CE6851HI-1&CE6851HI-2-vpn-instance-vpn2-af-ipv4] vpn-target 1:1 
    [*Leaf-CE6851HI-1&CE6851HI-2-vpn-instance-vpn2-af-ipv4] vpn-target 11:1 evpn    //Contain the EVPN parameter for interworking with an EVPN instance.
    [*Leaf-CE6851HI-1&CE6851HI-2-vpn-instance-vpn2-af-ipv4] quit
    [*Leaf-CE6851HI-1&CE6851HI-2-vpn-instance-vpn2] quit 
    [*Leaf-CE6851HI-1&CE6851HI-2] commit 

  4. Configure ingress replication on the stack.

    [~Leaf-CE6851HI-1&CE6851HI-2] interface nve 1 
    [*Leaf-CE6851HI-1&CE6851HI-2-Nve1] source 11.11.11.11 
    [*Leaf-CE6851HI-1&CE6851HI-2-Nve1] vni 10010 head-end peer-list protocol bgp 
    [*Leaf-CE6851HI-1&CE6851HI-2-Nve1] quit
    [*Leaf-CE6851HI-1&CE6851HI-2] commit 

  5. Configure Layer 3 gateways on the stack.

    [~Leaf-CE6851HI-1&CE6851HI-2] interface vbdif 10
    [*Leaf-CE6851HI-1&CE6851HI-2-Vbdif10] ip binding vpn-instance vpn2
    [*Leaf-CE6851HI-1&CE6851HI-2-Vbdif10] ip address 11.254.10.5 255.255.255.0
    [*Leaf-CE6851HI-1&CE6851HI-2-Vbdif10] arp distribute-gateway enable
    [*Leaf-CE6851HI-1&CE6851HI-2-Vbdif10] arp collect host enable
    [*Leaf-CE6851HI-1&CE6851HI-2-Vbdif10] quit
    [*Leaf-CE6851HI-1&CE6851HI-2] commit 

Configuring an M-LAG of Leaf Switches

Configure Leaf-CE6851HI-3 and Leaf-CE6851HI-4 to set up an M-LAG. Then connect the M-LAG to uplink Spine-CE12804-1 and Spine-CE12804-2 and downlink servers.

Configuration Roadmap
  1. Configuring IP addresses: Configure IP addresses for Layer 3 interconnection interfaces between the spine and leaf nodes, and configure IP addresses for Loopback0 (used as the VTEP IP address) and Loopback1 (used as the router ID).
  2. Configuring an M-LAG: On the switches, configure the M-LAG global mode, DFS group, and peer-link, enable server access through the M-LAG, and configure a Monitor Link group for uplink and downlink interfaces.
  3. Configuring routes: Configure the dynamic routing protocol BGP on the M-LAG group to ensure Layer 3 reachability between the M-LAG group and two neighboring spine nodes.
  4. Configuring BGP EVPN: Configure BGP EVPN as the VXLAN control plane and configure BGP EVPN peers, EVPN instances, ingress replication, and Layer 3 gateways.
Configuring IP Addresses
  1. Configure IP addresses for interconnection interfaces on Leaf-CE6851HI-3 and Leaf-CE6851HI-4.

    NOTE:

    Before switching an interface on the CE6855HI or CE7855EI to Layer 3 mode, run the vlan reserved for main-interface startvlanid to endvlanid command to configure a dedicated reserved VLAN for the Layer 3 main interface.

    [~HUAWEI] sysname Leaf-CE6851HI-3 
    [*HUAWEI] commit 
    [~Leaf-CE6851HI-3] interface 40ge 1/0/3 
    [~Leaf-CE6851HI-3-40GE1/0/3] description "to_Spine-CE12804-1-40GE1/0/2" 
    [*Leaf-CE6851HI-3-40GE1/0/3] undo portswitch 
    [*Leaf-CE6851HI-3-40GE1/0/3] ip address 11.254.41.157 30 
    [*Leaf-CE6851HI-3-40GE1/0/3] commit 
    [~Leaf-CE6851HI-3-40GE1/0/3] quit 
    [~Leaf-CE6851HI-3] interface 40ge 1/0/4 
    [~Leaf-CE6851HI-3-40GE1/0/4] description "to_Spine-CE12804-2-40GE1/0/3" 
    [*Leaf-CE6851HI-3-40GE1/0/4] undo portswitch 
    [*Leaf-CE6851HI-3-40GE1/0/4] ip address 11.254.41.165 30 
    [*Leaf-CE6851HI-3-40GE1/0/4] commit 
    [~Leaf-CE6851HI-3-40GE1/0/4] quit 
     
    [~HUAWEI] sysname Leaf-CE6851HI-4 
    [*HUAWEI] commit 
    [~Leaf-CE6851HI-4] interface 40ge 1/0/3 
    [~Leaf-CE6851HI-4-40GE1/0/3] description "to_Spine-CE12804-2-40GE1/0/2" 
    [*Leaf-CE6851HI-4-40GE1/0/3] undo portswitch 
    [*Leaf-CE6851HI-4-40GE1/0/3] ip address 11.254.41.169 30 
    [*Leaf-CE6851HI-4-40GE1/0/3] commit 
    [~Leaf-CE6851HI-4-40GE1/0/3] quit 
    [~Leaf-CE6851HI-4] interface 40ge 1/0/4 
    [~Leaf-CE6851HI-4-40GE1/0/4] description "to_Spine-CE12804-1-40GE1/0/3" 
    [*Leaf-CE6851HI-4-40GE1/0/4] undo portswitch 
    [*Leaf-CE6851HI-4-40GE1/0/4] ip address 11.254.41.161 30 
    [*Leaf-CE6851HI-4-40GE1/0/4] commit 
    [~Leaf-CE6851HI-4-40GE1/0/4] quit

  2. Configure loopback addresses. According to the planning, configure the Loopback0 address as the VTEP IP address and configure the Loopback1 address as the Router-ID.

    [~Leaf-CE6851HI-3] interface loopback 0    //Used as the VTEP address.
    [*Leaf-CE6851HI-3-LoopBack0] ip address 11.11.11.12 32  
    [*Leaf-CE6851HI-3-LoopBack0] commit 
    [~Leaf-CE6851HI-3-LoopBack0] quit 
    [~Leaf-CE6851HI-3] interface loopback 1 
    [*Leaf-CE6851HI-3-LoopBack1] ip address 13.13.13.13 32 
    [*Leaf-CE6851HI-3-LoopBack1] commit 
    [~Leaf-CE6851HI-3-LoopBack1] quit 
     
    [~Leaf-CE6851HI-4] interface loopback 0     //Used as the VTEP address.
    [*Leaf-CE6851HI-4-LoopBack0] ip address 11.11.11.12 32 
    [*Leaf-CE6851HI-4-LoopBack0] commit 
    [~Leaf-CE6851HI-4-LoopBack0] quit 
    [~Leaf-CE6851HI-4] interface loopback 1 
    [*Leaf-CE6851HI-4-LoopBack1] ip address 14.14.14.14 32 
    [*Leaf-CE6851HI-4-LoopBack1] commit 
    [~Leaf-CE6851HI-4-LoopBack1] quit

Configuring Routes

Configure BGP routes on the underlay network as follows (in practical operations, OSPF can be used).

  1. Configure BGP routes on Leaf-CE6851HI-3 to connect it to spine nodes.

    [~Leaf-CE6851HI-3] bgp 65022 
    [*Leaf-CE6851HI-3-bgp] router-id 13.13.13.13 
    [*Leaf-CE6851HI-3-bgp] timer keepalive 10 hold 30 
    [*Leaf-CE6851HI-3-bgp] group Spine-CE12804 external 
    [*Leaf-CE6851HI-3-bgp] peer Spine-CE12804 as-number 65010 
    [*Leaf-CE6851HI-3-bgp] peer 11.254.41.158 as-number 65010 
    [*Leaf-CE6851HI-3-bgp] peer 11.254.41.158 group Spine-CE12804  
    [*Leaf-CE6851HI-3-bgp] peer 11.254.41.166 as-number 65010 
    [*Leaf-CE6851HI-3-bgp] peer 11.254.41.166 group Spine-CE12804      
     
    [*Leaf-CE6851HI-3-bgp] ipv4-family unicast   
    [*Leaf-CE6851HI-3-bgp-af-ipv4] preference 20 200 10 
    [*Leaf-CE6851HI-3-bgp-af-ipv4] network 11.11.11.12 255.255.255.255 
    [*Leaf-CE6851HI-3-bgp-af-ipv4] network 13.13.13.13 255.255.255.255 
    [*Leaf-CE6851HI-3-bgp-af-ipv4] maximum load-balancing 32 
    [*Leaf-CE6851HI-3-bgp-af-ipv4] quit 
    [*Leaf-CE6851HI-3-bgp] quit 
    [*Leaf-CE6851HI-3] commit

  2. Configure BGP routes on Leaf-CE6851HI-4 to connect it to spine nodes.

    [~Leaf-CE6851HI-4] bgp 65022 
    [*Leaf-CE6851HI-4-bgp] router-id 14.14.14.14 
    [*Leaf-CE6851HI-4-bgp] timer keepalive 10 hold 30 
    [*Leaf-CE6851HI-4-bgp] group Spine-CE12804 external 
    [*Leaf-CE6851HI-4-bgp] peer Spine-CE12804 as-number 65010 
    [*Leaf-CE6851HI-4-bgp] peer 11.254.41.170 as-number 65010 
    [*Leaf-CE6851HI-4-bgp] peer 11.254.41.170 group Spine-CE12804  
    [*Leaf-CE6851HI-4-bgp] peer 11.254.41.162 as-number 65010 
    [*Leaf-CE6851HI-4-bgp] peer 11.254.41.162 group Spine-CE12804   
     
    [*Leaf-CE6851HI-4-bgp] ipv4-family unicast 
    [*Leaf-CE6851HI-4-bgp-af-ipv4] preference 20 200 10 
    [*Leaf-CE6851HI-4-bgp-af-ipv4] network 11.11.11.12 255.255.255.255 
    [*Leaf-CE6851HI-4-bgp-af-ipv4] network 14.14.14.14 255.255.255.255 
    [*Leaf-CE6851HI-4-bgp-af-ipv4] maximum load-balancing 32 
    [*Leaf-CE6851HI-4-bgp-af-ipv4] quit 
    [*Leaf-CE6851HI-4-bgp] quit 
    [*Leaf-CE6851HI-4] commit

Configuring an M-LAG
  1. Configure the M-LAG mode.

    <Leaf-CE6851HI-3> system-view 
    [~Leaf-CE6851HI-3] stp mode rstp 
    [*Leaf-CE6851HI-3] stp v-stp enable 
    [*Leaf-CE6851HI-3] commit 
    [~Leaf-CE6851HI-3] lacp m-lag system-id 00e0-fc00-0001 
    [*Leaf-CE6851HI-3] commit 
     
    <Leaf-CE6851HI-4> system-view 
    [~Leaf-CE6851HI-4] stp mode rstp 
    [*Leaf-CE6851HI-4] stp v-stp enable 
    [*Leaf-CE6851HI-4] commit 
    [~Leaf-CE6851HI-4] lacp m-lag system-id 00e0-fc00-0001 
    [*Leaf-CE6851HI-4] commit

  2. Configure the DFS group of the M-LAG on Leaf-CE6851HI-3 and Leaf-CE6851HI-4.

    [~Leaf-CE6851HI-3] dfs-group 1 
    [*Leaf-CE6851HI-3-dfs-group-1] source ip 13.13.13.13 
    [*Leaf-CE6851HI-3-dfs-group-1] priority 150 
    [*Leaf-CE6851HI-3-dfs-group-1] quit 
    [*Leaf-CE6851HI-3]commit  
     
    [~Leaf-CE6851HI-4] dfs-group 1 
    [*Leaf-CE6851HI-4-dfs-group-1] source ip 14.14.14.14 
    [*Leaf-CE6851HI-4-dfs-group-1] priority 120 
    [*Leaf-CE6851HI-4-dfs-group-1] quit 
    [*Leaf-CE6851HI-4] commit

  3. Configure the peer-link of the M-LAG on Leaf-CE6851HI-3 and Leaf-CE6851HI-4.

    [~Leaf-CE6851HI-3] interface eth-trunk 0 
    [*Leaf-CE6851HI-3-Eth-Trunk0] trunkport 40ge 1/0/1 
    [*Leaf-CE6851HI-3-Eth-Trunk0] trunkport 40ge 1/0/2 
    [*Leaf-CE6851HI-3-Eth-Trunk0] mode lacp-static 
    [*Leaf-CE6851HI-3-Eth-Trunk0] peer-link 1 
    [*Leaf-CE6851HI-3-Eth-Trunk0] quit 
    [*Leaf-CE6851HI-3] commit 
     
    [~Leaf-CE6851HI-4] interface eth-trunk 0 
    [*Leaf-CE6851HI-4-Eth-Trunk0] trunkport 40ge 1/0/1 
    [*Leaf-CE6851HI-4-Eth-Trunk0] trunkport 40ge 1/0/2 
    [*Leaf-CE6851HI-4-Eth-Trunk0] mode lacp-static 
    [*Leaf-CE6851HI-4-Eth-Trunk0] peer-link 1 
    [*Leaf-CE6851HI-4-Eth-Trunk0] quit 
    [*Leaf-CE6851HI-4] commit

  4. Enable server access through the M-LAG.

    # Configure the member interfaces of the M-LAG on Leaf-CE6851HI-3 and Leaf-CE6851HI-4 (when servers connect to the M-LAG in load balancing mode).

    [~Leaf-CE6851HI-3] interface eth-trunk 1 
    [*Leaf-CE6851HI-3-Eth-Trunk1] mode lacp-static 
    [*Leaf-CE6851HI-3-Eth-Trunk1] port link-type trunk 
    [*Leaf-CE6851HI-3-Eth-Trunk1] undo port trunk allow-pass vlan 1 
    [*Leaf-CE6851HI-3-Eth-Trunk1] trunkport 10ge 1/0/1 
    [*Leaf-CE6851HI-3-Eth-Trunk1] dfs-group 1 m-lag 1 
    [*Leaf-CE6851HI-3-Eth-Trunk1] quit 
    [*Leaf-CE6851HI-3] commit 
     
    [*Leaf-CE6851HI-4] interface eth-trunk 1 
    [*Leaf-CE6851HI-4-Eth-Trunk1] mode lacp-static 
    [*Leaf-CE6851HI-4-Eth-Trunk1] port link-type trunk 
    [*Leaf-CE6851HI-4-Eth-Trunk1] trunkport 10ge 1/0/1 
    [*Leaf-CE6851HI-4-Eth-Trunk1] undo port trunk allow-pass vlan 1 
    [*Leaf-CE6851HI-4-Eth-Trunk1] dfs-group 1 m-lag 1 
    [*Leaf-CE6851HI-4-Eth-Trunk1] quit 
    [*Leaf-CE6851HI-4] commit

  5. Configure VXLAN service access points on Leaf-CE6851HI-3 and Leaf-CE6851HI-4.

    [~Leaf-CE6851HI-3] bridge-domain 10 
    [*Leaf-CE6851HI-3-bd10] vxlan vni 10010 
    [*Leaf-CE6851HI-3-bd10] quit 
    [*Leaf-CE6851HI-3] commit 
     
    [~Leaf-CE6851HI-3] interface eth-trunk 1.1 mode l2 
    [*Leaf-CE6851HI-3-Eth-Trunk1.1] encapsulation dot1q vid 30 
    [*Leaf-CE6851HI-3-Eth-Trunk1.1] bridge-domain 10 
    [*Leaf-CE6851HI-3-Eth-Trunk1.1] quit 
    [*Leaf-CE6851HI-3] commit 
     
    [~Leaf-CE6851HI-4] bridge-domain 10 
    [*Leaf-CE6851HI-4-bd10] vxlan vni 10010 
    [*Leaf-CE6851HI-4-bd10] quit 
    [*Leaf-CE6851HI-4] commit 
     
    [~Leaf-CE6851HI-4] interface eth-trunk 1.1 mode l2 
    [*Leaf-CE6851HI-4-Eth-Trunk1.1] encapsulation dot1q vid 30 
    [*Leaf-CE6851HI-4-Eth-Trunk1.1] bridge-domain 10 
    [*Leaf-CE6851HI-4-Eth-Trunk1.1] quit 
    [*Leaf-CE6851HI-4] commit

  6. Associate uplink and downlink interfaces with a Monitor Link group on Leaf-CE6851HI-3 and Leaf-CE6851HI-4 to prevent a user-side traffic forwarding failure on a device in case all uplinks on the device fail.

    [~Leaf-CE6851HI-3] monitor-link group 1 
    [*Leaf-CE6851HI-3-mtlk-group1] port 40ge 1/0/3 uplink 
    [*Leaf-CE6851HI-3-mtlk-group1] port 40ge 1/0/4 uplink 
    [*Leaf-CE6851HI-3-mtlk-group1] port 10ge 1/0/1 downlink 1 
    [*Leaf-CE6851HI-3-mtlk-group1] commit 
     
    [~Leaf-CE6851HI-4] monitor-link group 1  
    [*Leaf-CE6851HI-4-mtlk-group1] port 40ge 1/0/3 uplink 
    [*Leaf-CE6851HI-4-mtlk-group1] port 40ge 1/0/4 uplink 
    [*Leaf-CE6851HI-4-mtlk-group1] port 10ge 1/0/1 downlink 1 
    [*Leaf-CE6851HI-4-mtlk-group1] commit

Configuring BGP EVPN

When BGP EVPN is configured as the VXLAN control plane, perform the following steps to configure the leaf nodes as the RR clients and spine nodes as the RRs, establish IBGP EVPN peer relationship between the spine and leaf nodes, and configure EVPN instances, ingress replication, and Layer 3 gateways.

  1. Configure EVPN as the VXLAN control plane on Leaf-CE6851HI-3 and Leaf-CE6851HI-4.

    [~Leaf-CE6851HI-3] evpn-overlay enable 
    [*Leaf-CE6851HI-3] commit 
     
    [~Leaf-CE6851HI-4] evpn-overlay enable 
    [*Leaf-CE6851HI-4] commit

  2. Establish BGP EVPN peer relationships between the leaf nodes (Leaf-CE6851HI-3 and Leaf-CE6851HI-4) and spine nodes, and configure the spine nodes as RRs.

    # Establish IBGP EVPN peer relationships between Leaf-CE6851HI-3 and spine nodes.

    [~Leaf-CE6851HI-3] bgp 100 instance evpn1 
    [*Leaf-CE6851HI-3-bgp-instance-evpn1] router-id 13.13.13.13 
    [*Leaf-CE6851HI-3-bgp-instance-evpn1] peer 11.11.11.14 as-number 100 
    //Establish a BGP EVPN peer relationship with Spine-CE12804-1. 
    [*Leaf-CE6851HI-3-bgp-instance-evpn1] peer 11.11.11.14 connect-interface loopback 1 
    [*Leaf-CE6851HI-3-bgp-instance-evpn1] peer 11.11.11.15 as-number 100 
    //Establish a BGP EVPN peer relationship with Spine-CE12804-2. 
    [*Leaf-CE6851HI-3-bgp-instance-evpn1] peer 11.11.11.15 connect-interface loopback 1 
    [*Leaf-CE6851HI-3-bgp-instance-evpn1] l2vpn-family evpn 
    [*Leaf-CE6851HI-3-bgp-instance-evpn1-af-evpn] peer 11.11.11.14 enable 
    [*Leaf-CE6851HI-3-bgp-instance-evpn1-af-evpn] peer 11.11.11.14 advertise irb 
    [*Leaf-CE6851HI-3-bgp-instance-evpn1-af-evpn] peer 11.11.11.15 enable 
    [*Leaf-CE6851HI-3-bgp-instance-evpn1-af-evpn] peer 11.11.11.15 advertise irb 
    [*Leaf-CE6851HI-3-bgp-instance-evpn1-af-evpn] quit 
    [*Leaf-CE6851HI-3-bgp-instance-evpn1] quit 
    [*Leaf-CE6851HI-3] commit

    # Establish IBGP EVPN peer relationships between Leaf-CE6851HI-4 and spine nodes.

    [~Leaf-CE6851HI-4] bgp 100 instance evpn1 
    [*Leaf-CE6851HI-4-bgp-instance-evpn1] router-id 14.14.14.14 
    [*Leaf-CE6851HI-4-bgp-instance-evpn1] peer 11.11.11.14 as-number 100 
    //Establish a BGP EVPN peer relationship with Spine-CE12804-1. 
    [*Leaf-CE6851HI-4-bgp-instance-evpn1] peer 11.11.11.14 connect-interface loopback 1 
    [*Leaf-CE6851HI-4-bgp-instance-evpn1] peer 11.11.11.15 as-number 100 
    //Establish a BGP EVPN peer relationship with Spine-CE12804-2. 
    [*Leaf-CE6851HI-4-bgp-instance-evpn1] peer 11.11.11.15 connect-interface loopback 1 
    [*Leaf-CE6851HI-4-bgp-instance-evpn1] l2vpn-family evpn 
    [*Leaf-CE6851HI-4-bgp-instance-evpn1-af-evpn] peer 11.11.11.14 enable 
    [*Leaf-CE6851HI-4-bgp-instance-evpn1-af-evpn] peer 11.11.11.14 advertise irb 
    [*Leaf-CE6851HI-4-bgp-instance-evpn1-af-evpn] peer 11.11.11.15 enable 
    [*Leaf-CE6851HI-4-bgp-instance-evpn1-af-evpn] peer 11.11.11.15 advertise irb 
    [*Leaf-CE6851HI-4-bgp-instance-evpn1-af-evpn] quit 
    [*Leaf-CE6851HI-4-bgp-instance-evpn1] quit 
    [*Leaf-CE6851HI-4] commit

  3. # Configure an EVPN instance on Leaf-CE6851HI-3.

    [~Leaf-CE6851HI-3] bridge-domain 10  
    [~Leaf-CE6851HI-3-bd10] evpn  
    [*Leaf-CE6851HI-3-bd10-evpn] route-distinguisher 13:1 
    [*Leaf-CE6851HI-3-bd10-evpn] vpn-target 1:1  
    [*Leaf-CE6851HI-3-bd10-evpn] vpn-target 11:1 export-extcommunity    //Configure an RT for interworking with a VPN instance.
    [*Leaf-CE6851HI-3-bd10-evpn] quit 
    [*Leaf-CE6851HI-3-bd10] quit  
    [*Leaf-CE6851HI-3] commit

    # Configure a VPN instance on Leaf-CE6851HI-3.

    [~Leaf-CE6851HI-3] ip vpn-instance vpn2  
    [*Leaf-CE6851HI-3-vpn-instance-vpn2] vxlan vni 10  
    [*Leaf-CE6851HI-3-vpn-instance-vpn2] ipv4-family 
    [*Leaf-CE6851HI-3-vpn-instance-vpn2-af-ipv4] route-distinguisher 12:2  
    [*Leaf-CE6851HI-3-vpn-instance-vpn2-af-ipv4] vpn-target 11:1 evpn    //Contain the EVPN parameter for interworking with an EVPN instance.
    [*Leaf-CE6851HI-3-vpn-instance-vpn2-af-ipv4] quit 
    [*Leaf-CE6851HI-3-vpn-instance-vpn2] quit  
    [*Leaf-CE6851HI-3] commit

    # Configure an EVPN instance on Leaf-CE6851HI-4.

    [~Leaf-CE6851HI-4] bridge-domain 10  
    [~Leaf-CE6851HI-4-bd10] evpn  
    [*Leaf-CE6851HI-4-bd10-evpn] route-distinguisher 13:1 
    [*Leaf-CE6851HI-4-bd10-evpn] vpn-target 10:1  
    [*Leaf-CE6851HI-4-bd10-evpn] vpn-target 11:1 export-extcommunity    //Configure an RT for interworking with a VPN instance.
    [*Leaf-CE6851HI-4-bd10-evpn] quit 
    [*Leaf-CE6851HI-4-bd10] quit  
    [*Leaf-CE6851HI-4] commit

    # Configure a VPN instance on Leaf-CE6851HI-4.

    [~Leaf-CE6851HI-4] ip vpn-instance vpn2  
    [*Leaf-CE6851HI-4-vpn-instance-vpn2] vxlan vni 10  
    [*Leaf-CE6851HI-4-vpn-instance-vpn2] ipv4-family 
    [*Leaf-CE6851HI-4-vpn-instance-vpn2-af-ipv4] route-distinguisher 12:3  
    [*Leaf-CE6851HI-4-vpn-instance-vpn2-af-ipv4] vpn-target 1:1  
    [*Leaf-CE6851HI-4-vpn-instance-vpn2-af-ipv4] vpn-target 11:1 evpn    //Contain the EVPN parameter for interworking with an EVPN instance.
    [*Leaf-CE6851HI-4-vpn-instance-vpn2-af-ipv4] quit 
    [*Leaf-CE6851HI-4-vpn-instance-vpn2] quit  
    [*Leaf-CE6851HI-4] commit

  4. Configure ingress replication.

    # Configure ingress replication on Leaf-CE6851HI-3.

    [~Leaf-CE6851HI-3] interface nve 1  
    [*Leaf-CE6851HI-3-Nve1] source 11.11.11.12  
    [*Leaf-CE6851HI-3-Nve1] vni 10010 head-end peer-list protocol bgp  
    [*Leaf-CE6851HI-3-Nve1] quit 
    [*Leaf-CE6851HI-3] commit

    # Configure ingress replication on Leaf-CE6851HI-4.

    [~Leaf-CE6851HI-4] interface nve 1  
    [*Leaf-CE6851HI-4-Nve1] source 11.11.11.12  
    [*Leaf-CE6851HI-4-Nve1] vni 10010 head-end peer-list protocol bgp  
    [*Leaf-CE6851HI-4-Nve1] quit 
    [*Leaf-CE6851HI-4] commit

  5. Configure a Layer 3 gateway on Leaf-CE6851HI-3.

    [~Leaf-CE6851HI-3] interface vbdif 10 
    [*Leaf-CE6851HI-3-Vbdif10] ip binding vpn-instance vpn2 
    [*Leaf-CE6851HI-3-Vbdif10] ip address 11.254.10.5 255.255.255.0 
    [*Leaf-CE6851HI-3-Vbdif10] arp distribute-gateway enable 
    [*Leaf-CE6851HI-3-Vbdif10] arp collect host enable 
    [*Leaf-CE6851HI-3-Vbdif10] quit 
    [*Leaf-CE6851HI-3] commit

    Configure a Layer 3 gateway on Leaf-CE6851HI-4.

    [~Leaf-CE6851HI-4] interface vbdif 10 
    [*Leaf-CE6851HI-4-Vbdif10] ip binding vpn-instance vpn2 
    [*Leaf-CE6851HI-4-Vbdif10] ip address 11.254.10.5 255.255.255.0 
    [*Leaf-CE6851HI-4-Vbdif10] arp distribute-gateway enable 
    [*Leaf-CE6851HI-4-Vbdif10] arp collect host enable 
    [*Leaf-CE6851HI-4-Vbdif10] quit 
    [*Leaf-CE6851HI-4] commit

Configuring an SVF System of Leaf Switches

Configure Leaf-CE6851HI-5 and Leaf-CE6851HI-6 as L3 distributed gateways and set up a stack. Then connect the stack to multiple cost-effective CE series switches to increase the number of access interfaces.

Configuration Roadmap
  1. Establishing a stack: Configure two CE6851HI series switches as SVF parent nodes and set up a stack. Then configure dual-active detection (DAD), restart the devices, and connect cables to make the stack take effect.
  2. Configuring SVF: Configure fabric ports on the SVF parent nodes to connect to SVF leaf nodes (CE5810 series switches), and set up SVF links between the parent nodes and leaf nodes.
  3. Configuring IP addresses: Configure IP addresses for Layer 3 interconnection interfaces between the leaf and spine nodes, and configure an IP address for Loopback0 (used as the router ID and VTEP IP address).
  4. Configuring server access: Configure the switches to enable service servers to access the stack.
  5. Configuring routes: Configure BGP dynamic routes on the stack to establish peer relationships with two spine devices, and ensure that the routes between the stack and spine devices are reachable at Layer 3.
  6. Configuring BGP EVPN: Configure BGP EVPN as the VXLAN control plane and configure BGP EVPN peers, EVPN instances, ingress replication, and Layer 3 gateways.
Establishing a Stack
  1. On Leaf-CE6851HI-5, set the stack member ID to 1, priority to 150, and domain ID to 10.

    <HUAWEI> system-view 
    [~HUAWEI] sysname Leaf-CE6851HI-5 
    [*HUAWEI] commit 
    [~Leaf-CE6851HI-5] stack 
    [~Leaf-CE6851HI-5-stack] stack member 1 priority 150 
    [*Leaf-CE6851HI-5-stack] stack member 1 domain 10 
    [*Leaf-CE6851HI-5-stack] quit 
    [*Leaf-CE6851HI-5] commit

  2. On Leaf-CE6851HI-6, set the stack domain ID to 10.

    <HUAWEI> system-view 
    [~HUAWEI] sysname Leaf-CE6851HI-6 
    [*HUAWEI] commit 
    [~Leaf-CE6851HI-6] stack 
    [~Leaf-CE6851HI-6-stack] stack member 1 renumber 2 inherit-config 
    [*Leaf-CE6851HI-6-stack] stack member 1 domain 10 
    [*Leaf-CE6851HI-6-stack] quit 
    [*Leaf-CE6851HI-6] commit
    NOTE:

    When setting up an SVF system, a parent switch must have a stack member ID that is less than or equal to 4.

  3. Configure stack ports.

    # On Leaf-CE6851HI-5, add 40GE1/0/1 through 40GE1/0/2 to Stack-Port1/1.

    [~Leaf-CE6851HI-5] interface stack-port 1/1 
    [*Leaf-CE6851HI-5-Stack-Port1/1] port member-group interface 40ge 1/0/1 to 1/0/2 
    Warning: After the configuration is complete,1.The interface(s) (40GE1/0/1-1/0/2) will be converted to stack mode and be configured with the port crc-statistics trigger error-down command if the configuration does not exist.2.The interface(s) may go Error-Down (crc-statistics) because there is no shutdown configuration on the interfaces.Continue? [Y/N]: y 
    [*Leaf-CE6851HI-5-Stack-Port1/1] quit 
    [*Leaf-CE6851HI-5] commit 
    [~Leaf-CE6851HI-5] quit

    # On Leaf-CE6851HI-6, add 40GE1/0/1 through 40GE1/0/2 to Stack-Port1/1.

    [~Leaf-CE6851HI-6] interface stack-port 1/1 
    [*Leaf-CE6851HI-6-Stack-Port1/1] port member-group interface 40ge 1/0/1 to 1/0/2 
    Warning: After the configuration is complete,1.The interface(s) (40GE1/0/1-1/0/2) will be converted to stack mode and be configured with the port crc-statistics trigger error-down command if the configuration does not exist.2.The interface(s) may go Error-Down (crc-statistics) because there is no shutdown configuration on the interfaces.Continue? [Y/N]: y 
    [*Leaf-CE6851HI-6-Stack-Port1/1] quit 
    [*Leaf-CE6851HI-6] commit 
    [~Leaf-CE6851HI-6] quit

  4. Save the configuration, and restart the switches.

    # Save the configuration of Leaf-CE6851HI-5, and restart Leaf-CE6851HI-5. Perform the same operation for Leaf-CE6851HI-6.

    <Leaf-CE6851HI-5> save 
    Warning: The current configuration will be written to the device. Continue? [Y/N]: y 
    <Leaf-CE6851HI-5> reboot 
    Warning: The system will reboot. Continue? [Y/N]:y

  5. Connect stacking cables to set up a stack.
  6. After the stack is set up, run the save command to save the configuration.
  7. Configure DAD on Leaf-CE6851HI-5 and Leaf-CE6851HI-6 to avoid that the network has two network devices with conflicting configurations in case the stack splits.

    # For out-of-band management, you can configure DAD on the management interface.

    [~Leaf-CE6851HI-5] sysname Leaf-CE6851HI-5&CE6851HI-6 
    [*Leaf-CE6851HI-5] commit 
    [~Leaf-CE6851HI-5&CE6851HI-6] interface 10ge 1/0/30 
    [~Leaf-CE6851HI-5&CE6851HI-6-10GE1/0/30] description "for DAD" 
    [*Leaf-CE6851HI-5&CE6851HI-6-10GE1/0/30] dual-active detect mode direct 
    Warning: The interface will block common data packets, except BPDU packets. Continue? [Y/N]: y 
    [*Leaf-CE6851HI-5&CE6851HI-6-10GE1/0/30] quit 
    [*Leaf-CE6851HI-5&CE6851HI-6] interface 10ge 1/0/31 
    [*Leaf-CE6851HI-5&CE6851HI-6-10GE1/0/31] description "for DAD" 
    [*Leaf-CE6851HI-5&CE6851HI-6-10GE1/0/31] dual-active detect mode direct 
    Warning: The interface will block common data packets, except BPDU packets. Continue? [Y/N]: y 
    [*Leaf-CE6851HI-5&CE6851HI-6-10GE1/0/31] quit 
    [*Leaf-CE6851HI-5&CE6851HI-6] interface 10ge 2/0/30 
    [*Leaf-CE6851HI-5&CE6851HI-6-10GE2/0/30] description "for DAD" 
    [*Leaf-CE6851HI-5&CE6851HI-6-10GE2/0/30] dual-active detect mode direct 
    Warning: The interface will block common data packets, except BPDU packets. Continue? [Y/N]: y 
    [*Leaf-CE6851HI-5&CE6851HI-6-10GE2/0/30] quit 
    [*Leaf-CE6851HI-5&CE6851HI-6] interface 10ge 2/0/31 
    [*Leaf-CE6851HI-5&CE6851HI-6-10GE2/0/31] description "for DAD" 
    [*Leaf-CE6851HI-5&CE6851HI-6-10GE2/0/31] dual-active detect mode direct 
    Warning: The interface will block common data packets, except BPDU packets. Continue? [Y/N]: y 
    [*Leaf-CE6851HI-5&CE6851HI-6-10GE2/0/31] commit 
    [~Leaf-CE6851HI-5&CE6851HI-6-10GE2/0/31] quit

Configuring an SVF System
  1. Configure Fabric-port1, add member ports 10GE1/0/1 and 10GE2/0/1 to Fabric-port1, and bind leaf ID 101 to Fabric-port1.

    [~Leaf-CE6851HI-5&CE6851HI-6] interface fabric-port 1 
    [*Leaf-CE6851HI-5&CE6851HI-6-Fabric-Port1] port bind member 101 
    [*Leaf-CE6851HI-5&CE6851HI-6-Fabric-Port1] port member-group interface 10ge 1/0/1 2/0/1 
    Warning: The interface(s) (10GE1/0/1-1/0/4,10GE2/0/1-2/0/4) will be converted to stack mode and have "port crc-statistics trigger error-down" configured. [Y/N]: y 
    [*Leaf-CE6851HI-5&CE6851HI-6-Fabric-Port1] quit 
    [*Leaf-CE6851HI-5&CE6851HI-6] commit

  2. Configure Fabric-port2, add member ports 10GE1/0/2 and 10GE2/0/2 to Fabric-port2, and bind leaf ID 102 to Fabric-port2.

    [~Leaf-CE6851HI-5&CE6851HI-6] interface fabric-port 2 
    [*Leaf-CE6851HI-5&CE6851HI-6-Fabric-Port2] port bind member 102 
    [*Leaf-CE6851HI-5&CE6851HI-6-Fabric-Port2] port member-group interface 10ge 1/0/2 2/0/2 
    [*Leaf-CE6851HI-5&CE6851HI-6-Fabric-Port2] quit 
    [*Leaf-CE6851HI-5&CE6851HI-6] commit

  3. Configure Fabric-port3, add member ports 10GE1/0/3 and 10GE2/0/3 to Fabric-port3, and bind leaf ID 103 to Fabric-port3.

    [~Leaf-CE6851HI-5&CE6851HI-6] interface fabric-port 3 
    [*Leaf-CE6851HI-5&CE6851HI-6-Fabric-Port3] port bind member 103 
    [*Leaf-CE6851HI-5&CE6851HI-6-Fabric-Port3] port member-group interface 10ge 1/0/3 2/0/3 
    [*Leaf-CE6851HI-5&CE6851HI-6-Fabric-Port3] commit 
    [~Leaf-CE6851HI-5&CE6851HI-6-Fabric-Port3] quit

  4. Set the forwarding mode of the SVF system to centralized so that leaf switches send all traffic to the parent switches for forwarding.

    [~Leaf-CE6851HI-5&CE6851HI-6] stack 
    [*Leaf-CE6851HI-5&CE6851HI-6-stack] stack forwarding-model centralized 
    [*Leaf-CE6851HI-5&CE6851HI-6-stack] commit 
    [*Leaf-CE6851HI-5&CE6851HI-6-stack] return 
    <Leaf-CE6851HI-5&CE6851HI-6> save 
    <Leaf-CE6851HI-5&CE6851HI-6> reboot //Restart the parent switches to make the forwarding mode take effect.

  5. Connect leaf switches to the parent switches and power on the leaf switches. The leaf switches connect to the parent switches through uplink 10GE ports.

    # This example assumes that the leaf switches start without any configuration file and can join the SVF system through auto-negotiation. No manual configuration is required on the leaf switches.

    # If the leaf switches have startup configuration files, start the switches (in auto-negotiation mode) after clearing the configuration files for the next startup or run the following commands to set the working mode of the leaf switches to the leaf mode.

    [~Leaf-CE6851HI-5&CE6851HI-6] stack 
    [~Leaf-CE6851HI-5&CE6851HI-6-stack] switch mode leaf member all //The configured working mode takes effect after the switches restart. 
    [*Leaf-CE6851HI-5&CE6851HI-6-stack] commit 
    [*Leaf-CE6851HI-5&CE6851HI-6-stack] return 
    <*Leaf-CE6851HI-5&CE6851HI-6> save 
    <*Leaf-CE6851HI-5&CE6851HI-6> reboot

Configuring IP Addresses
  1. Configure IP addresses for interconnection interfaces.

    NOTE:

    Before switching an interface on the CE6855HI or CE7855EI to Layer 3 mode, run the vlan reserved for main-interface startvlanid to endvlanid command to configure a dedicated reserved VLAN for the Layer 3 main interface.

    [~Leaf-CE6851HI-5&CE6851HI-6] interface 40ge 1/0/3 
    [~Leaf-CE6851HI-5&CE6851HI-6-40GE1/0/3] description "to_Spine-CE12804-1-40GE1/0/6" 
    [*Leaf-CE6851HI-5&CE6851HI-6-40GE1/0/3] undo portswitch 
    [*Leaf-CE6851HI-5&CE6851HI-6-40GE1/0/3] ip address 11.254.46.157 30 
    [*Leaf-CE6851HI-5&CE6851HI-6-40GE1/0/3] commit 
    [~Leaf-CE6851HI-5&CE6851HI-6-40GE1/0/3] quit 
    [~Leaf-CE6851HI-5&CE6851HI-6] interface 40ge 1/0/4 
    [~Leaf-CE6851HI-5&CE6851HI-6-40GE1/0/4] description "to_Spine-CE12804-2-40GE1/0/6" 
    [*Leaf-CE6851HI-5&CE6851HI-6-40GE1/0/4] undo portswitch 
    [*Leaf-CE6851HI-5&CE6851HI-6-40GE1/0/4] ip address 11.254.46.165 30 
    [*Leaf-CE6851HI-5&CE6851HI-6-40GE1/0/4] commit 
    [~Leaf-CE6851HI-5&CE6851HI-6-40GE1/0/4] quit 
    [~Leaf-CE6851HI-5&CE6851HI-6] interface 40ge 2/0/3 
    [~Leaf-CE6851HI-5&CE6851HI-6-40GE2/0/3] description "to_Spine-CE12804-1-40GE1/0/7" 
    [*Leaf-CE6851HI-5&CE6851HI-6-40GE2/0/3] undo portswitch 
    [*Leaf-CE6851HI-5&CE6851HI-6-40GE2/0/3] ip address 11.254.46.169 30 
    [*Leaf-CE6851HI-5&CE6851HI-6-40GE2/0/3] commit 
    [~Leaf-CE6851HI-5&CE6851HI-6-40GE2/0/3] quit 
    [~Leaf-CE6851HI-5&CE6851HI-6] interface 40ge 2/0/4 
    [~Leaf-CE6851HI-5&CE6851HI-6-40GE2/0/4] description "to_Spine-CE12804-2-40GE1/0/7" 
    [*Leaf-CE6851HI-5&CE6851HI-6-40GE2/0/4] undo portswitch 
    [*Leaf-CE6851HI-5&CE6851HI-6-40GE2/0/4] ip address 11.254.46.161 30 
    [*Leaf-CE6851HI-5&CE6851HI-6-40GE2/0/4] commit 
    [~Leaf-CE6851HI-5&CE6851HI-6-40GE2/0/4] quit

  2. Configure an IP address for the loopback interface.

    [~Leaf-CE6851HI-5&CE6851HI-6] interface loopback 0 
    [*Leaf-CE6851HI-5&CE6851HI-6] description VTEP&Router-ID 
    [*Leaf-CE6851HI-5&CE6851HI-6-LoopBack0] ip address 11.11.11.17 32 
    [*Leaf-CE6851HI-5&CE6851HI-6-LoopBack0] commit 
    [~Leaf-CE6851HI-5&CE6851HI-6-LoopBack0] quit

Configuring Server Access

The following describes configurations of switches in an SVF system to which a service server connects.

  1. Connect servers to the SVF in load balancing mode.

    [*Leaf-CE6851HI-5&CE6851HI-6] interface eth-trunk 1 
    [*Leaf-CE6851HI-5&CE6851HI-6-Eth-Trunk1] mode lacp-static 
    [*Leaf-CE6851HI-5&CE6851HI-6-Eth-Trunk1] port link-type trunk 
    [*Leaf-CE6851HI-5&CE6851HI-6-Eth-Trunk1] undo port trunk allow-pass vlan 1 
    [*Leaf-CE6851HI-5&CE6851HI-6-Eth-Trunk1] trunkport 10ge 101/0/1 
    [*Leaf-CE6851HI-5&CE6851HI-6-Eth-Trunk1] trunkport 10ge 101/0/2 
    [*Leaf-CE6851HI-5&CE6851HI-6-Eth-Trunk1] quit 
    [*Leaf-CE6851HI-5&CE6851HI-6] commit

  2. Configure VXLAN service access points.

    [~Leaf-CE6851HI-5&CE6851HI-6] bridge-domain 10 
    [*Leaf-CE6851HI-5&CE6851HI-6-bd10] vxlan vni 10010 
    [*Leaf-CE6851HI-5&CE6851HI-6-bd10] quit 
    [*Leaf-CE6851HI-5&CE6851HI-6] commit 
     
    [~Leaf-CE6851HI-5&CE6851HI-6] interface eth-trunk 1.1 mode l2 
    [*Leaf-CE6851HI-5&CE6851HI-6-Eth-Trunk1.1] encapsulation dot1q vid 30 
    [*Leaf-CE6851HI-5&CE6851HI-6-Eth-Trunk1.1] bridge-domain 10 
    [*Leaf-CE6851HI-5&CE6851HI-6-Eth-Trunk1.1] quit 
    [*Leaf-CE6851HI-5&CE6851HI-6] commit

Configuring Routes

Configure BGP routes on the underlay network as follows (in practical operations, OSPF can be used).

  1. Configure BGP routes on the SVF system to connect the system to spine nodes.

    [~Leaf-CE6851HI-5&CE6851HI-6] bgp 65024 
    [*Leaf-CE6851HI-5&CE6851HI-6-bgp] router-id 11.11.11.17 
    [*Leaf-CE6851HI-5&CE6851HI-6-bgp] timer keepalive 10 hold 30 
     
    [*Leaf-CE6851HI-5&CE6851HI-6-bgp] group Spine-CE12804 external 
    [*Leaf-CE6851HI-5&CE6851HI-6-bgp] peer Spine-CE12804 as-number 65010 
    [*Leaf-CE6851HI-5&CE6851HI-6-bgp] peer 11.254.40.158 as-number 65010 
    [*Leaf-CE6851HI-5&CE6851HI-6-bgp] peer 11.254.40.158 group Spine-CE12804  
    [*Leaf-CE6851HI-5&CE6851HI-6-bgp] peer 11.254.40.170 as-number 65010 
    [*Leaf-CE6851HI-5&CE6851HI-6-bgp] peer 11.254.40.170 group Spine-CE12804 
    [*Leaf-CE6851HI-5&CE6851HI-6-bgp] peer 11.254.40.166 as-number 65010 
    [*Leaf-CE6851HI-5&CE6851HI-6-bgp] peer 11.254.40.166 group Spine-CE12804      
    [*Leaf-CE6851HI-5&CE6851HI-6-bgp] peer 11.254.40.162 as-number 65010 
    [*Leaf-CE6851HI-5&CE6851HI-6-bgp] peer 11.254.40.162 group Spine-CE12804 
    [*Leaf-CE6851HI-5&CE6851HI-6-bgp] ipv4-family unicast 
    [*Leaf-CE6851HI-5&CE6851HI-6-bgp-af-ipv4] preference 20 200 10 
    [*Leaf-CE6851HI-5&CE6851HI-6-bgp-af-ipv4] network 11.11.11.17 255.255.255.255 
    [*Leaf-CE6851HI-5&CE6851HI-6-bgp-af-ipv4] maximum load-balancing 32 
    [*Leaf-CE6851HI-5&CE6851HI-6-bgp-af-ipv4] quit 
    [*Leaf-CE6851HI-5&CE6851HI-6-bgp] quit 
    [*Leaf-CE6851HI-5&CE6851HI-6] commit

Configuring BGP EVPN

When BGP EVPN is configured as the VXLAN control plane, perform the following steps to configure the leaf nodes as the RR clients and spine nodes as the RRs, establish IBGP EVPN peer relationship between the spine and leaf nodes, and configure EVPN instances, ingress replication, and Layer 3 gateways.

  1. Configure EVPN as the VXLAN control plane on the SVF system.

    [~Leaf-CE6851HI-5&CE6851HI-6] evpn-overlay enable 
    [*Leaf-CE6851HI-5&CE6851HI-6] commit

  2. Establish IBGP EVPN peer relationships between the SVF and spine nodes and use the spine nodes as the RRs.

    [~Leaf-CE6851HI-5&CE6851HI-6] bgp 100 instance evpn1 
    [*Leaf-CE6851HI-5&CE6851HI-6-bgp-instance-evpn1] router-id 11.11.11.17 
    [*Leaf-CE6851HI-5&CE6851HI-6-bgp-instance-evpn1] peer 11.11.11.14 as-number 100 
    //Establish a BGP EVPN peer relationship with Spine-CE12804-1. 
    [*Leaf-CE6851HI-5&CE6851HI-6-bgp-instance-evpn1] peer 11.11.11.14 connect-interface LoopBack0 
    [*Leaf-CE6851HI-5&CE6851HI-6-bgp-instance-evpn1] peer 11.11.11.15 as-number 100 
    //Establish a BGP EVPN peer relationship with Spine-CE12804-2.
    [*Leaf-CE6851HI-5&CE6851HI-6-bgp-instance-evpn1] peer 11.11.11.15 connect-interface LoopBack0 
    [*Leaf-CE6851HI-5&CE6851HI-6-bgp-instance-evpn1] l2vpn-family evpn 
    [*Leaf-CE6851HI-5&CE6851HI-6-bgp-instance-evpn1-af-evpn] peer 11.11.11.14 enable 
    [*Leaf-CE6851HI-5&CE6851HI-6-bgp-instance-evpn1-af-evpn] peer 11.11.11.14 advertise irb 
    [*Leaf-CE6851HI-5&CE6851HI-6-bgp-instance-evpn1-af-evpn] peer 11.11.11.15 enable 
    [*Leaf-CE6851HI-5&CE6851HI-6-bgp-instance-evpn1-af-evpn] peer 11.11.11.15 advertise irb 
    [*Leaf-CE6851HI-5&CE6851HI-6-bgp-instance-evpn1-af-evpn] quit 
    [*Leaf-CE6851HI-5&CE6851HI-6-bgp-instance-evpn1] quit 
    [*Leaf-CE6851HI-5&CE6851HI-6] commit

  3. Configure an EVPN instance in the SVF system.

    # Configure an EVPN instance in the SVF system.

    [~Leaf-CE6851HI-5&CE6851HI-6] bridge-domain 10  
    [~Leaf-CE6851HI-5&CE6851HI-6-bd10] evpn  
    [*Leaf-CE6851HI-5&CE6851HI-6-bd10-evpn] route-distinguisher 15:1 
    [*Leaf-CE6851HI-5&CE6851HI-6-bd10-evpn] vpn-target 10:1  
    [*Leaf-CE6851HI-5&CE6851HI-6-bd10-evpn] vpn-target 11:1 export-extcommunity    //Configure the RT value for the VPN instance.
    [*Leaf-CE6851HI-5&CE6851HI-6-bd10-evpn] quit 
    [*Leaf-CE6851HI-5&CE6851HI-6-bd10] quit  
    [*Leaf-CE6851HI-5&CE6851HI-6] commit

    # Configure a VPN instance in the SVF system.

    [~Leaf-CE6851HI-5&CE6851HI-3] ip vpn-instance vpn2  
    [*Leaf-CE6851HI-5&CE6851HI-3-vpn-instance-vpn2] vxlan vni 10  
    [*Leaf-CE6851HI-5&CE6851HI-3-vpn-instance-vpn2] ipv4-family 
    [*Leaf-CE6851HI-5&CE6851HI-3-vpn-instance-vpn2-af-ipv4] route-distinguisher 12:4  
    [*Leaf-CE6851HI-5&CE6851HI-3-vpn-instance-vpn2-af-ipv4] vpn-target 11:1 
    [*Leaf-CE6851HI-5&CE6851HI-3-vpn-instance-vpn2-af-ipv4] vpn-target 11:1 evpn    //Contain the EVPN parameter for interworking with an EVPN instance.
    [*Leaf-CE6851HI-5&CE6851HI-3-vpn-instance-vpn2-af-ipv4] quit 
    [*Leaf-CE6851HI-5&CE6851HI-3-vpn-instance-vpn2] quit  
    [*Leaf-CE6851HI-5&CE6851HI-3] commit

  4. Configure ingress replication in the SVF system.

    [~Leaf-CE6851HI-5&CE6851HI-6] interface nve 1  
    [*Leaf-CE6851HI-5&CE6851HI-6-Nve1] source 11.11.11.17  
    [*Leaf-CE6851HI-5&CE6851HI-6-Nve1] vni 10010 head-end peer-list protocol bgp  
    [*Leaf-CE6851HI-5&CE6851HI-6-Nve1] quit 
    [*Leaf-CE6851HI-5&CE6851HI-6] commit

  5. Configure Layer 3 gateways in the SVF system.

    [~Leaf-CE6851HI-5&CE6851HI-6] interface vbdif 10 
    [*Leaf-CE6851HI-5&CE6851HI-6-Vbdif10] ip binding vpn-instance vpn2 
    [*Leaf-CE6851HI-5&CE6851HI-6-Vbdif10] ip address 11.254.10.5 255.255.255.0 
    [*Leaf-CE6851HI-5&CE6851HI-6-Vbdif10] arp distribute-gateway enable 
    [*Leaf-CE6851HI-5&CE6851HI-6-Vbdif10] arp collect host enable 
    [*Leaf-CE6851HI-5&CE6851HI-6-Vbdif10] quit 
    [*Leaf-CE6851HI-5&CE6851HI-6] commit

Configuring Spine Nodes

Configure addresses and routes for the uplink and downlink interconnection interfaces on the two spine nodes to enable Layer 3 communication on the underlay network.

Configuration Roadmap
  1. Configuring IP address: Configure IP addresses for Layer 3 interconnection between the spine and leaf nodes as well as between the spine nodes and gateways, and loopback addresses (used as the router ID).
  2. Configuring routes: Configure BGP dynamic routes on the two spine nodes to establish peer relationships with two stacked switches (leaf nodes), two M-LAG switches (leaf nodes), and two M-LAG gateways. Ensure that the routes are reachable at Layer 3.
  3. Configuring BGP EVPN: Configure EVPN as the VXLAN control plane and configure BGP EVPN peers. Use the spine nodes as the RRs and TORs or gateways as the RR clients.
Configuring IP Addresses
  1. Configure IP addresses for interconnection interfaces.

    # Configure IP addresses for interfaces on Spine-CE12804-1.

    [~HUAWEI] sysname Spine-CE12804-1 
    [*SHUAWEI] commit 
    [~Spine-CE12804-1] interface 40ge 1/0/0 
    [~Spine-CE12804-1-40GE1/0/0] undo portswitch 
    [*Spine-CE12804-1-40GE1/0/0] ip address 11.254.40.158 30 
    [*Spine-CE12804-1-40GE1/0/0] commit 
    [~Spine-CE12804-1-40GE1/0/0] quit 
    [~Spine-CE12804-1] interface 40ge 1/0/1 
    [~Spine-CE12804-1-40GE1/0/1] undo portswitch 
    [*Spine-CE12804-1-40GE1/0/1] ip address 11.254.40.170 30 
    [*Spine-CE12804-1-40GE1/0/1] commit 
    [~Spine-CE12804-1-40GE1/0/1] quit 
    [~Spine-CE12804-1] interface 40ge 1/0/2 
    [~Spine-CE12804-1-40GE1/0/2] description "to-Leaf-CE6851-3-40GE1/0/3" 
    [*Spine-CE12804-1-40GE1/0/2] undo portswitch 
    [*Spine-CE12804-1-40GE1/0/2] ip address 11.254.41.158 30 
    [*Spine-CE12804-1-40GE1/0/2] commit 
    [~Spine-CE12804-1-40GE1/0/2] quit 
    [~Spine-CE12804-1] interface 40ge 1/0/3 
    [*Spine-CE12804-1-40GE1/0/3] description "to-Leaf-CE6851-4-40GE1/0/4" 
    [*Spine-CE12804-1-40GE1/0/3] undo portswitch 
    [*Spine-CE12804-1-40GE1/0/3] ip address 11.254.41.170 30 
    [*Spine-CE12804-1-40GE1/0/3] commit 
    [~Spine-CE12804-1-40GE1/0/3] quit 
    [~Spine-CE12804-1] interface 40ge 1/0/4 
    [~Spine-CE12804-1-40GE1/0/4] undo portswitch 
    [*Spine-CE12804-1-40GE1/0/4] ip address 11.254.42.157 30 
    [*Spine-CE12804-1-40GE1/0/4] commit 
    [~Spine-CE12804-1-40GE1/0/4] quit 
    [~Spine-CE12804-1] interface 40ge 1/0/5 
    [~Spine-CE12804-1-40GE1/0/5] undo portswitch 
    [*Spine-CE12804-1-40GE1/0/5] ip address 11.254.42.161 30 
    [*Spine-CE12804-1-40GE1/0/5] commit 
    [~Spine-CE12804-1-40GE1/0/5] quit 
    [~Spine-CE12804-1] interface 40ge 1/0/6 
    [~Spine-CE12804-1-40GE1/0/6] undo portswitch 
    [*Spine-CE12804-1-40GE1/0/6] ip address 11.254.46.158 30 
    [*Spine-CE12804-1-40GE1/0/6] commit 
    [~Spine-CE12804-1-40GE1/0/6] quit 
    [~Spine-CE12804-1] interface 40ge 1/0/7 
    [~Spine-CE12804-1-40GE1/0/7] undo portswitch 
    [*Spine-CE12804-1-40GE1/0/7] ip address 11.254.46.170 30 
    [*Spine-CE12804-1-40GE1/0/7] commit 
    [~Spine-CE12804-1-40GE1/0/7] quit

    # Configure IP addresses for interfaces on Spine-CE12804-2.

    [~HUAWEI] sysname Spine-CE12804-2 
    [*HUAWEI] commit 
    [~Spine-CE12804-2] interface 40ge 1/0/0 
    [~Spine-CE12804-2-40GE1/0/0] undo portswitch 
    [*Spine-CE12804-2-40GE1/0/0] ip address 11.254.40.162 30 
    [*Spine-CE12804-2-40GE1/0/0] commit 
    [~Spine-CE12804-2-40GE1/0/0] quit 
    [~Spine-CE12804-2] interface 40ge 1/0/1 
    [~Spine-CE12804-2-40GE1/0/1] undo portswitch 
    [*Spine-CE12804-2-40GE1/0/1] ip address 11.254.40.166 30 
    [*Spine-CE12804-2-40GE1/0/1] commit 
    [~Spine-CE12804-2-40GE1/0/1] quit 
    [~Spine-CE12804-2] interface 40ge 1/0/2 
    [*Spine-CE12804-2-40GE1/0/2] description "to-Leaf-CE6851-4-40GE1/0/3" 
    [*Spine-CE12804-2-40GE1/0/2] undo portswitch 
    [*Spine-CE12804-2-40GE1/0/2] ip address 11.254.41.162 30 
    [*Spine-CE12804-2-40GE1/0/2] commit 
    [~Spine-CE12804-2-40GE1/0/2] quit 
    [~Spine-CE12804-2] interface 40ge 1/0/3 
    [*Spine-CE12804-2-40GE1/0/3] description "to-Leaf-CE6851-3-40GE1/0/4" 
    [*Spine-CE12804-2-40GE1/0/3] undo portswitch 
    [*Spine-CE12804-2-40GE1/0/3] ip address 11.254.41.166 30 
    [*Spine-CE12804-2-40GE1/0/3] commit 
    [~Spine-CE12804-2-40GE1/0/3] quit 
    [~Spine-CE12804-2] interface 40ge 1/0/4 
    [~Spine-CE12804-2-40GE1/0/4] undo portswitch 
    [*Spine-CE12804-2-40GE1/0/4] ip address 11.254.43.157 30 
    [*Spine-CE12804-2-40GE1/0/4] commit 
    [~Spine-CE12804-2-40GE1/0/4] quit 
    [~Spine-CE12804-2] interface 40ge 1/0/5 
    [~Spine-CE12804-2-40GE1/0/5] undo portswitch 
    [*Spine-CE12804-2-40GE1/0/5] ip address 11.254.43.161 30 
    [*Spine-CE12804-2-40GE1/0/5] commit 
    [~Spine-CE12804-2-40GE1/0/5] quit 
    [~Spine-CE12804-2] interface 40ge 1/0/6 
    [~Spine-CE12804-2-40GE1/0/6] undo portswitch 
    [*Spine-CE12804-2-40GE1/0/6] ip address 11.254.46.166 30 
    [*Spine-CE12804-2-40GE1/0/6] commit 
    [~Spine-CE12804-2-40GE1/0/6] quit 
    [~Spine-CE12804-2] interface 40ge 1/0/7 
    [~Spine-CE12804-2-40GE1/0/7] undo portswitch 
    [*Spine-CE12804-2-40GE1/0/7] ip address 11.254.46.162 30 
    [*Spine-CE12804-2-40GE1/0/7] commit 
    [~Spine-CE12804-2-40GE1/0/7] quit

  2. Configure IP addresses for management interfaces.

    [~Spine-CE12804-1] interface loopback 0 
    [*Spine-CE12804-1-LoopBack0] ip address 11.11.11.14 32 
    [*Spine-CE12804-1-LoopBack0] commit 
    [~Spine-CE12804-1-LoopBack0] quit 
     
    [~Spine-CE12804-2] interface loopback 0 
    [*Spine-CE12804-2-LoopBack0] ip address 11.11.11.15 32 
    [*Spine-CE12804-2-LoopBack0] commit 
    [~Spine-CE12804-2-LoopBack0] quit

Configuring Routes

Configure BGP routes on the underlay network as follows (in practical operations, OSPF can be used).

  1. Configure BGP routes on Spine-CE12804-1.

    [~Spine-CE12804-1] bgp 65009 
    [*Spine-CE12804-1-bgp] router-id 11.11.11.14 
    [*Spine-CE12804-1-bgp] timer keepalive 10 hold 30 
    [*Spine-CE12804-1-bgp] group Leaf-CE6851HI-1&CE6851HI-2 external 
    [*Spine-CE12804-1-bgp] peer Leaf-CE6851HI-1&CE6851HI-2 as-number 65021 
    [*Spine-CE12804-1-bgp] peer 11.254.40.157 as-number 65021  
    [*Spine-CE12804-1-bgp] peer 11.254.40.157 group Leaf-CE6851HI-1&CE6851HI-2 
    [*Spine-CE12804-1-bgp] peer 11.254.40.169 as-number 65021  
    [*Spine-CE12804-1-bgp] peer 11.254.40.169 group Leaf-CE6851HI-1&CE6851HI-2 
    [*Spine-CE12804-1-bgp] group Leaf-CE6851HI-3&4 external 
    [*Spine-CE12804-1-bgp] peer Leaf-CE6851HI-3&4 as-number 65022 
    [*Spine-CE12804-1-bgp] peer 11.254.41.157 as-number 65022 
    [*Spine-CE12804-1-bgp] peer 11.254.41.157 group Leaf-CE6851HI-3&4 
    [*Spine-CE12804-1-bgp] peer 11.254.41.169 as-number 65022 
    [*Spine-CE12804-1-bgp] peer 11.254.41.169 group Leaf-CE6851HI-3&4 
     
    [*Spine-CE12804-1-bgp] group Leaf-CE6851HI-5&CE6851HI-6 external 
    [*Spine-CE12804-1-bgp] peer Leaf-CE6851HI-5&CE6851HI-6 as-number 65024 
    [*Spine-CE12804-1-bgp] peer 11.254.46.157 as-number 65024  
    [*Spine-CE12804-1-bgp] peer 11.254.46.157 group Leaf-CE6851HI-5&CE6851HI-6 
    [*Spine-CE12804-1-bgp] peer 11.254.46.169 as-number 65024  
    [*Spine-CE12804-1-bgp] peer 11.254.46.169 group Leaf-CE6851HI-5&CE6851HI-6  
    [*Spine-CE12804-1-bgp] group Gateway-CE12808 external 
    [*Spine-CE12804-1-bgp] peer Gateway-CE12808 as-number 65000 
    [*Spine-CE12804-1-bgp] peer 11.254.42.158 as-number 65000 
    [*Spine-CE12804-1-bgp] peer 11.254.42.158 group Gateway-CE12808 
    [*Spine-CE12804-1-bgp] peer 11.254.42.162 as-number 65000 
    [*Spine-CE12804-1-bgp] peer 11.254.42.162 group Gateway-CE12808 
     
    [*Spine-CE12804-1-bgp] ipv4-family unicast 
    [*Spine-CE12804-1-bgp-af-ipv4] preference 20 200 10 
    [*Spine-CE12804-1-bgp-af-ipv4] network 11.11.11.14 255.255.255.255 
    [*Spine-CE12804-1-bgp-af-ipv4] maximum load-balancing 32 
    [*Spine-CE12804-1-bgp-af-ipv4] quit 
    [*Spine-CE12804-1-bgp] quit 
    [*Spine-CE12804-1] commit

  2. Configure BGP routes on Spine-CE12804-2.

    [~Spine-CE12804-2] bgp 65010 
    [*Spine-CE12804-2-bgp] router-id 11.11.11.15 
    [*Spine-CE12804-2-bgp] timer keepalive 10 hold 30 
    [*Spine-CE12804-2-bgp] group Leaf-CE6851HI-1&CE6851HI-2 external 
    [*Spine-CE12804-2-bgp] peer Leaf-CE6851HI-1&CE6851HI-2 as-number 65021 
    [*Spine-CE12804-2-bgp] peer 11.254.40.165 as-number 65021 
    [*Spine-CE12804-2-bgp] peer 11.254.40.165 group Leaf-CE6851HI-1&CE6851HI-2 
    [*Spine-CE12804-2-bgp] peer 11.254.40.161 as-number 65021 
    [*Spine-CE12804-2-bgp] peer 11.254.40.161 group Leaf-CE6851HI-1&CE6851HI-2 
    [*Spine-CE12804-2-bgp] group Leaf-CE6851HI-3&4 external 
    [*Spine-CE12804-2-bgp] peer Leaf-CE6851HI-3&4 as-number 65022 
    [*Spine-CE12804-2-bgp] peer 11.254.41.165 as-number 65022 
    [*Spine-CE12804-2-bgp] peer 11.254.41.165 group Leaf-CE6851HI-3&4 
    [*Spine-CE12804-2-bgp] peer 11.254.41.161 as-number 65022 
    [*Spine-CE12804-2-bgp] peer 11.254.41.161 group Leaf-CE6851HI-3&4 
     
    [*Spine-CE12804-2-bgp] group Gateway-CE12808 external 
    [*Spine-CE12804-2-bgp] peer Gateway-CE12808 as-number 65000 
    [*Spine-CE12804-2-bgp] peer 11.254.43.162 as-number 65000 
    [*Spine-CE12804-2-bgp] peer 11.254.43.162 group Gateway-CE12808 
    [*Spine-CE12804-2-bgp] peer 11.254.43.158 as-number 65000 
    [*Spine-CE12804-2-bgp] peer 11.254.43.158 group Gateway-CE12808 
    [*Spine-CE12804-2-bgp] group Leaf-CE6851HI-5&CE6851HI-6 external 
    [*Spine-CE12804-2-bgp] peer Leaf-CE6851HI-5&CE6851HI-6 as-number 65024 
    [*Spine-CE12804-2-bgp] peer 11.254.46.165 as-number 65024 
    [*Spine-CE12804-2-bgp] peer 11.254.46.165 group Leaf-CE6851HI-5&CE6851HI-6 
    [*Spine-CE12804-2-bgp] peer 11.254.46.161 as-number 65024 
    [*Spine-CE12804-2-bgp] peer 11.254.46.161 group Leaf-CE6851HI-5&CE6851HI-6 
     
    [*Spine-CE12804-2-bgp] ipv4-family unicast 
    [*Spine-CE12804-2-bgp-af-ipv4] preference 20 200 10 
    [*Spine-CE12804-2-bgp-af-ipv4] network 11.11.11.15 255.255.255.255 
    [*Spine-CE12804-2-bgp-af-ipv4] maximum load-balancing 32 
    [*Spine-CE12804-2-bgp-af-ipv4] quit 
    [*Spine-CE12804-2-bgp] quit 
    [*Spine-CE12804-2] commit

Configuring BGP EVPN

When BGP EVPN is configured as the VXLAN control plane, perform the following steps to configure the TORs as the RR clients and spine nodes as the RRs and establish IBGP EVPN peer relationships between the spine and leaf nodes as well as between the spine nodes and gateways.

  1. Configure EVPN as the VXLAN control plane on Spine-CE12804-1 and Spine-CE12804-2.

    [~Spine-CE12804-1] evpn-overlay enable 
    [*Spine-CE12804-1] commit 
     
    [~Spine-CE12804-2] evpn-overlay enable 
    [*Spine-CE12804-2] commit

  2. On Spine-CE12804-1 and Spine-CE12804-2, configure BGP EVPN peer relationships to the leaf nodes or gateways and use the spine nodes as the RRs.

    # On Spine-CE12804-1, configure IBGP EVPN peer relationships to the leaf nodes or gateways and use Spine-CE12804-1 as the RRs and leaf nodes or gateways as the RR clients.

    [~Spine-CE12804-1] bgp 100 instance evpn1 
    [*Spine-CE12804-1-bgp-instance-evpn1] router-id 11.11.11.14 
    [*Spine-CE12804-1-bgp-instance-evpn1] peer 11.11.11.11 as-number 100 
    //Establish a BGP EVPN peer relationship with switches in the stack. 
    [*Spine-CE12804-1-bgp-instance-evpn1] peer 11.11.11.11 connect-interface loopback 0 
    [*Spine-CE12804-1-bgp-instance-evpn1] peer 13.13.13.13 as-number 100 
    //Establish a BGP EVPN peer relationship with Leaf-CE6851HI-3. 
    [*Spine-CE12804-1-bgp-instance-evpn1] peer 13.13.13.13 connect-interface loopback 0 
    [*Spine-CE12804-1-bgp-instance-evpn1] peer 14.14.14.14 as-number 100 
    //Establish a BGP EVPN peer relationship with Leaf-CE6851HI-4. 
    [*Spine-CE12804-1-bgp-instance-evpn1] peer 14.14.14.14 connect-interface loopback 0 
    [*Spine-CE12804-1-bgp-instance-evpn1] peer 11.11.11.17 as-number 100 
    //Establish a BGP EVPN peer relationship with switches in the SVF system. 
    [*Spine-CE12804-1-bgp-instance-evpn1] peer 11.11.11.17 connect-interface loopback 0 
    [*Spine-CE12804-1-bgp-instance-evpn1] peer 18.18.18.18 as-number 100 
    ///Establish a BGP EVPN peer relationship with Exit-Gateway-CE12808-1. 
    [*Spine-CE12804-1-bgp-instance-evpn1] peer 18.18.18.18 connect-interface loopback 0 
    [*Spine-CE12804-1-bgp-instance-evpn1] peer 19.19.19.19 as-number 100 
    //Establish a BGP EVPN peer relationship with Exit-Gateway-CE12808-2. 
    [*Spine-CE12804-1-bgp-instance-evpn1] peer 19.19.19.19 connect-interface loopback 0 
    [*Spine-CE12804-1-bgp-instance-evpn1] l2vpn-family evpn 
    [*Spine-CE12804-1-bgp-instance-evpn1-af-evpn] undo policy vpn-target 
    [*Spine-CE12804-1-bgp-instance-evpn1-af-evpn] peer 11.11.11.11 enable 
    [*Spine-CE12804-1-bgp-instance-evpn1-af-evpn] peer 11.11.11.11 reflect-client 
    [*Spine-CE12804-1-bgp-instance-evpn1-af-evpn] peer 11.11.11.11 next-hop-invariable 
    [*Spine-CE12804-1-bgp-instance-evpn1-af-evpn] peer 11.11.11.11 advertise irb 
    [*Spine-CE12804-1-bgp-instance-evpn1-af-evpn] peer 13.13.13.13 enable 
    [*Spine-CE12804-1-bgp-instance-evpn1-af-evpn] peer 13.13.13.13 reflect-client 
    [*Spine-CE12804-1-bgp-instance-evpn1-af-evpn] peer 13.13.13.13 next-hop-invariable 
    [*Spine-CE12804-1-bgp-instance-evpn1-af-evpn] peer 13.13.13.13 advertise irb 
    [*Spine-CE12804-1-bgp-instance-evpn1-af-evpn] peer 14.14.14.14 enable 
    [*Spine-CE12804-1-bgp-instance-evpn1-af-evpn] peer 14.14.14.14 reflect-client 
    [*Spine-CE12804-1-bgp-instance-evpn1-af-evpn] peer 14.14.14.14 next-hop-invariable 
    [*Spine-CE12804-1-bgp-instance-evpn1-af-evpn] peer 14.14.14.14 advertise irb 
    [*Spine-CE12804-1-bgp-instance-evpn1-af-evpn] peer 11.11.11.17 enable 
    [*Spine-CE12804-1-bgp-instance-evpn1-af-evpn] peer 11.11.11.17 reflect-client 
    [*Spine-CE12804-1-bgp-instance-evpn1-af-evpn] peer 11.11.11.17 next-hop-invariable 
    [*Spine-CE12804-1-bgp-instance-evpn1-af-evpn] peer 11.11.11.17 advertise irb 
    [*Spine-CE12804-1-bgp-instance-evpn1-af-evpn] peer 18.18.18.18 enable 
    [*Spine-CE12804-1-bgp-instance-evpn1-af-evpn] peer 18.18.18.18 reflect-client 
    [*Spine-CE12804-1-bgp-instance-evpn1-af-evpn] peer 18.18.18.18 next-hop-invariable 
    [*Spine-CE12804-1-bgp-instance-evpn1-af-evpn] peer 18.18.18.18 advertise irb 
    [*Spine-CE12804-1-bgp-instance-evpn1-af-evpn] peer 19.19.19.19 enable 
    [*Spine-CE12804-1-bgp-instance-evpn1-af-evpn] peer 19.19.19.19 reflect-client 
    [*Spine-CE12804-1-bgp-instance-evpn1-af-evpn] peer 19.19.19.19 next-hop-invariable 
    [*Spine-CE12804-1-bgp-instance-evpn1-af-evpn] peer 19.19.19.19 advertise irb 
    [*Spine-CE12804-1-bgp-instance-evpn1-af-evpn] quit 
    [*Spine-CE12804-1-bgp-instance-evpn1] quit 
    [*Spine-CE12804-1] commit

    # On Spine-CE12804-2, configure IBGP EVPN peer relationships to the leaf nodes and use Spine-CE12804-2 as the RRs and leaf nodes as the RR clients.

    [~Spine-CE12804-2] bgp 100 instance evpn1 
    [*Spine-CE12804-2-bgp-instance-evpn1] router-id 11.11.11.15 
    [*Spine-CE12804-2-bgp-instance-evpn1] peer 11.11.11.11 as-number 100 
    //Establish a BGP EVPN peer relationship with TOR switches in the stack.
    [*Spine-CE12804-2-bgp-instance-evpn1] peer 11.11.11.11 connect-interface loopback 0 
    [*Spine-CE12804-2-bgp-instance-evpn1] peer 13.13.13.13 as-number 100 
    //Establish a BGP EVPN peer relationship with Leaf-CE6851HI-3. 
    [*Spine-CE12804-2-bgp-instance-evpn1] peer 13.13.13.13 connect-interface loopback 0 
    [*Spine-CE12804-2-bgp-instance-evpn1] peer 14.14.14.14 as-number 100 
    //Establish a BGP EVPN peer relationship with Leaf-CE6851HI-4. 
    [*Spine-CE12804-2-bgp-instance-evpn1] peer 14.14.14.14 connect-interface loopback 0 
    [*Spine-CE12804-2-bgp-instance-evpn1] peer 11.11.11.17 as-number 100 
    //Establish a BGP EVPN peer relationship with switches in the SVF system. 
    [*Spine-CE12804-2-bgp-instance-evpn1] peer 11.11.11.17 connect-interface loopback 0 
    [*Spine-CE12804-2-bgp-instance-evpn1] peer 18.18.18.18 as-number 100 
    //Establish a BGP EVPN peer relationship with Exit-Gateway-CE12808-1. 
    [*Spine-CE12804-2-bgp-instance-evpn1] peer 18.18.18.18 connect-interface loopback 0 
    [*Spine-CE12804-2-bgp-instance-evpn1] peer 19.19.19.19 as-number 100 
    //Establish a BGP EVPN peer relationship with Exit-Gateway-CE12808-1. 
    [*Spine-CE12804-2-bgp-instance-evpn1] peer 19.19.19.19 connect-interface loopback 0 
    [*Spine-CE12804-2-bgp-instance-evpn1] l2vpn-family evpn 
    [*Spine-CE12804-2-bgp-instance-evpn1-af-evpn] undo policy vpn-target 
    [*Spine-CE12804-2-bgp-instance-evpn1-af-evpn] peer 11.11.11.11 enable 
    [*Spine-CE12804-2-bgp-instance-evpn1-af-evpn] peer 11.11.11.11 reflect-client 
    [*Spine-CE12804-2-bgp-instance-evpn1-af-evpn] peer 11.11.11.11 next-hop-invariable 
    [*Spine-CE12804-2-bgp-instance-evpn1-af-evpn] peer 11.11.11.11 advertise irb 
    [*Spine-CE12804-2-bgp-instance-evpn1-af-evpn] peer 13.13.13.13 enable 
    [*Spine-CE12804-2-bgp-instance-evpn1-af-evpn] peer 13.13.13.13 reflect-client 
    [*Spine-CE12804-2-bgp-instance-evpn1-af-evpn] peer 13.13.13.13 next-hop-invariable 
    [*Spine-CE12804-2-bgp-instance-evpn1-af-evpn] peer 13.13.13.13 advertise irb 
    [*Spine-CE12804-2-bgp-instance-evpn1-af-evpn] peer 14.14.14.14 enable 
    [*Spine-CE12804-2-bgp-instance-evpn1-af-evpn] peer 14.14.14.14 reflect-client 
    [*Spine-CE12804-2-bgp-instance-evpn1-af-evpn] peer 14.14.14.14 next-hop-invariable 
    [*Spine-CE12804-2-bgp-instance-evpn1-af-evpn] peer 14.14.14.14 advertise irb 
    [*Spine-CE12804-2-bgp-instance-evpn1-af-evpn] peer 11.11.11.17 enable 
    [*Spine-CE12804-2-bgp-instance-evpn1-af-evpn] peer 11.11.11.17 reflect-client 
    [*Spine-CE12804-2-bgp-instance-evpn1-af-evpn] peer 11.11.11.17 next-hop-invariable 
    [*Spine-CE12804-2-bgp-instance-evpn1-af-evpn] peer 11.11.11.17 advertise irb 
    [*Spine-CE12804-2-bgp-instance-evpn1-af-evpn] peer 18.18.18.18 enable 
    [*Spine-CE12804-2-bgp-instance-evpn1-af-evpn] peer 18.18.18.18 reflect-client 
    [*Spine-CE12804-2-bgp-instance-evpn1-af-evpn] peer 18.18.18.18 next-hop-invariable 
    [*Spine-CE12804-2-bgp-instance-evpn1-af-evpn] peer 18.18.18.18 advertise irb 
    [*Spine-CE12804-2-bgp-instance-evpn1-af-evpn] peer 19.19.19.19 enable 
    [*Spine-CE12804-2-bgp-instance-evpn1-af-evpn] peer 19.19.19.19 reflect-client 
    [*Spine-CE12804-2-bgp-instance-evpn1-af-evpn] peer 19.19.19.19 next-hop-invariable 
    [*Spine-CE12804-2-bgp-instance-evpn1-af-evpn] peer 19.19.19.19 advertise irb 
    [*Spine-CE12804-2-bgp-instance-evpn1-af-evpn] quit 
    [*Spine-CE12804-2-bgp-instance-evpn1] quit 
    [*Spine-CE12804-2] commit

Configuring a Gateway Group

The CE series switches Exit-Gateway-CE12808-1 and Exit-Gateway-CE12808-2 function as egress gateways on the distributed VXLAN network. The two gateways can be deployed as a gateway group through stacking or M-LAG. In this example, an M-LAG is configured.

Configuration Roadmap
  1. Configuring IP addresses: Configure interface addresses to connect to the spine nodes, management VLANs of the firewalls, Loopback0 addresses (used as VTEP IP addresses), and Loopback1 addresses (used as router IDs).
  2. Configuring an M-LAG: Configure a global M-LAG, DFS group, and peer-link on the Layer 3 egress gateways, and configure a management VLAN and service links for the two firewalls.
  3. Configuring routes: Configure BGP dynamic routes on the two Layer 3 egress gateways to establish peer relationships with two spine devices and ensure that the routes are reachable at Layer 3. Configure routes between the Layer 3 egress gateways and external routers.
  4. Configuring BGP EVPN: Configure BGP EVPN as the VXLAN control plane and configure BGP EVPN peers.
  5. Configuring a MAC address flapping detection whitelist: Add the MAC addresses of the Layer 3 interfaces on the spine devices that connect to the gateways to a whitelist, so that MAC address flapping detection will not be performed on these addresses.
  6. Configuring interconnection between the gateways and external routers in mesh mode: Configure IP addresses for interfaces connecting the gateways and external routers as well as interfaces connecting the gateways, and configure default routes destined for the external routers on the gateways.
Configuring IP Addresses
  1. Configure IP addresses for interconnection interfaces.

    # Configure IP addresses for interfaces on Exit-Gateway-CE12808-1.

    [~Exit-Gateway-CE12808-1] interface 40ge 1/0/0 //Configure an interface address to connect to a spine node.  
    [~Exit-Gateway-CE12808-1-40GE1/0/0] undo portswitch 
    [*Exit-Gateway-CE12808-1-40GE1/0/0] ip address 11.254.42.158 30 
    [*Exit-Gateway-CE12808-1-40GE1/0/0] commit 
    [~Exit-Gateway-CE12808-1-40GE1/0/0] quit 
    [~Exit-Gateway-CE12808-1] interface 40ge 1/0/1   //Configure an interface address to connect to a spine node.  
    [~Exit-Gateway-CE12808-1-40GE1/0/1] undo portswitch 
    [*Exit-Gateway-CE12808-1-40GE1/0/1] ip address 11.254.43.162 30 
    [*Exit-Gateway-CE12808-1-40GE1/0/1] commit 
    [~Exit-Gateway-CE12808-1-40GE1/0/1] quit 
    [~Exit-Gateway-CE12808-1] vlan batch 11 
    [*Exit-Gateway-CE12808-1] interface vlanif 11 //Configure a VLAN for interconnection between the gateway and firewall through a public external network.
    [*Exit-Gateway-CE12808-1-Vlanif11] description "to firewall-1~2" 
    [*Exit-Gateway-CE12808-1-Vlanif11] ip address 11.254.45.154 29     
    [*Exit-Gateway-CE12808-1-Vlanif11] vrrp vrid 1 virtual-ip 11.254.45.153 
    [*Exit-Gateway-CE12808-1-Vlanif11] commit 
    [~Exit-Gateway-CE12808-1-Vlanif11] quit 
    [~Exit-Gateway-CE12808-1] vlan batch 12 
    [*Exit-Gateway-CE12808-1] interface vlanif 12 //Configure a VLAN for service interconnection between the gateway and firewall.
    [*Exit-Gateway-CE12808-1-Vlanif12] description "to firewall-1~2" 
    [*Exit-Gateway-CE12808-1-Vlanif12] ip address 11.254.45.162 29     
    [*Exit-Gateway-CE12808-1-Vlanif12] vrrp vrid 2 virtual-ip 11.254.45.161 
    [*Exit-Gateway-CE12808-1-Vlanif12] commit 
    [~Exit-Gateway-CE12808-1-Vlanif12] quit

    # Configure IP addresses for interfaces on Exit-Gateway-CE12808-2.

    [~Exit-Gateway-CE12808-2] interface 40ge 1/0/0   //Configure an interface address to connect to a spine node.  
    [~Exit-Gateway-CE12808-2-40GE1/0/0] undo portswitch 
    [*Exit-Gateway-CE12808-2-40GE1/0/0] ip address 11.254.42.162 30 
    [*Exit-Gateway-CE12808-2-40GE1/0/0] commit 
    [~Exit-Gateway-CE12808-2-40GE1/0/0] quit 
    [~Exit-Gateway-CE12808-2] interface 40ge 1/0/1    //Configure an interface address to connect to a spine node.  
    [~Exit-Gateway-CE12808-2-40GE1/0/1] undo portswitch 
    [*Exit-Gateway-CE12808-2-40GE1/0/1] ip address 11.254.43.158 30 
    [*Exit-Gateway-CE12808-2-40GE1/0/1] commit 
    [~Exit-Gateway-CE12808-2-40GE1/0/1] quit 
    [~Exit-Gateway-CE12808-2] vlan batch 11   
    [*Exit-Gateway-CE12808-2] interface vlanif 11   //Configure a VLAN for service interconnection between the gateway and firewall.
    [*Exit-Gateway-CE12808-2-Vlanif11] description "to firewall-1-2" 
    [*Exit-Gateway-CE12808-2-Vlanif11] ip address 11.254.45.155 29 
    [*Exit-Gateway-CE12808-2-Vlanif11] vrrp vrid 1 virtual-ip 11.254.45.153 
    [*Exit-Gateway-CE12808-2-Vlanif11] commit 
    [~Exit-Gateway-CE12808-2-Vlanif11] quit 
     
    [~Exit-Gateway-CE12808-2] vlan batch 12   
    [*Exit-Gateway-CE12808-2] interface vlanif 12   //Configure a VLAN for service interconnection between the gateway and firewall.
    [*Exit-Gateway-CE12808-2-Vlanif12] description "to firewall-1-2" 
    [*Exit-Gateway-CE12808-2-Vlanif12] ip address 11.254.45.163 29 
    [*Exit-Gateway-CE12808-2-Vlanif12] vrrp vrid 2 virtual-ip 11.254.45.161 
    [*Exit-Gateway-CE12808-2-Vlanif12] commit 
    [~Exit-Gateway-CE12808-2-Vlanif12] quit

  2. Configure IP addresses for loopback interfaces.

    [~Exit-Gateway-CE12808-1] interface loopback 0    //Configure the Loopback0 address as the VTEP IP address.  
    [*Exit-Gateway-CE12808-1-LoopBack0] ip address 11.11.11.16 32 
    [*Exit-Gateway-CE12808-1-LoopBack0] commit 
    [~Exit-Gateway-CE12808-1-LoopBack0] quit 
    [~Exit-Gateway-CE12808-1] interface loopback 1 
    [*Exit-Gateway-CE12808-1-LoopBack1] ip address 18.18.18.18 32 
    [*Exit-Gateway-CE12808-1-LoopBack1] commit 
    [~Exit-Gateway-CE12808-1-LoopBack1] quit 
    [~Exit-Gateway-CE12808-1] interface loopback 2 
    [*Exit-Gateway-CE12808-1-LoopBack2] ip address 21.21.21.21 32 
    [*Exit-Gateway-CE12808-1-LoopBack2] commit 
    [~Exit-Gateway-CE12808-1-LoopBack2] quit 
     
    [~Exit-Gateway-CE12808-2] interface loopback 0     //Configure the Loopback0 address as the VTEP IP address.  
    [*Exit-Gateway-CE12808-2-LoopBack0] ip address 11.11.11.16 32 
    [*Exit-Gateway-CE12808-2-LoopBack0] commit 
    [~Exit-Gateway-CE12808-2-LoopBack0] quit 
    [~Exit-Gateway-CE12808-2] interface loopback 1 
    [*Exit-Gateway-CE12808-2-LoopBack1] ip address 19.19.19.19 32 
    [*Exit-Gateway-CE12808-2-LoopBack1] commit 
    [~Exit-Gateway-CE12808-2-LoopBack1] quit 
    [~Exit-Gateway-CE12808-2] interface loopback 2 
    [*Exit-Gateway-CE12808-2-LoopBack2] ip address 22.22.22.22 32 
    [*Exit-Gateway-CE12808-2-LoopBack2] commit 
    [~Exit-Gateway-CE12808-2-LoopBack2] quit

Configuring Routes

Configure BGP routes on the underlay network as follows (in practical operations, OSPF can be used).

  1. Configure BGP routes on Exit-Gateway-CE12808-1 to set up routes for the underlay network.

    [~Exit-Gateway-CE12808-1] bgp 65000 
    [*Exit-Gateway-CE12808-1-bgp] router-id 18.18.18.18 
    [*Exit-Gateway-CE12808-1-bgp] timer keepalive 10 hold 30 
    [*Exit-Gateway-CE12808-1-bgp] group Spine-CE12804 external   //Configure routes to the spine nodes.  
    [*Exit-Gateway-CE12808-1-bgp] peer Spine-CE12804 as-number 65010 
    [*Exit-Gateway-CE12808-1-bgp] peer 11.254.42.157 as-number 65010 
    [*Exit-Gateway-CE12808-1-bgp] peer 11.254.42.157 group Spine-CE12804 
    [*Exit-Gateway-CE12808-1-bgp] peer 11.254.43.161 as-number 65010 
    [*Exit-Gateway-CE12808-1-bgp] peer 11.254.43.161 group Spine-CE12804 
     
    [*Exit-Gateway-CE12808-1-bgp] ipv4-family unicast 
    [*Exit-Gateway-CE12808-1-bgp-af-ipv4] preference 20 200 10 
    [*Exit-Gateway-CE12808-1-bgp-af-ipv4] network 11.11.11.16 255.255.255.255 
    [*Exit-Gateway-CE12808-1-bgp-af-ipv4] network 18.18.18.18 255.255.255.255 
    [*Exit-Gateway-CE12808-1-bgp-af-ipv4] network 11.254.45.152 255.255.255.248 
    [*Exit-Gateway-CE12808-1-bgp-af-ipv4] maximum load-balancing 32 
    [*Exit-Gateway-CE12808-1-bgp-af-ipv4] quit 
    [*Exit-Gateway-CE12808-1-bgp] quit 
    [*Exit-Gateway-CE12808-1] commit

  2. Configure BGP routes on Exit-Gateway-CE12808-2 to set up routes for the underlay network.

    [~Exit-Gateway-CE12808-2] bgp 65000 
    [*Exit-Gateway-CE12808-2-bgp] router-id 19.19.19.19 
    [*Exit-Gateway-CE12808-2-bgp] timer keepalive 10 hold 30 
    [*Exit-Gateway-CE12808-2-bgp] group Spine-CE12804 external    //Configure routes to the spine nodes.  
    [*Exit-Gateway-CE12808-2-bgp] peer Spine-CE12804 as-number 65010 
    [*Exit-Gateway-CE12808-2-bgp] peer 11.254.42.161 as-number 65010 
    [*Exit-Gateway-CE12808-2-bgp] peer 11.254.42.161 group Spine-CE12804 
    [*Exit-Gateway-CE12808-2-bgp] peer 11.254.43.157 as-number 65010 
    [*Exit-Gateway-CE12808-2-bgp] peer 11.254.43.157 group Spine-CE12804 
     
    [*Exit-Gateway-CE12808-2-bgp] ipv4-family unicast 
    [*Exit-Gateway-CE12808-2-bgp-af-ipv4] preference 20 200 10 
    [*Exit-Gateway-CE12808-2-bgp-af-ipv4] network 11.11.11.16 255.255.255.255 
    [*Exit-Gateway-CE12808-2-bgp-af-ipv4] network 19.19.19.19 255.255.255.255 
    [*Exit-Gateway-CE12808-2-bgp-af-ipv4] network 11.254.45.152 255.255.255.248 
    [*Exit-Gateway-CE12808-2-bgp-af-ipv4] maximum load-balancing 32 
    [*Exit-Gateway-CE12808-2-bgp-af-ipv4] quit 
    [*Exit-Gateway-CE12808-2-bgp] quit 
    [*Exit-Gateway-CE12808-2] commit

Configuring an M-LAG
  1. Configure the M-LAG mode.

    < Exit-Gateway-CE12808-1> system-view 
    [~Exit-Gateway-CE12808-1] stp mode rstp 
    [*Exit-Gateway-CE12808-1] stp v-stp enable 
    [*Exit-Gateway-CE12808-1] lacp m-lag priority 10 
    [~Exit-Gateway-CE12808-1] lacp m-lag system-id 00e0-fc00-0101      //You are advised to set system-id to the MAC address of the master device system of the M-LAG. Set system-id on the remote device to the same value. You can run the display system mac-address command to check the MAC address of a system.[*Exit-Gateway-CE12808-1] commit 
     
    <Exit-Gateway-CE12808-2> system-view 
    [~Exit-Gateway-CE12808-2] stp mode rstp 
    [*Exit-Gateway-CE12808-2] stp v-stp enable 
    [*Exit-Gateway-CE12808-2] commit 
    [*Exit-Gateway-CE12808-2] lacp m-lag priority 10 
    [~Exit-Gateway-CE12808-2] lacp m-lag system-id 00e0-fc00-0101 
    [*Exit-Gateway-CE12808-2] commit

  2. On Exit-Gateway-CE12808-1 and Exit-Gateway-CE12808-2, configure a DFS group for the M-LAG and set up a dual-active system.

    [~Exit-Gateway-CE12808-1] dfs-group 1 
    [*Exit-Gateway-CE12808-1-dfs-group-1] source ip 18.18.18.18 
    [*Exit-Gateway-CE12808-1-dfs-group-1] priority 150 
    [*Exit-Gateway-CE12808-1-dfs-group-1] quit 
    [*Exit-Gateway-CE12808-1] commit 
     
    [~Exit-Gateway-CE12808-2] dfs-group 1 
    [*Exit-Gateway-CE12808-2-dfs-group-1] source ip 19.19.19.19 
    [*Exit-Gateway-CE12808-2-dfs-group-1] priority 120 
    [*Exit-Gateway-CE12808-2-dfs-group-1] quit 
    [*Exit-Gateway-CE12808-2] commit

  3. On Exit-Gateway-CE12808-1 and Exit-Gateway-CE12808-2, configure a peer-link for the M-LAG.

    [~Exit-Gateway-CE12808-1] interface eth-trunk 0 
    [*Exit-Gateway-CE12808-1-Eth-Trunk0] trunkport 40ge 1/0/23 
    [*Exit-Gateway-CE12808-1-Eth-Trunk0] trunkport 40ge 2/0/23 
    [*Exit-Gateway-CE12808-1-Eth-Trunk0] mode lacp-static 
    [*Exit-Gateway-CE12808-1-Eth-Trunk0] peer-link 1 
    [*Exit-Gateway-CE12808-1-Eth-Trunk0] quit 
    [*Exit-Gateway-CE12808-1] commit 
     
    [~Exit-Gateway-CE12808-2] interface eth-trunk 0 
    [*Exit-Gateway-CE12808-2-Eth-Trunk0] trunkport 40ge 1/0/23 
    [*Exit-Gateway-CE12808-2-Eth-Trunk0] trunkport 40ge 2/0/23 
    [*Exit-Gateway-CE12808-2-Eth-Trunk0] mode lacp-static 
    [*Exit-Gateway-CE12808-2-Eth-Trunk0] peer-link 1 
    [*Exit-Gateway-CE12808-2-Eth-Trunk0] quit 
    [*Exit-Gateway-CE12808-2] commit

  4. On Exit-Gateway-CE12808-1 and Exit-Gateway-CE12808-2, configure M-LAG member interfaces. The following is an example for connecting to the firewalls.

    # Configure interconnection between Exit-Gateway-CE12808-1 and the firewalls.

    [~Exit-Gateway-CE12808-1] interface eth-trunk 21   
    [*Exit-Gateway-CE12808-1-Eth-Trunk21] description "to-FW-USG9560-1-GE1/0/3" 
    [*Exit-Gateway-CE12808-1-Eth-Trunk21] port link-type trunk 
    [*Exit-Gateway-CE12808-1-Eth-Trunk21] undo port trunk allow-pass vlan 1 
    [*Exit-Gateway-CE12808-1-Eth-Trunk21] port trunk allow-pass vlan 11 
    [*Exit-Gateway-CE12808-1-Eth-Trunk21] port trunk allow-pass vlan 12 
    [*Exit-Gateway-CE12808-1-Eth-Trunk21] trunkport 10ge 3/0/0 
    [*Exit-Gateway-CE12808-1-Eth-Trunk21] trunkport 10ge 3/0/1 
    [*Exit-Gateway-CE12808-1-Eth-Trunk21] dfs-group 1 m-lag 1 
    [*Exit-Gateway-CE12808-1-Eth-Trunk21] quit 
    [*Exit-Gateway-CE12808-1] interface eth-trunk 31 
    [*Exit-Gateway-CE12808-1-Eth-Trunk31] description "to-FW-USG9560-2-GE1/0/3" 
    [*Exit-Gateway-CE12808-1-Eth-Trunk31] port link-type trunk 
    [*Exit-Gateway-CE12808-1-Eth-Trunk31] undo port trunk allow-pass vlan 1 
    [*Exit-Gateway-CE12808-1-Eth-Trunk31] port trunk allow-pass vlan 11 
    [*Exit-Gateway-CE12808-1-Eth-Trunk31] port trunk allow-pass vlan 12 
    [*Exit-Gateway-CE12808-1-Eth-Trunk31] trunkport 10ge 3/0/2 
    [*Exit-Gateway-CE12808-1-Eth-Trunk31] trunkport 10ge 3/0/3 
    [*Exit-Gateway-CE12808-1-Eth-Trunk31] dfs-group 1 m-lag 2 
    [*Exit-Gateway-CE12808-1-Eth-Trunk31] quit 
    [*Exit-Gateway-CE12808-1] commit

    # Configure interconnection between Exit-Gateway-CE12808-2 and the firewalls.

    [~Exit-Gateway-CE12808-2] interface eth-trunk 21   
    [*Exit-Gateway-CE12808-2-Eth-Trunk21] description "to-FW-USG9560-1-GE1/0/4" 
    [*Exit-Gateway-CE12808-2-Eth-Trunk21] port link-type trunk 
    [*Exit-Gateway-CE12808-2-Eth-Trunk21] undo port trunk allow-pass vlan 1 
    [*Exit-Gateway-CE12808-2-Eth-Trunk21] port trunk allow-pass vlan 11 
    [*Exit-Gateway-CE12808-2-Eth-Trunk21] port trunk allow-pass vlan 12 
    [*Exit-Gateway-CE12808-2-Eth-Trunk21] trunkport 10ge 3/0/0 
    [*Exit-Gateway-CE12808-2-Eth-Trunk21] trunkport 10ge 3/0/1 
    [*Exit-Gateway-CE12808-2-Eth-Trunk21] dfs-group 1 m-lag 1 
    [*Exit-Gateway-CE12808-2-Eth-Trunk21] quit 
    [*Exit-Gateway-CE12808-2] interface eth-trunk 31 
    [*Exit-Gateway-CE12808-2-Eth-Trunk31] description "to-FW-USG9560-2-GE1/0/4" 
    [*Exit-Gateway-CE12808-2-Eth-Trunk31] port link-type trunk 
    [*Exit-Gateway-CE12808-2-Eth-Trunk31] undo port trunk allow-pass vlan 1 
    [*Exit-Gateway-CE12808-2-Eth-Trunk31] port trunk allow-pass vlan 11 
    [*Exit-Gateway-CE12808-2-Eth-Trunk31] port trunk allow-pass vlan 12 
    [*Exit-Gateway-CE12808-2-Eth-Trunk31] trunkport 10ge 3/0/2 
    [*Exit-Gateway-CE12808-2-Eth-Trunk31] trunkport 10ge 3/0/3 
    [*Exit-Gateway-CE12808-2-Eth-Trunk31] dfs-group 1 m-lag 2 
    [*Exit-Gateway-CE12808-2-Eth-Trunk31] quit 
    [*Exit-Gateway-CE12808-2] commit
    NOTE:

    If firewalls interconnect with gateways through M-LAG, the LAG modes at the two ends must be manual load balancing.

Configuring BGP EVPN

When BGP EVPN is configured as the VXLAN control plane, perform the following steps to configure the gateways as the RR clients and spine nodes as the RRs and establish IBGP EVPN peer relationships between the spine nodes and gateways.

  1. Configure EVPN as the VXLAN control plane on Exit-Gateway-CE12808-1 and Exit-Gateway-CE12808-2.

    [~Exit-Gateway-CE12808-1] evpn-overlay enable 
    [*Exit-Gateway-CE12808-1] commit 
     
    [~Exit-Gateway-CE12808-2] evpn-overlay enable 
    [*Exit-Gateway-CE12808-2] commit

  2. Establish BGP EVPN peer relationships between the egress gateways (Exit-Gateway-CE12808-1 and Exit-Gateway-CE12808-2) and spine nodes, and configure the spine nodes as RRs.

    # Establish BGP EVPN peer relationships between Exit-Gateway-CE12808-1 and spine nodes.

    [~Exit-Gateway-CE12808-1] bgp 100 instance evpn1 
    [*Exit-Gateway-CE12808-1-bgp-instance-evpn1] router-id 18.18.18.18 
    [*Exit-Gateway-CE12808-1-bgp-instance-evpn1] peer 11.11.11.14 as-number 100 
    //Establish a BGP EVPN peer relationship with Spine-CE12804-1. 
    [*Exit-Gateway-CE12808-1-bgp-instance-evpn1] peer 11.11.11.14 connect-interface loopback 1 
    [*Exit-Gateway-CE12808-1-bgp-instance-evpn1] peer 11.11.11.15 as-number 100 
    //Establish a BGP EVPN peer relationship with Spine-CE12804-2. 
    [*Exit-Gateway-CE12808-1-bgp-instance-evpn1] peer 11.11.11.15 connect-interface loopback 1 
    [*Exit-Gateway-CE12808-1-bgp-instance-evpn1] l2vpn-family evpn 
    [*Exit-Gateway-CE12808-1-bgp-instance-evpn1-af-evpn] peer 11.11.11.14 enable 
    [*Exit-Gateway-CE12808-1-bgp-instance-evpn1-af-evpn] peer 11.11.11.14 advertise irb 
    [*Exit-Gateway-CE12808-1-bgp-instance-evpn1-af-evpn] peer 11.11.11.15 enable 
    [*Exit-Gateway-CE12808-1-bgp-instance-evpn1-af-evpn] peer 11.11.11.15 advertise irb 
    [*Exit-Gateway-CE12808-1-bgp-instance-evpn1-af-evpn] quit 
    [*Exit-Gateway-CE12808-1-bgp-instance-evpn1] quit 
    [*Exit-Gateway-CE12808-1] commit

    # Establish BGP EVPN peer relationships between Exit-Gateway-CE12808-2 and spine nodes.

    [~Exit-Gateway-CE12808-2] bgp 100 instance evpn1 
    [*Exit-Gateway-CE12808-2-bgp-instance-evpn1] router-id 19.19.19.19 
    [*Exit-Gateway-CE12808-2-bgp-instance-evpn1] peer 11.11.11.14 as-number 100 
    //Establish a BGP EVPN peer relationship with Spine-CE12804-1. 
    [*Exit-Gateway-CE12808-2-bgp-instance-evpn1] peer 11.11.11.14 connect-interface loopback 1 
    [*Exit-Gateway-CE12808-2-bgp-instance-evpn1] peer 11.11.11.15 as-number 100 
    //Establish a BGP EVPN peer relationship with Spine-CE12804-2. 
    [*Exit-Gateway-CE12808-2-bgp-instance-evpn1] peer 11.11.11.15 connect-interface loopback 1 
    [*Exit-Gateway-CE12808-2-bgp-instance-evpn1] l2vpn-family evpn 
    [*Exit-Gateway-CE12808-2-bgp-instance-evpn1-af-evpn] peer 11.11.11.14 enable 
    [*Exit-Gateway-CE12808-2-bgp-instance-evpn1-af-evpn] peer 11.11.11.14 advertise irb 
    [*Exit-Gateway-CE12808-2-bgp-instance-evpn1-af-evpn] peer 11.11.11.15 enable 
    [*Exit-Gateway-CE12808-2-bgp-instance-evpn1-af-evpn] peer 11.11.11.15 advertise irb 
    [*Exit-Gateway-CE12808-2-bgp-instance-evpn1-af-evpn] quit 
    [*Exit-Gateway-CE12808-2-bgp-instance-evpn1] quit 
    [*Exit-Gateway-CE12808-2] commit

  3. Bind VPN instances to the gateways.

    [~Exit-Gateway-CE12808-1] ip vpn-instance vpn1       
    [*Exit-Gateway-CE12808-1-vpn-instance-vpn1] vxlan vni 10 
    [*Exit-Gateway-CE12808-1-vpn-instance-vpn1] route-distinguisher 11:1 
    [*Exit-Gateway-CE12808-1-vpn-instance-vpn1-af-ipv4] vpn-target 1:1 
    [*Exit-Gateway-CE12808-1-vpn-instance-vpn1-af-ipv4] vpn-target 11:1 evpn 
    [*Exit-Gateway-CE12808-1-vpn-instance-vpn1-af-ipv4] quit 
    [*Exit-Gateway-CE12808-1-vpn-instance-vpn1] quit 
    [*Exit-Gateway-CE12808-1] commit 
     
    [~Exit-Gateway-CE12808-2] ip vpn-instance vpn1     
    [*Exit-Gateway-CE12808-2-vpn-instance-vpn1] vxlan vni 10 
    [*Exit-Gateway-CE12808-2-vpn-instance-vpn1] route-distinguisher 11:2 
    [*Exit-Gateway-CE12808-2-vpn-instance-vpn1-af-ipv4] vpn-target 1:1 
    [*Exit-Gateway-CE12808-2-vpn-instance-vpn1-af-ipv4] vpn-target 11:1 evpn 
    [*Exit-Gateway-CE12808-2-vpn-instance-vpn1-af-ipv4] quit 
    [*Exit-Gateway-CE12808-2-vpn-instance-vpn1] quit 
    [*Exit-Gateway-CE12808-2] commit  
     
    [~Exit-Gateway-CE12808-1] interface vlanif 12 //Configure a VLAN for service interconnection between the VRF of the gateway and firewall.
    [~Exit-Gateway-CE12808-1-Vlanif12] description "to firewall-1~2Vsys" 
    [*Exit-Gateway-CE12808-1-Vlanif12] ip binding vpn-instance vpn1 
    [*Exit-Gateway-CE12808-1-Vlanif12] commit 
    [~Exit-Gateway-CE12808-1-Vlanif12] quit 
    [~Exit-Gateway-CE12808-1] ip route-static vpn-instance vpn1 0.0.0.0 0.0.0.0 11.254.45.164  //Configure a default route for the VRF vpn1 of the gateway. The route is destined for the firewall.
    [*Exit-Gateway-CE12808-1] commit 
     
    [~Exit-Gateway-CE12808-2] interface vlanif 12 //Configure a VLAN for service interconnection between the VRF of the gateway and firewall.
    [~Exit-Gateway-CE12808-2-Vlanif12] description "to firewall-1~2Vsys" 
    [*Exit-Gateway-CE12808-2-Vlanif12] ip binding vpn-instance vpn1 
    [*Exit-Gateway-CE12808-2-Vlanif12] commit 
    [~Exit-Gateway-CE12808-2-Vlanif12] quit 
    [~Exit-Gateway-CE12808-2] ip route-static vpn-instance vpn1 0.0.0.0 0.0.0.0 11.254.45.164 //Configure a default route for the VRF vpn1 of the gateway. The route is destined for the firewall.
    [*Exit-Gateway-CE12808-2] commit

  4. Associates a specified VPN instance with an IPv4 address family.

    [~Exit-Gateway-CE12808-1] bgp 100 instance evpn1  
    [*Exit-Gateway-CE12808-1-bgp-instance-evpn1] ipv4-family vpn-instance vpn1  
    [*Exit-Gateway-CE12808-1-bgp-instance-evpn1-vpn1] default-route imported  
    [*Exit-Gateway-CE12808-1-bgp-instance-evpn1-vpn1] import-route direct  
    [*Exit-Gateway-CE12808-1-bgp-instance-evpn1-vpn1] import-route static  
    [*Exit-Gateway-CE12808-1-bgp-instance-evpn1-vpn1] maximum load-balancing 32  //Set the maximum number of equal cost routes to 32.
    [*Exit-Gateway-CE12808-1-bgp-instance-evpn1-vpn1] advertise l2vpn evpn       //Enable the VPN instance to advertise IP routes to the EVPN instance.
    [*Exit-Gateway-CE12808-1-bgp-instance-evpn1-vpn1] quit 
    [*Exit-Gateway-CE12808-1-bgp-instance-evpn1] quit 
    [*Exit-Gateway-CE12808-1] commit  
     
    [~Exit-Gateway-CE12808-2] bgp 100 instance evpn1  
    [*Exit-Gateway-CE12808-2-bgp-instance-evpn1] ipv4-family vpn-instance vpn1  
    [*Exit-Gateway-CE12808-2-bgp-instance-evpn1-vpn1] default-route imported  
    [*Exit-Gateway-CE12808-2-bgp-instance-evpn1-vpn1] import-route direct  
    [*Exit-Gateway-CE12808-2-bgp-instance-evpn1-vpn1] import-route static  
    [*Exit-Gateway-CE12808-2-bgp-instance-evpn1-vpn1] maximum load-balancing 32  //Set the maximum number of equal cost routes to 32.
    [*Exit-Gateway-CE12808-2-bgp-instance-evpn1-vpn1] advertise l2vpn evpn       //Enable the VPN instance to advertise IP routes to the EVPN instance.
    [*Exit-Gateway-CE12808-2-bgp-instance-evpn1-vpn1] quit 
    [*Exit-Gateway-CE12808-2-bgp-instance-evpn1] quit 
    [*Exit-Gateway-CE12808-2] commit

Configuring a MAC Address Flapping Detection Whitelist

In the scenario where only one role can be configured for one device, when VXLAN traffic reaches a gateway through a spine device, a MAC address flapping alarm will be generated if the gateway has learned an incorrect MAC address due to product constraints. To prevent this problem, configure a MAC address flapping detection whitelist on gateways and add the MAC addresses of the Layer 3 interfaces on the spine devices that connect to the gateways to the whitelist, so that MAC address flapping detection will not be performed for these addresses. In normal cases, these MAC addresses are the outer source MAC addresses of tunnel packets and cannot be learned. Therefore, the adjustment does not affect services.

  1. Obtain the MAC addresses of Layer 3 interfaces on the two spine devices that connect to the gateways.

    [~Spine-CE12804-1] interface 40ge 1/0/4  //Obtain the MAC addresses of Layer 3 interfaces on Spine-CE12804-1 that connect to the gateways.  
    [*Spine-CE12804-1-40GE1/0/4] undo portswitch 
    [*Spine-CE12804-1-40GE1/0/4] commit 
    [~Spine-CE12804-1-40GE1/0/4] display this interface | include  Hardware address 
    IP Sending Frames' Format is PKTFMT_ETHNT_2, Hardware address is 200b-c732-d202 
    [~Spine-CE12804-1-40GE1/0/4] quit 
    [~Spine-CE12804-1] interface 40ge 1/0/5 
    [~Spine-CE12804-1-40GE1/0/5] undo portswitch 
    [*Spine-CE12804-1-40GE1/0/5] commit 
    [~Spine-CE12804-1-40GE1/0/5] display this interface | include  Hardware address 
    IP Sending Frames' Format is PKTFMT_ETHNT_2, Hardware address is 200b-c732-d202 
    [~Spine-CE12804-1-40GE1/0/5] quit 
     
    [~Spine-CE12804-2] interface 40ge 1/0/4 //Obtain the MAC addresses of Layer 3 interfaces on Spine-CE12804-2 that connect to the gateways.  
    [~Spine-CE12804-2-40GE1/0/4] undo portswitch 
    [*Spine-CE12804-2-40GE1/0/4] commit 
    [~Spine-CE12804-2-40GE1/0/4] display this interface | include  Hardware address 
    IP Sending Frames' Format is PKTFMT_ETHNT_2, Hardware address is 346a-c246-be01 
    [~Spine-CE12804-2-40GE1/0/4] quit 
    [~Spine-CE12804-2] interface 40ge 1/0/5 
    [~Spine-CE12804-2-40GE1/0/5] undo portswitch 
    [*Spine-CE12804-2-40GE1/0/5] commit 
    [~Spine-CE12804-2-40GE1/0/5] display this interface | include  Hardware address 
    IP Sending Frames' Format is PKTFMT_ETHNT_2, Hardware address is 346a-c246-be01 
    [~Spine-CE12804-2-40GE1/0/5] quit

  2. Add the obtained MAC addresses to a whitelist.

    [~Exit-Gateway-CE12808-1] mac-address flapping detection exclude 200b-c732-d202 48 
    [~Exit-Gateway-CE12808-1] mac-address flapping detection exclude 346a-c246-be01 48 
     
    [~Exit-Gateway-CE12808-2] mac-address flapping detection exclude 200b-c732-d202 48 
    [~Exit-Gateway-CE12808-2] mac-address flapping detection exclude 346a-c246-be01 48

Configuring Interconnection Between the Layer 3 Egress Gateways and External Routers
  1. Configure interfaces on the gateways to connect to external routers.

    # Configure an interface on Exit-Gateway-CE12808-1 to connect to Router.

    [~Exit-Gateway-CE12808-1] interface 10ge 3/0/4  //Configure an interface to connect to the external PE device Router-1.  
    [~Exit-Gateway-CE12808-1-10GE3/0/4] undo portswitch 
    [*Exit-Gateway-CE12808-1-10GE3/0/4] ip address 11.254.44.157 30 
    [*Exit-Gateway-CE12808-1-10GE3/0/4] set up-delay 180   //Configure a delay for the interface to report status change events.
    [*Exit-Gateway-CE12808-1-10GE3/0/4] quit 
    [*Exit-Gateway-CE12808-1] interface 10ge 3/0/5 //Configure an interface to connect to the external PE device Router-2. 
    [*Exit-Gateway-CE12808-1-10GE3/0/5] undo portswitch 
    [*Exit-Gateway-CE12808-1-10GE3/0/5] ip address 11.254.44.169 30 
    [*Exit-Gateway-CE12808-1-10GE3/0/5] set up-delay 180   //Configure a delay for the interface to report status change events.
    [*Exit-Gateway-CE12808-1-10GE3/0/5] quit 
    [*Exit-Gateway-CE12808-1] commit

    # Configure an interface on Exit-Gateway-CE12808-2 to connect to Router.

    [~Exit-Gateway-CE12808-2] interface 10ge 3/0/4  //Configure an interface to connect to the external PE device Router-1.  
    [~Exit-Gateway-CE12808-2-10GE3/0/4] undo portswitch 
    [*Exit-Gateway-CE12808-2-10GE3/0/4] ip address 11.254.44.173 30 
    [*Exit-Gateway-CE12808-2-10GE3/0/4] set up-delay 180  
    [*Exit-Gateway-CE12808-2-10GE3/0/4] quit 
    [*Exit-Gateway-CE12808-2] interface 10ge 3/0/5  //Configure an interface to connect to the external PE device Router-2. 
    [*Exit-Gateway-CE12808-2-10GE3/0/5] undo portswitch 
    [*Exit-Gateway-CE12808-2-10GE3/0/5] ip address 11.254.44.161 30 
    [*Exit-Gateway-CE12808-2-10GE3/0/5] set up-delay 180 
    [*Exit-Gateway-CE12808-2-10GE3/0/5] quit 
    [*Exit-Gateway-CE12808-2] commit

  2. Configure Layer 3 interconnection interfaces between the gateways to connect them in mesh mode.

    # Configure an interface on Exit-Gateway-CE12808-1 to connect to Exit-Gateway-CE12808-2.

    [~Exit-Gateway-CE12808-1] interface 10ge 3/0/6  //Configure an interface to connect to Exit-Gateway-CE12808-2. 
    [~Exit-Gateway-CE12808-1-10GE3/0/6] undo portswitch 
    [*Exit-Gateway-CE12808-1-10GE3/0/6] ip address 11.254.44.165 30 
    [*Exit-Gateway-CE12808-1-10GE3/0/6] commit 
    [~Exit-Gateway-CE12808-1-10GE3/0/6] quit

    # Configure an interface on Exit-Gateway-CE12808-2 to connect to Exit-Gateway-CE12808-1.

    [~Exit-Gateway-CE12808-2] interface 10ge 3/0/6  //Configure an interface to connect to Exit-Gateway-CE12808-1. 
    [~Exit-Gateway-CE12808-2-10GE3/0/6] undo portswitch 
    [*Exit-Gateway-CE12808-2-10GE3/0/6] ip address 11.254.44.166 30 
    [*Exit-Gateway-CE12808-2-10GE3/0/6] commit 
    [~Exit-Gateway-CE12808-2-10GE3/0/6] quit

  3. Configure default routes destined for the external routers on the gateways.

    # Configure default routes on Gateway-CE12808-1.

    [~Exit-Gateway-CE12808-1] ip route-static 0.0.0.0 0.0.0.0 11.254.44.158  
    [*Exit-Gateway-CE12808-1] ip route-static 0.0.0.0 0.0.0.0 11.254.44.170 
    [*Exit-Gateway-CE12808-1] ip route-static 0.0.0.0 0.0.0.0 11.254.44.166 preference 80  //Configure a route destined for Exit-Gateway-CE12808-2, as a backup.
    [*Exit-Gateway-CE12808-1] commit 
    [~Exit-Gateway-CE12808-1] quit

    # Configure default routes on Gateway-CE12808-2.

    [~Exit-Gateway-CE12808-2] ip route-static 0.0.0.0 0.0.0.0 11.254.44.174  
    [*Exit-Gateway-CE12808-2] ip route-static 0.0.0.0 0.0.0.0 11.254.44.162 
    [*Exit-Gateway-CE12808-2] ip route-static 0.0.0.0 0.0.0.0 11.254.44.165 preference 80  //Configure a route destined for Exit-Gateway-CE12808-1, as a backup.
    [*Exit-Gateway-CE12808-2] commit 
    [~Exit-Gateway-CE12808-2] quit

Configuring Enhanced Functions

The commands used to enable enhanced functions on the CE12800 series switches depend on the cards installed on them. The following table lists the commands by the card type. (You do not need to configure these commands on CE12800E series switches.)

Table 3-2 Commands for enabling enhanced functions on CE12800 series switches

Card

Configuration Command

E series cards

  1. assign forward nvo3 service extend enable
  2. assign forward nvo3 acl extend enable (CE12800 series switches need to be restarted for the command to take effect.)
  3. assign forward nvo3-gateway enhanced L3

FD or FDA series cards

(Only SFUFs and SFUGs can be inserted).

  1. assign forward nvo3 acl extend enable (CE12800 series switches need to be restarted for the command to take effect.)
  2. set serdes capability enhanced (CE12800 series switches need to be restarted for the command to take effect.)
  3. set forward capability enhanced (CE12800 series switches need to be restarted for the command to take effect.)

Intermixing of E and F series cards (not recommended)

(Only SFUFs and SFUGs can be inserted).

  1. assign forward nvo3 service extend enable
  2. assign forward nvo3 acl extend enable (CE12800 series switches need to be restarted for the command to take effect.)
  3. assign forward nvo3-gateway enhanced L3
  4. assign forward nvo3 f-linecard compatibility enable
  1. Configure the NVO3 service extension function on a gateway.

    When a CE12800 switch functions as the gateway, the NVO3 service extension function is disabled by default. After the NVO3 service is deployed on the switch, there is a high possibility that other ACL resource-consuming services such as MQC, simplified ACL, traffic policing, BD traffic statistics collection, and DHCP fail to be configured on the switch.

    On an NVO3-enabled switch, use the following commands to reduce the possibility of service deployment failures:

    [~Exit-Gateway-CE12808-1] assign forward nvo3 service extend enable 
    [*Exit-Gateway-CE12808-1] assign forward nvo3 acl extend enable 
    [*Exit-Gateway-CE12808-1] commit 
    [~Exit-Gateway-CE12808-1] quit 
    <~Exit-Gateway-CE12808-1> save 
    <~Exit-Gateway-CE12808-1> reboot 
     
    [~Exit-Gateway-CE12808-2] assign forward nvo3 service extend enable 
    [*Exit-Gateway-CE12808-2] assign forward nvo3 acl extend enable 
    [*Exit-Gateway-CE12808-2] commit 
    [~Exit-Gateway-CE12808-2] quit 
    <~Exit-Gateway-CE12808-2> save 
    <~Exit-Gateway-CE12808-2> reboot

    Pay attention to the following points when using these two commands:

    • Run the assign forward nvo3 service extend enable command in the system view on Exit-Gateway to enable the NVO3 service extension function. After you run this command, the possibility of service deployment failure is lowered on switches that do not have CE-L48GT-EA, CE-L48GT-EC, CE-L48GS-EA, CE-L48GS-EC, CE-L24XS-BA, CE-L24XS-EA, CE-L48XS-BA, CE-L48XS-EA, or CE-L24LQ-EA cards installed.
    NOTE:

    This command does not take effect on CE-L48GT-EA, CE-L48GT-EC, CE-L48GS-EA, CE-L48GS-EC, CE-L24XS-BA, CE-L24XS-EA, CE-L48XS-BA, CE-L48XS-EA, and CE-L24LQ-EA cards.

    After this command is run, packets with length ranging from 230 to 294 bytes transmitted over the NVO3 tunnels by other cards except the preceding ones cannot be sent to the preceding cards.

    After you run this command, you can use VXLAN path detection only on cards except the following: CE-L48GT-EA, CE-L48GT-EC, CE-L48GS-EA, CE-L48GS-EC, CE-L24XS-BA, CE-L24XS-EA, CE-L48XS-BA, CE-L48XS-EA, and CE-L24LQ-EA.

    • Run the assign forward nvo3 acl extend enable command in the system view on Exit-Gateway to enable the NVO3 ACL extension function.
    NOTE:

    You can run this command in the Admin-VS only, and the configuration takes effect for all VSs.

    After you run this command to enable the NVO3 ACL extension function, restart the switch to make the configuration take effect.

  2. Configure the enhanced mode of the NVO3 gateway.

    If the enhanced mode of the NVO3 gateway is not configured, a CE12800 switch works in loopback mode by default. That is, the switch loops back NVO3-encapsulated packets before forwarding them. There is a high possibility that the gateway drops VXLAN-encapsulated or decapsulated Layer 3 packets if the traffic exceeds 50% of the line cards' total forwarding performance. You can configure the enhanced mode of the NVO3 gateway to solve this problem in actual situations.

    Run the assign forward nvo3-gateway enhanced l3 command in the system view on Exit-Gateway to configure the enhanced mode of the NVO3 gateway.

    [~Exit-Gateway-CE12808-1] assign forward nvo3-gateway enhanced l3 
     
    [~Exit-Gateway-CE12808-2] assign forward nvo3-gateway enhanced l3
    NOTE:

    Run the assign forward nvo3 service extend enable command to enable the NVO3 service extension function first. Ensure that the switch does not contain the CE-L48GT-EA, CE-L48GT-EC, CE-L48GS-EA, CE-L48GS-EC, CE-L24XS-BA, CE-L24XS-EA, CE-L48XS-BA, CE-L48XS-EA, or CE-L24LQ-EA card, or VXLAN-related services are not configured on the card.

    If the CE-L24XS-EC, CE-L48XS-EC, CE-L24LQ-EC, CE-L48XT-EC, CE-L24LQ-EC1, CE-L08CC-EC, CE-L02LQ-EC, or CE-L06LQ-EC card transmits VXLAN-related services, it can only query the ARP host table for VXLAN tunnel encapsulation, and cannot query the longest match routing table for VXLAN tunnel encapsulation.

    The two access switches to which the server NICs in active/standby mode connect need to set up a stack, or the server is dual-homed to two access switches through M-LAG. In this case, interfaces on the access switches connecting to the server cannot be M-LAG member interfaces.

  3. Configure the enhanced mode for CE12800 series switches. (CE12800E series switches do not support commands in this step.)

    1. Set the card interoperability mode to enhanced mode.
      [~Exit-Gateway-CE12808-1] set forward capability enhanced  
      Warning: Current configuration should be committed and saved, and it will take effect after reboot. [Y/N]: y
      NOTE:

      When the card interoperability mode is enhanced mode, only FD/FDA LPUs as well as SFUFs and SFUGs can be installed on the switch.

      When the card interoperability mode is enhanced mode, EA, EC, ED, EF, EG, BA, CE-FWA, and CE-IPSA LPUs as well as SFUAs, SFUBs, and SFUCs cannot be installed on the switch. Otherwise, cards will be powered off. If the switch uses SFUAs, SFUBs, or SFUCs, FD or FDA LPUs cannot be inserted. If the switch uses FD or FDA LPUs, SFUAs, SFUBs, or SFUCs cannot be inserted.

      After you run this command, save the configuration and restart the device to make the configuration take effect. (The saved configuration file is used as the next startup file.) After the device restarts, run the save command again to save the configuration.

      After running this command, you are advised to run the set serdes capability enhanced command to set the Serdes rate mode between LPUs and SFUs to enhanced mode. This configuration can ensure the maximum forwarding performance.

      This command is mutually exclusive with the following commands. Before running this command, delete the following commands:

      • assign forward nvo3-gateway enhanced { l2 | l3 }
      • assign forward nvo3 eth-trunk hash disable

      If the EA, EC, ED, EF, EG, BA, CE-FWA, or CE-IPSA LPUs are used together with the FD or FDA LPUs, set the card interoperability mode to the enhanced mode. To prevent VXLAN traffic interruption in this case, run the assign forward nvo3 f-linecard compatibility enable command.

    2. Set the Serdes rate mode to enhanced mode.
      [~Exit-Gateway-CE12808-1] set serdes capability enhanced  
      Warning: Current configuration should be committed and saved, and it will take effect after reboot. [Y/N]: y
    NOTE:

    The command configuration takes effect after the switch is restarted.

    If the switch uses SFUAs, SFUBs, or SFUCs, FD or FDA LPUs cannot be inserted. If the switch uses FD or FDA LPUs, SFUAs, SFUBs, or SFUCs cannot be inserted.

  4. Configure ARP broadcast suppression.

    In the previous sections, you have run the peer { ipv4-address | group-name } advertise irb and arp collect host enable commands in the gateway group and leaf node group to configure the BGP EVPN route advertisement and host information collection functions. To provide ARP broadcast suppression, you also need to perform the following steps in the gateway group and leaf node group:

    In all leaf node groups, enable ARP broadcast suppression. (The following lists commands for a stack of leaf nodes as an example.)

    [~Leaf-CE6851HI-1&CE6851HI-2] bridge-domain 10 
    [~Leaf-CE6851HI-1&CE6851HI-2-bd10] arp broadcast-suppress enable 
    [*Leaf-CE6851HI-1&CE6851HI-2-bd10] commit 
    [~Leaf-CE6851HI-1&CE6851HI-2-bd10] quit

  5. In all leaf node groups, enable BUM traffic suppression. (The following lists commands for a stack of leaf nodes as an example.)

    [~Leaf-CE6851HI-1&CE6851HI-2] interface 10ge 1/0/1 
    [~Leaf-CE6851HI-1&CE6851HI-2-10GE1/0/1] storm suppression unknown-unicast 1 
    [*Leaf-CE6851HI-1&CE6851HI-2-10GE1/0/1] storm suppression multicast packets 200 
    [*Leaf-CE6851HI-1&CE6851HI-2-10GE1/0/1] storm suppression broadcast packets 1000 
    [*Leaf-CE6851HI-1&CE6851HI-2-10GE1/0/1] commit 
    [~Leaf-CE6851HI-1&CE6851HI-2-10GE1/0/1] quit 
     
    [~Leaf-CE6851HI-1&CE6851HI-2-10GE1/0/2] interface 10ge 1/0/2 
    [~Leaf-CE6851HI-1&CE6851HI-2-10GE1/0/2] storm suppression unknown-unicast 1  
    [*Leaf-CE6851HI-1&CE6851HI-2-10GE1/0/2] storm suppression multicast packets 200 
    [*Leaf-CE6851HI-1&CE6851HI-2-10GE1/0/2] storm suppression broadcast packets 1000 
    [*Leaf-CE6851HI-1&CE6851HI-2-10GE1/0/2] commit 
    [~Leaf-CE6851HI-1&CE6851HI-2-10GE1/0/2] quit

  6. In all leaf node groups, set the system resource mode to large routing mode. (The following lists commands for a stack of leaf nodes as an example.)

    [~Leaf-CE6851HI-1&CE6851HI-2] system resource large-route

Translation
Download
Updated: 2018-07-02

Document ID: EDOC1100004176

Views: 17966

Downloads: 554

Average rating:
This Document Applies to these Products
Related Documents
Related Version
Share
Previous Next