Data Center Network
ata center networks are the infrastructure of data center services,
requiring computer systems, storage systems, communications systems, communication connections, and management and monitoring systems. Enterprises or organizations can store, manage, and advertise information in data centers.
Multiple data center networks can connect branches of enterprises or organizations in different areas. In addition, data center networks connect to the Internet, so that enterprises or organizations can access the Internet.
In a data center, data exchange is centralized and east-west traffic increases, posing the following requirements on the data center network: large scale, high scalability, enhanced robustness, low configuration costs, high bandwidths between servers, efficient network protocols, flexible topology and link capacity control, energy conservation, traffic isolation between services, and low costs. The traditional three-layer architecture cannot meet the requirements. A flattened, virtualized, programmable, and definable network architecture becomes a new trend of data center network development.
To meet the preceding requirements, data center networks begin to apply the VXLAN and SDN technologies. The technologies allow cloud platforms to implement adaptation and association between networks and services, improving resource usage and service provisioning efficiency.
Centralized and Distributed Networking
On a VXLAN network, Layer 3 gateways are integrated on one or more groups of switches. The virtual tunnel end points (VTEPs) of leaf switches connecting to firewalls, load balancers (LBs), and servers only function as Layer 2 gateways on the VXLAN network.
- Distributed network overlay VXLAN: All physical switches deployed as leaf nodes can function as L3 gateways. Spine nodes only forward traffic, but not function as VTEPs.
- Distributed hybrid overlay VXLAN: Some physical switches deployed as leaf nodes and all CE1800V switches can function as L3 gateways. Spine nodes only forward traffic, but not function as VTEPs.
Figure1 Centralized VXLAN networking shows a centralized VXLAN network. Leaf1, Leaf2, and the spine node act as VTEPs. Leaf1 and Leaf2 each sets up a VXLAN tunnel with the spine node. The spine node acts as the VXLAN Layer 3 gateway to allow communication between VMs in different departments.
Figure2 Networking of a distributed network overlay VXLAN shows the networking of a distributed network overlay VXLAN. Leaf1 and Leaf2 act as VTEPs and the VXLAN Layer 3 gateways, and set up a VXLAN tunnel with each other. When VM1 and VM2 communicate, traffic is forwarded by Leaf1. When VM1 and VM4 need to communicate, traffic passes through the leaf nodes and VXLAN tunnel. The spine node only forwards traffic.
Figure1 Centralized VXLAN networking
Figure2 Networking of a distributed network overlay VXLAN
Network Overlay/Host Overlay/Hybrid Overlay
Figure1 shows in the network overlay networking, all VXLAN overlay tunnel end points are deployed on hardware switches.
Hardware overlay networking can be centralized or distributed.
- Centralized network overlay: VXLAN Layer 3 gateways are deployed in centralized mode, and the leaf nodes act as VXLAN Layer 2 bridges.
- Distributed network overlay: The leaf nodes function as VXLAN Layer 3 gateways, and the spine nodes only forward IP packets.
The left of figure2 shows the networking of centralized network overlay, in which the spine nodes also function as VXLAN Layer 3 gateways. Figure 2 shows the distributed network overlay, and the L3 VXLAN gateway is configured on the leaf node.
Figure1 VXLAN overlay tunnel end points deployed on hardware switches
Figure2 Networking of centralized network overlay and distributed network overlay
In the host overly networking, all VXLAN overlay tunnel end points are deployed on software switches (installed on servers). That is, both the ingress and egress of a VXLAN tunnel are software switches.
In the hybrid overlay networking, VXLAN overlay tunnel end points are deployed on both hardware and software switches. That is, the ingress and egress of a VXLAN tunnel can be hardware or software.
Figure3 Host overlay networking
Figure4 Hybrid overlay networking
Network Virtualization/Cloud-Network Integration
Network virtualization and cloud-network integration are two scenarios where Huawei CloudFabric DCN Solution is applied.
In the network virtualization scenario, no cloud platform is used. The cloud-network integration scenario requires cloud platforms.
Cloud-network integration requires the AC-DCN, network, VMM, and cloud platform. The AC-DCN and VMM connect to the cloud platform. The network administrator delivers and manages services through the unified computing and network management GUI provided by the cloud platform.
Network virtualization includes the hosting and computing scenarios.
- Network virtualization – hosting: provides the AC-DCN and network, but no cloud platform. The AC-DCN is not associated with a VMM. A network administrator can centrally manage networks through the independent management GUI provided by the AC-DCN.
- Network virtualization – computing: provides the AC-DCN, network, and VMM, but no cloud platform. The network administrator manages the physical and virtual networks through the independent management GUI provided by the AC-DCN. The network system is associated with the computing system. The computing administrator manages the computing system.
Overlay and Underlay
In an overlay network model, a software-defined logical network is added to a physical network without changing the physical network. Service logic of the logical network is defined by software to solve problems of data center networks. The overlay technology transmits Layer 2 packets (of services) to Layer 3 or Layer 4 (of a traditional network).
The overlay technology includes VXLAN, Network Virtualization using Generic Routing Encapsulation (NVGRE), and Geneve. The principle of the overlay technology is as follows: Overlay encapsulates a Layer 2 packet in a tunnel, transmits the encapsulated packet transparently, and then decapsulates the packet to obtain the raw packet after the packet arrives its destination. In this way, the current network is overlaid with a large Layer 2 network.
The underlay network is a bearer network consisting of physical devices, such as TOR switches, aggregation switches, core switches, LBs, and firewalls.
The overlay technology can be implemented to build a logical network over an underlay network.
- An overlay network is a virtual network established over an existing underlay network, consisting of logical nodes and logical links.
- An overlay network has independent control and forwarding planes. The physical network is transparent for the terminals connecting to the overlay network virtualization edge (NVE).
In Huawei CloudFabric DCN Solution, an overlay network is established using the VXLAN technology. Service packets are transmitted on the VXLAN overlay network and decoupled from the physical bearer network.VXLAN-based overlays are classified into network overlay, host overlay, and hybrid overlay according to the deployment mode of VXLAN NVEs.
- Hardware overlay: All NVEs are deployed on physical switches.
- Software overlay: All NVEs are deployed on vSwitches.
- Hybrid overlay: Some NVEs are deployed on physical switches and others are deployed on vSwitches.
Leaf and Spine Nodes
A leaf node is an access node on a VXLAN fabric network, which connects various network devices to the VXLAN network.A spine node is a core node on a VXLAN fabric network, which provides high-speed IP forwarding and connects to leaf nodes using high-speed interfaces.In Huawei CloudFabric DCN Solution, the underlay network uses the spine-leaf structure.
Leaf and spine nodes are fully meshed so that equal-cost multiple-path routing (ECMP) paths are available to ensure high availability of the network.
- Generally, spine nodes are two or more large-capacity switches, and connect to leaf nodes at Layer 3 to establish an IP fabric network.
- Access switches can be deployed at leaf nodes and fully meshed to the spine nodes. Leaf nodes are the boundary between Layer 2 and Layer 3 on the underlay network, and connect to the spine nodes at Layer 3. If many TOR switches are deployed at the leaf nodes, it is recommended that the TOR switches be configured to set up an iStack, a multi-chassis link aggregation group (M-LAG), or a super virtual fabric (SVF) system for server access.
Leaf nodes are access nodes in the physical networking. They connect servers to the VXLAN network. Leaf nodes can be classified into the following types according to the access objects.
- Border leaf: connects to routers or transmission devices to forward traffic from external networks to the VXLAN fabric network of a data center.
- Server leaf: connects virtual and physical servers to the VXLAN fabric network.
- Service leaf: connects Layer 4 to Layer 7 value-added services (VASs), such as firewalls and LBs, to the VXLAN fabric network.
Racking leasing is the main service of an Internet Data Center (IDC) carrier. A tenant leases resources, including equipment room spaces, racks, ingress and egress bandwidths, security protection, and load balancing, from a carrier, so that the tenant can connect its services to networks and provide services for external users without building its own data equipment room.
Rack leasing can be classified into the following types according to the property ownership.
- Equipment room space leasing: A tenant provides servers and the carrier provides TOR devices and an interconnection network.
- Cabinet leasing: A tenant leases cabinets and provides servers and TOR devices. The carrier provides an interconnection network.
In Huawei CloudFabric DCN Solution, rack leasing refers to the traditional rack leasing service provided by SDN in the network virtualization scenario.
Rack leasing includes the following application scenarios:
- A tenant leases racks from the carrier and provides gateways.
- A tenant does not provide gateways. The carrier provides gateways and Layer 3 interfaces.
- A tenant does not provide gateways. The carrier leases the virtual private cloud (VPC) service.
Inline and Bypass Firewall Connection Modes
Inline connection mode (or serial connection mode): Firewalls are deployed between the border leaf nodes and PE device.
Bypass connection mode: Firewalls are connected to gateways in bypass mode.
Service Function Chain
Service function chain (SFC) technology enables service flows to be processed by functional points in a required sequence.
The following figure shows forwarding mode of a service chain
Table1 Basic SFC concepts
||SFs check and process data packets as required. For example, they can filter packets and implement network address translation (NAT). In a data center network, SFs are called network VASs and are implemented by modules on a device or virtual instances.In an SFC, devices such as firewalls and LBs act as SF nodes.
||SCs identify service flows, set service IDs, and encapsulate service packet headers.
|Service function forwarder
||SFFs connect the SF nodes, identify service flow information, and forward service packets based on the information.In an SFC, devices such as TOR switches, gateways, and vSwitches act as SFFs.
Data Center Interconnect
Data Center Interconnect (DCI) refers to the network interconnection between two data centers for cross-DC service transmission and migration.
Layer 2 communication across multiple data centers can be implemented using Border Gateway Protocol-Ethernet Virtual Private Network (BGP-EVPN) and VXLAN technology based on overlay networks.
Packet Conservation Algorithm for Internet (iPCA) is developed by Huawei. The principle of iPCA is as follows: For packets passing through a measured system, ifNumber of packets arriving at the system + Number of packets generated by the system = Number of packets leaving the system + Number of packets absorbed by the system .The system is in the normal state. If the condition is not met, some packets have been dropped in the system.
In Huawei CloudFabric DCN Solution, iPCA is used to measure service flows to detect the network quality.
- Detect packet loss and delay of service flows.
- Calculate packet loss ratio and delay of specified service flows (for example, common service flows defined using 5-tuple ACL rules, and tenant service flows defined using 5-tuple ACL rules and VNIs or VLANs).
- Display detection and calculation results.
When the access to a service becomes slow, times out, or times out intermittently, you can use the iPCA to detect the quality of the service flow. The iPCA can accurately locate the packet loss point of the service flow on the network, link, and device, and measure the delay, accelerating service fault location.
Multiple Groups of Gateways
In the centralized gateway networking, gateway resources such as virtual routing and forwarding (VRF), subnets, and access control lists (ACLs) are limited.
When a data center network expands, independent gateways are added. In this case, two or more groups of gateways are deployed to multiply the gateway resources.
The new gateway groups run independently and do not affect the original gateway group. VXLAN network identifiers (VNIs) on the same TOR switch may belong to different gateway groups. The capacity and reliability of a single gateway group remain unchanged.
Gateways in a gateway group can work in stacking, active and standby, or all-active (dual-active or quad-active) mode.
The VRFs and subnets of a tenant belong to the same gateway group.
Distributed L3 Gateway and Egress L3 Gateway
A VXLAN distributed L3 gateway (east-west gateway) connects to servers or VMs and provides addressing for communication between VMs in a data center.
A VXLAN egress L3 gateway (north-south gateway) connects to external edge devices (connecting to the Internet or external private networks) and provides addressing for the data center network to access the Internet or an external private network.
Th following shows the networking of distributed L3 gateways and egress L3 gateway in the distributed network overlay.
Dual-Activ Egresses/Active and Standby Egresses/Integrated Egresses
Network egresses are deployed to connect a data center to external networks (Internet or intranet).
Dual-active egresses: Two or more data centers communicate with external networks through two different links. The two links can forward the traffic sent from data centers to external networks simultaneously.
Active and standby egresses: Two or more data centers communicate with external networks through two links. One is the active link and the other is the standby link. In normal conditions, the active link forwards the traffic sent from data centers to external networks. When the active link fails, the traffic is switched to the standby link.
Integrated egress: Two or more data centers have only one egress for communication with external networks. All outgoing traffic of the data centers is forwarded through this egress to external networks.
Stacking, Active and Standby, and All-Active Gateways
In Huawei CloudFabric DCN Solution, gateways can be deployed in stacking, active and standby, and all-active modes.
When deployed in stacking mode, two gateway devices set up a CSS or an iStack, forming a unified control plane to act as a VXLAN Layer 3 gateway.
When deployed in active and standby mode, two independent gateway devices set up a Virtual Router Redundancy Protocol (VRRP) group and are configured with different tunnel end point addresses. The active and standby gateways use the same VRRP virtual IP address as the VXLAN Layer 3 gateway address and act as one logical gateway for servers.
When deployed in all-active mode, two or four independent gateways set up a DFS group. They have the same gateway address and tunnel end point address, and act as one logical gateway for servers.
Table1 Comparison of gateways deployed in three modes
||Configuration on Complexity
||The gateways function as a single device. The configuration is simple and the O&M GUI is clear.Traffic on the gateways can be load balanced between the stack members.
|Active and standby
||Two gateways have independent control planes and back up each other.
||Two gateways have independent control planes and back up each other.Traffic on the gateways can be load balanced between the DFS group members.
vDC Active/Standby DR
To implement intra-city vDC and unified resource pool management and active/standby deployment for services, the CloudFabric solution supports vDC active/standby DR for unified management and network-layer DR of multiple DCs. The following figure shows the logical diagram of vDC active/standby DR.
Unified management of multiple DCs
A single Agile Controller-DCN cluster is used to manage multiple fabric resources, forming a unified large Layer 2 network among multiple fabrics. In this manner, the distributed data center network resources are centrally managed, improving service provisioning and O&M efficiency.
DR of multi-DC networks
VPC services are deployed across fabrics, and active and standby gateways and egresses are provided, improving service reliability. The Agile Controller-DCN cluster can be deployed in active/standby mode to improve the reliability of the management plane.
The CloudFabric solution supports multi-DC interconnection to meet DR and active-active service requirements among remote DCs. The following figure shows the logical diagram of multi-DC interconnection. Multi-DC interconnection can help construct a DC service architecture featuring intra-city active-active DC deployment and geographical redundancy (for example, two-place three-DC). In addition, flexible on-demand interworking is available for cross-DC services.
Multiple independent vDCs or DCs are connected on the overlay network through the three-segment VXLAN to enable Layer 2 and Layer 3 communication among vDCs or DCs.
The same service system is deployed in two independent DCs to implement system interconnection at Layer 2 and Layer 3 and provide inter-DC active-active services, improving service quality.
Cross-DC microsegmentation and service function chain are available, enabling flexible service deployment.
When IPv4 north-south traffic passes through the firewall, DNAT, SNAT, or EIP needs to be deployed on the firewall based on service access scenarios. In this way, private IPv4 addresses can be translated or mapped to public IPv4 addresses.
When IPv6 service traffic exists, only IPv6 addresses in the data center and IPv4 addresses outside the data center are supported. In this case, only IPv6 hosts proactively access IPv4 resources and IPv4 hosts proactively access IPv6 resources are supported. In this case, the traffic passing through the firewall needs to be translated using NAT64. The following sections describe the two service scenarios.
External IPv4 hosts proactively access internal IPv6 resources
When an external IPv4 host proactively accesses internal IPv6 resources, the static NAT64 mapping needs to be deployed on the firewall. The Figure Static NAT64 flowchart shows the main process of this scenario.
An IPv4 user sends request A (www.admin.com) to the DNS.
After receiving the request packet, the DNS parses the IPv4 address (188.8.131.52) corresponding to the domain name (If the request packet does not contain any IPv4 address, the packet is discarded. In this scenario, mapping between the domain name and address is predefined in the DNS), and sends the reply packet to the user.
After receiving the DNS reply, the user sends the parsed address as a destination address to the remote server.
After receiving the IPv4 packet from the user, the NAT64 device refers to configured static mapping (a generated server map table), translates the destination IPv4 address to an IPv6 address (3000::2) as the destination address for the IPv6 packet, combines the IPv4 packet with the configured NAT64 prefix into the source address (64:ff9b::0101:0101) for the IPv6 packet (which is translation from an IPv4 packet to an IPv6 packet), and sends it to the server on the IPv6 network. A session table is generated.
After receiving the packet, the server replies to the packet.
After receiving the reply packet from the IPv6 server, the NAT64 device translates the IPv6 packet into an IPv4 packet according to the session table, and sends the IPv4 packet to the IPv4 user.
Internal IPv6 hosts proactively access external IPv4 resources
When IPv6 hosts in a data center proactively access an external IPv4 resources, the dynamic NAT64 mapping needs to be deployed on the firewall. The Figure Dynamic NAT64 flowchart shows the main process of this scenario.
A single-stack IPv6 user initiates an AAAA DNS request for remote services (www.admin.com).
After receiving the request, DNS64 parses the AAAA request. If an IPv6 address cannot be found, the user sends an A request initiated again. DNS64 parses the request and finds an IPv4 address. Based on the configuration of Prefix64::/n (64:ff9b::/96), DNS64 sends the NAT64 address (64:ff9b::0101:0101) to the user. The address parsing is completed.
After receiving the DNS64 reply, the user sends the parsed address as a destination address to the remote server.
The NAT64 device receives the IPv6 packet from the user, and uses the address translation algorithm to extract the IPv4 address (184.108.40.206) from the IPv6 packet as the destination address for the IPv4 packet. The NAT64 device refers to the mapping configured in the NAT64 policy, uses the address in the NAT address pool as the source address (220.127.116.11) of the IPv4 packet, translates the IPv6 packet to an IPv4 packet, and sends it to the server on the IPv4 network. A session table with mapped addresses is generated.
After receiving the packet, the server replies to the packet.
After receiving the reply packet from the IPv4 server, the NAT64 device translates the IPv4 packet into an IPv6 packet according to the session table, and sends the IPv6 packet to the IPv6 user.
Huawei officially announced its Intent-Driven Network (IDN) Solution at the 2018 Mobile World Congress (MWC). This solution bridges the gap between a physical network and business intent, drives evolution from the SDN network to the autonomous network, and maximizes business value.
The solution digitizes user experience and networks, shifts from hardware device focus to user-centric, and provides services based on the digital world, improving user experience and achieving preventive maintenance.
The core of the solution is based on artificial intelligence (AI), big data, and cloud computing technologies. Driven by user business logic and business strategies, the solution builds intelligent, simplified, ultra-broadband, secure, and open networks. IDN brings the following changes:
From network-centric to user-centric
From decentralized management to closed-loop management
From passive response to proactive prediction
From technology dependency to AI and automation
The IDN solution based on intent orchestration has the following characteristics:
Translation and verification: The Agile Controller-DCN translates intents and converts them into network configurations. It generates the configurations and verifies the correctness of the generated design and configurations.
Automatic implementation: The Agile Controller-DCN supports intent orchestration and network automation, facilitating network changes based on the existing network infrastructure.
Understanding of the network status: The Agile Controller-DCN can extract the real-time status of the network and is irrelevant to protocols and transmission.
Guarantee and dynamic optimization: The Agile Controller-DCN can continuously verify whether the original service intents are met in real time and take corrective actions (such as notifying, blocking traffic, and modifying the network capacity) when the required intent is not met.