Configuring PFC

CloudEngine 16800, 8800, and 6800 Series Switches AI Fabric Deployment Best Practices

This document describes the best practices for deploying the AI Fabric intelligent and lossless data center network solution (referred to as AI Fabric solution), helping network engineers understand how to use the functions provided by the AI Fabric solution to implement zero packet loss, low latency, and high throughput on data center networks.

Rate and give feedback:

Huawei uses machine translation combined with human proofreading to translate this document to different languages in order to help you better understand the content of this document. Note: Even the most advanced machine translation cannot match the quality of professional translators. Huawei shall not bear any responsibility for translation accuracy and it is recommended that you refer to the English document (a link for which has been provided).

Configuring PFC

Before configuring PFC, you are advised to plan priorities and configure the queue scheduling function to allow traffic of different services to enter different queues for forwarding. In this way, protocol packets and RoCEv2 packets that require zero packet loss can be preferentially forwarded, improving network resource utilization.

After planning priorities, enable PFC for the queues that require zero packet loss.

Planning Priorities
Enabling PFC

Planning Priorities

Before configuring the PFC function, plan priorities based on the QoS function. Determine whether to use the priority mapped from the 802.1p value or DSCP value to carry RoCEv2 traffic. Then, you need to plan queues. Figure 2-3 lists the recommended queue plan. (For the models recommended in this document, priorities and queues have a one-to-one mapping.)

Queues 7 and 6 are used to carry various protocol packets. To transmit CNP packets in a timely manner, queue 6 can be used to carry CNP packets.
At least one queue is required to carry RoCEv2 services. The PFC function must be enabled for the priorities mapped to the queues carrying RoCEv2 services. Queues 5, 4, and 3 are recommended to carry RoCEv2 service traffic.
If both TCP and RoCE traffic need to be transmitted, queues 2 and 1 can be used to carry TCP service traffic.
The strict priority (SP) scheduling mode (configured using the qos pq command) is recommended for queues 7 and 6, and the WDRR scheduling mode (configured using the qos drr command) is recommended for other queues. (The SP scheduling mode is also called the PQ scheduling mode.) If the WDRR scheduling mode is used, the recommended weights in descending order are as follows: queue carrying RoCEv2 packets > queue carrying TCP packets > queue 0.

Figure 2-3 Recommended queue plan

802.1p Value

# Configure the PFC function based on the priority mapped from the 802.1p value, and plan the 802.1p value of 3 for RoCEv2 packets. According to Table 2-4, RoCEv2 packets are transmitted in queue 3 by default. If a spine node needs to use the DSCP value of packets, you can enable the mapping between PHBs (internal priorities) and DSCP values in the outbound direction on the uplink interface of the leaf node.

<HUAWEI> system-view
[~HUAWEI]  interface 100ge 1/0/1
[~HUAWEI-100GE1/0/1] qos phb marking dscp enable

Table 2-4 Default mappings between 802.1p values and internal priorities/drop priorities

802.1p Value	Internal Priority (CoS)	Drop Priority (Color)	Queue
0	BE	Green	0
1	AF1	Green	1
2	AF2	Green	2
3	AF3	Green	3
4	AF4	Green	4
5	EF	Green	5
6	CS6	Green	6
7	CS7	Green	7

DSCP Value

# Configure PFC based on the priority mapped from the DSCP value. Assume that the DSCP value of RoCEv2 packets on the server is 23, the DSCP value of CNP packets is 24, and RoCEv2 and CNP packets need to be transmitted in queues with priorities 3 and 6, respectively. According to Table 2-5, the default mapping rules do not meet the requirements.

In this case, you can configure a priority mapping profile for the DiffServ domain as follows: Map the priority of RoCEv2 packets to priority 3 (queue 3), map the priority of CNP packets to priority 6 (queue 6), and configure PQ scheduling.

<HUAWEI> system-view
[~HUAWEI] diffserv domain ds1 
[*HUAWEI-dsdomain-ds1] ip-dscp-inbound 23 phb af3 green   //Map the priority of RoCEv2 packets to priority 3.
[*HUAWEI-dsdomain-ds1] ip-dscp-inbound 24 phb cs6 green   //Map the priority of CNP packets to priority 6.
[*HUAWEI-dsdomain-ds1] quit 
[*HUAWEI] interface 100ge 1/0/1
[*HUAWEI-100GE1/0/1] trust dscp   //Configure mapping for packets based on DSCP values. By default, Layer 3 packets are processed based on the mapping of DSCP values.
[*HUAWEI-100GE1/0/1] trust upstream ds1   //Apply the priority mapping profile for the DiffServ domain to the interface.
[*HUAWEI-100GE1/0/1] qos pq 6   //Configure PQ scheduling for queue 6. This operation is optional because PQ scheduling is the default scheduling mode for queue 6.

Table 2-5 Default mappings between DSCP values and internal priorities/drop priorities

DSCP Value	Internal Priority (CoS)	Drop Priority (Color)	DSCP Value	Internal Priority (CoS)	Drop Priority (Color)
0	BE	Green	32	AF4	Green
1	BE	Green	33	AF4	Green
2	BE	Green	34	AF4	Green
3	BE	Green	35	AF4	Green
4	BE	Green	36	AF4	Yellow
5	BE	Green	37	AF4	Green
6	BE	Green	38	AF4	Red
7	BE	Green	39	AF4	Green
8	AF1	Green	40	EF	Green
9	AF1	Green	41	EF	Green
10	AF1	Green	42	EF	Green
11	AF1	Green	43	EF	Green
12	AF1	Yellow	44	EF	Green
13	AF1	Green	45	EF	Green
14	AF1	Red	46	EF	Green
15	AF1	Green	47	EF	Green
16	AF2	Green	48	CS6	Green
17	AF2	Green	49	CS6	Green
18	AF2	Green	50	CS6	Green
19	AF2	Green	51	CS6	Green
20	AF2	Yellow	52	CS6	Green
21	AF2	Green	53	CS6	Green
22	AF2	Red	54	CS6	Green
23	AF2	Green	55	CS6	Green
24	AF3	Green	56	CS7	Green
25	AF3	Green	57	CS7	Green
26	AF3	Green	58	CS7	Green
27	AF3	Green	59	CS7	Green
28	AF3	Yellow	60	CS7	Green
29	AF3	Green	61	CS7	Green
30	AF3	Red	62	CS7	Green
31	AF3	Green	63	CS7	Green

Enabling PFC

RoCEv2 service traffic needs to be transmitted without packet loss. Therefore, an independent priority needs to be planned for RoCEv2 traffic, and the PFC function needs to be enabled based on the priority.

# Configure the queue with priority 3 to carry RoCEv2 traffic on the network. To implement this, enable PFC for priority 3 on each interface and implement PFC based on the priority mapped from the DSCP value.

<HUAWEI> system-view
[~HUAWEI] dcb pfc mypfc   //Create a PFC profile.
[*HUAWEI-dcb-pfc-mypfc] priority 3   //Enable PFC for priority 3.
[*HUAWEI-dcb-pfc-mypfc] quit
[*HUAWEI] interface 100ge 1/0/1
[*HUAWEI-100GE1/0/1] dcb pfc enable mypfc mode manual   //Configure PFC to work in forcible mode.
[*HUAWEI-100GE1/0/1] quit
[*HUAWEI] dcb pfc dscp-mapping enable slot 1   //Implement PFC based on the priority mapped from the DSCP value.
[*HUAWEI] commit

Precautions:

If you configure both priority mapping and PFC, exercise caution when modifying the mappings between the priorities of PFC-enabled queues and priorities in the DiffServ domain. Otherwise, PFC may not work.

In the following scenarios, if a queue is congested, PFC may fail to be triggered and does not take effect:
- The configured threshold for triggering PFC frames is too high.
- The configured queue buffer threshold is too low.
- The burst traffic buffer mode is set to low-latency.

Verifying the PFC configuration

After PFC is configured successfully, the default PFC threshold is automatically allocated to lossless queues. The default thresholds vary according to the version and device. For details about how to manually configure the PFC thresholds, refer to Adjusting the Buffer and PFC Thresholds.

<HUAWEI> display dcb pfc buffer                                                    
Xon:        PFC backpressure stop threshold                                     
Xoff:       PFC backpressure threshold                                          
Hdrm:       Headroom buffer threshold                                           
Guaranteed: PFC guaranteed buffer threshold                                     
The actual PFC backpressure stop threshold is the higher value between the value
of xon and the difference between the value of xoff and the value of xon-offset.
C:cells   B:bytes   K:kilobytes   M:megabytes   D:dynamic alpha                 
------------------------------------------------------------------------------------
Interface      Queue    Guaranteed         Xon Xon-Offset       Xoff        Hdrm
------------------------------------------------------------------------------------
100GE1/0/1         3         10(C)       50(C)      20(C)      85(C)      168(C)

------------------------------------------------------------------------------------

Planning Priorities
Enabling PFC

Translation

简体中文

Download

Updated: 2021-12-16

Document ID: EDOC1100155250

Downloads: 270

Average rating:

This Document Applies to these Products

Related Version

Digital Signature File

Digital Signature Authentication Mode