Network topology: NE40 (R)-OSN3500-OSN3500-NE40 (Y)-OSN3500-OSN3500-NE40 (D)
Fault phenomenon: NE40 routers are distributed in different nodes (R, Y, and D), and communicate with each other through the OSN3500 on the MPLS BGP VPN and transmit VPN services to devices (CEs) on the core network. The CE in node R can ping through the CE in node Y instead of node D through the packet with the length being greater than 1468 bytes. The transmission failure of large-size packets affects the maintenance of CEs and thus leads to the intermittent interruption of network elements on the M2000 connecting to the NE40.
Change the MTU value of the interface on the transmission device OSN3500 from the default 1522 to 1548.
The MTU value of the device is highly likely leads to the transmission failure of large-size packets across the MPLS network.
1. The CE in node R can transmit the packet with the length being smaller than or equal to 1468 bytes to the CE in node D. This is because the length of the original packet sent from the CE in node R is 1496 bytes (1468 bytes + 8-byte ICMP header + 20-byte IP header). When the packet is sent to the NE40, the NE40 does not fragment the packet because the packet length (1496) is smaller than the MTU value (1500) on the NE40. After checking its forwarding table, the NE40 adds double MPLS labels (2 x 4 = 8 bytes) and 18-byte Layer 2 header to the packet, and then sends the packet with the length being 1522 bytes to the OSN3500. The default MTU value of the interface on the OSN3500 is 1522 bytes. Therefore, the packet can be successfully transmitted.
2. The packet with the length ranging from 1468 to 1472 (included) is discarded on the OSN3500. For example, when a 1471-byte packet is sent to the NE40, its length is 1499 bytes (1471 + 8 + 20). On the NE40, its length is increased to 1525 bytes (1499 + 8 + 18), which exceeds the default value of the interface on the OSN3500. Therefore, the packet is discarded by the OSN3500.
3. When a packet with the length being 1473 bytes is sent to the NE40, its length becomes 1501 bytes (1473 + 8 + 20), exceeding the MTU of the interface on the NE40. Therefore, the packet is fragmented on the NE40, with the maximum packet length being 1500 bytes. Then, the NE40 adds 26 bytes (8 + 18) to the 1500-byte fragmented packet, making the packet length being 1522 bytes and thus the packet is discarded on the OSN3500.
4. The packet with the length being greater than 1472 bytes can be successfully transmitted from the CE in node R to the directly connected CE in node R. This is because the packet is added with only one MPLS label (4 bytes) on the PE. The total length of the packet is 1522 bytes (1500 + 4 + 18), and therefore can be received and then transmitted by the OSN3500.
5. The MTU of the interface on the NE40 is the default value (1500) and does not include the length of the MPLS labels. The length of the fragmented packet is always smaller than 1500 bytes and therefore can be transmitted.
The transmission failure of packets across the MPLS VPN network is highly likely caused by the MTU value. You can check the MTU value of every node along the path that the Ping packet passes to locate the fault.