Services Are Interrupted Several Seconds Due to a Short Preemption Time on the Standby Device

Publication Date:  2015-07-02 Views:  244 Downloads:  0
Issue Description
Networking:

As shown in Figure 4-3, two USGs and two switches form a rectangle network. Hot standby is enabled on the USGs.

Figure 4-2 Networking where services are interrupted several seconds due to a short preemption time on the standby device




Fault Symptom:

After GigabitEthernet0/0/1 connecting the active firewall FW1 to SW1, services immediately switch to the standby firewall FW2. After the interface becomes Up, services are interrupted several seconds before switching back to the active firewall.
Handling Process
After FW1 preempts to the active firewall, it sends gratuitous ARP packets to refresh the ARP table and MAC forwarding table on the connected switch. During the seven to eight seconds when services are interrupted, the ARP table on the switch is not refreshed, and the switch continues to forward packets to the port of the standby firewall. Before FW1 preempts to the active firewall, FW1 broadcasts five packets, including one gratuitous ARP packet destined for the interface IP address and four gratuitous ARP packets destined for the virtual IP address of the VRRP group. Compare packet statistics on GigabitEthernet0/0/1 connected to the switch before and after preemption. It is found that the number of sent broadcast packets is increased from 187 to 192, which indicates that FW1 has sent gratuitous ARP packets to the switch.

0.18482116 ECP-FW-USG-A %%01ARP/7/arp_send(d): Send an ARP Packet, operation : 1, sender_eth_addr : 0022-a104-5b4d,sender_ip_addr : 192.168.255.2, target_eth_addr : 0000-0000-0000, target_ip_addr : 192.168.255.2
0.18492433 ECP-FW-USG-A %%01ARP/7/arp_send(d): Send an ARP Packet, operation : 1, sender_eth_addr : 0000-5e00-0101,sender_ip_addr : 192.168.255.1, target_eth_addr : 0000-0000-0000, target_ip_addr : 192.168.255.1
0.18496433 ECP-FW-USG-A %%01ARP/7/arp_send(d): Send an ARP Packet, operation : 1, sender_eth_addr : 0000-5e00-0101,sender_ip_addr : 192.168.255.1, target_eth_addr : 0000-0000-0000, target_ip_addr : 192.168.255.1
0.18502433 ECP-FW-USGG-A %%01ARP/7/arp_send(d): Send an ARP Packet, operation : 1, sender_eth_addr : 0000-5e00-0101,sender_ip_addr : 192.168.255.1, target_eth_addr : 0000-0000-0000, target_ip_addr : 192.168.255.1
0.18508433 ECP-FW-USG-A %%01ARP/7/arp_send(d): Send an ARP Packet, operation : 1, sender_eth_addr : 0000-5e00-0101,sender_ip_addr : 192.168.255.1, target_eth_addr : 0000-0000-0000, target_ip_addr : 192.168.255.1
¡­¡­


Compare packet statistics on GigabitEthernet0/0/24 connecting SW1 to FW1 before and after preemption. It is found that the number of received broadcast packets is increased from 600 to 605, which indicates that SW1 has received the five gratuitous ARP packets from FW1. Enable ARP debugging on SW1. The debugging information shows that the CPU of SW1 receives only one gratuitous ARP packet destined for the virtual VRRP group address and the first four ARP packets are not sent to the CPU of SW1. When the debugging information on the switch shows that the gratuitous ARP packet for the virtual VRRP group address is received, services recover immediately. The CPU of SW1 can receive ARP packets properly several seconds after the status of the interface connecting SW1 to FW1 changes from Down to Up.

Aug 15 2011 19:28:17.590.1-05:13 ECP-S5328C-EI-A ARP/7/arp_rcv:Receive an ARP Packet, operation : 1, sender_eth_addr : 0000-5e00-010 1, sender_ip_addr : 192.168.255.1, target_eth_addr : 0000-0000-0000, target_ip_addr : 192.168.255.1
Root Cause
The interface connecting the S5328 to the firewall can receive ARP packets properly several seconds after its status changes from Down to Up. As a result, the switch cannot refresh its ARP entries in time and still sends services to the firewall, causing a short-time service interruption.
Solution
Run the hrp preempt delay30 command to change the HRP preemption time of the active firewall from 10s to the default value (30) for smooth service switching from the active firewall to the standby firewall.

END