FAQ-Split-Brain Occurs on the VMU Two-Node Cluster

Publication Date:  2015-10-14 Views:  806 Downloads:  0
Issue Description
In a VMU two-node cluster, the active server is restarted due to a power outage. After services are switched to the standby server, some data is inconsistent between the active and standby servers.
Solution
Step 1 Check whether DRBD split-brain occurs on the VMU two-node cluster.

1. Log in to a node of the VMU two-node cluster as the root user.

2. Run the cat /proc/drbd command.

If the following information in bold is displayed, DRBD split-brain occurs.
0: cs:WFConnection ro:Secondary/Unknown ds:UpToDate/DUnknown C r----

3. Check whether DRBD split-brain occurs on the VMU two-node cluster.

− If no, contact Huawei technical support engineers.
− If yes, go to the next step.

Notice:
   You can retain only the data on one node when rectifying the DRDB split-brain. The data on the other node will be discarded. Exercise caution when selecting the node whose data you want to retain.

Step 2 Verify that the power module and heartbeat cables are properly connected.

Step 3 Run the ethtool heartbeat network adapter name command to check whether the network adapter rate of the server is the same as the planned bandwidth, as shown in Figure 1-1.

Figure 1-1 Checking the network adapter rate

  • If no, locate and rectify the network fault and check whether the fault is rectified.
If the fault is rectified, no further action is required. If the fault persists, go to Step 4.
  • If yes, go to the next step.
Step 4 On the node whose data you want to discard, run the following commands:

1. Run the drbdadm secondary r0 command to set the DRBD as the standby node.
2. Run the drbdadm disconnect r0 command to disconnect the standby node DRBD from resources.
3. Run the drbdadm -- --discard-my-data connect r0 command to discard the DRBD data on the standby node.

Step 5 On the node whose data you want to retain, run the drbdadm connect r0 command to set the DRBD as the active node.

Step 6 On the active or standby node, run the cat /proc/drbd command to check whether DRBD split-brain is rectified.

If the following information is displayed, DRBD split-brain is rectified.
cs:Connected ro:Primary/Secondary ds:UpToDate/UpToDate C r----

Step 7 Check whether the fault is rectified.

If the fault is rectified, no further action is required. If the fault persists, contact Huawei technical support engineers.

END