No relevant resource is found in the selected language.

This site uses cookies. By continuing to browse the site you are agreeing to our use of cookies. Read our privacy policy>

Reminder

To have a better experience, please upgrade your IE browser.

upgrade

Enable Fencing feature, nodes unexpected power loss and unable to join cluster

Publication Date:  2014-09-30 Views:  70 Downloads:  0
Issue Description
In the N8000 cluster environment, enabled fencing feature. when unexpected power loss for N8300 and all nodes are not online at the same time after power resume, some of nodes will be failed to join the cluster.
Alarm Information
None
Handling Process
1 After the power resume, run the command in node:

N8000_02:~ # gabconfig -a
GAB Port Memberships
===============================================================
Port a gen  1368302 membership 01

2 Clear the key in one of the node:

N8000_02:/opt/VRTSvcs/vxfen/bin/vxfenclearpre
……
         ******** WARNING!!!!!!!! ********

THIS SCRIPT CAN ONLY BE USED IF THERE ARE NO OTHER ACTIVE NODES IN
THE CLUSTER!  VERIFY ALL OTHER NODES ARE POWERED OFF OR INCAPABLE OF
ACCESSING SHARED STORAGE.
If this is not the case, data corruption will result.
Do you still want to continue: [y/n] (default : n)y
Cleaning up the coordinator disks...
……                       

3 Restart all nodes in the same time

N8000_02:~ # reboot
Broadcast message from root (pts/2) (Fri Aug 29 13:01:23 2014):
n8000_01:~ # reboot
Broadcast message from root (pts/3) (Fri Aug 29 13:01:23 2014):

4 After restart complete, all nodes join the Cluster:

N8000_02:~ # gabconfig -a
GAB Port Memberships
===============================================================
Port a gen  1368302 membership 01
Port b gen  1368304 membership 01
Port f gen  136830d membership 01
Port h gen  1368306 membership 01
Port v gen  1368309 membership 01
Port w gen  136830b membership 01
n8000_02:~ #
N8000_02:~ # vxclustadm nidmap
Name                             CVM Nid    CM Nid     State              
N8000_01                          1          0          Joined: Slave      
N8000_02                          0          1          Joined: Master     

Root Cause
When fencing is enabled in N8000, all nodes detect other nodes status by the heartbeat link. if some nodes are not detectable, it will repeated for 5 times to check, for a total detection time of 25 seconds, after that the undetectable node will be considered offline, the current node will seize the coordinator disks successfully , other nodes will not be able to write information to coordinate disks, and failed to join Cluster and provide service.

END