No relevant resource is found in the selected language.

This site uses cookies. By continuing to browse the site you are agreeing to our use of cookies. Read our privacy policy>

Reminder

To have a better experience, please upgrade your IE browser.

upgrade

Server is contonuosly switching from Primary to Secondary very frequently

Publication Date:  2012-07-25 Views:  3 Downloads:  0
Issue Description
Server is continuosly switching from Primary to Secondary server, and other server is getting faulted.
Alarm Information
And server is showing faulty, need to manually clear the fault.
Handling Process
Checked & found Sybase issue, Sybase & U2000 both patches needs to be installed.
Root Cause
The HA system failed to query the U2000 process running status, causing the active/standby switchover.
Analysis process:
NMS server logs (stored in the /opt/HWENGR/NMSApp/log/monitor directory) are as follows:
12-05-21 18:46:35 || svc_adm query retrytimes start at 2012-05-21_18:46:34
12-05-21 18:46:35 || haMonitor:Success to get the process online retry count at 2012-05-21_18:46:35.
12-05-21 18:46:35 || svc_deploy query svc start at 2012-05-21_18:46:35
          Error : Failed to query svc.May be no host exists.
12-05-21 18:46:37 || haMonitor:Fail to get the process start mode 1 times at 2012-05-21_18:46:37!
12-05-21 18:46:37 || svc_deploy query svc start at 2012-05-21_18:46:37
          Error : Failed to query svc.May be no host exists.
12-05-21 18:46:38 || haMonitor:Fail to get the process start mode 2 times at 2012-05-21_18:46:38!
12-05-21 18:46:38 || svc_deploy query svc start at 2012-05-21_18:46:38
          Error : Failed to query svc.May be no host exists.
12-05-21 18:46:40 || haMonitor:Fail to get the process start mode 3 times at 2012-05-21_18:46:40!
12-05-21 18:46:40 || haMonitor:Fail to execute svc_deploy -cmd querysvc -col sysdname,svcname,startuptype 3 tiems.
 
Causes for the failure to query the U2000 process running status:
        The HA system called the svc_deploy function to query the current U2000 process running status recorded in the database. No correct status query result was returned due to errors on the database. Therefore, the HA system considered that the U2000 processes are faulty.
VCS logs show that the VCS went offline at 18:46:43 and the U2000 started an active/standby switchover.
2012/05/21 18:46:42 VCS ERROR V-16-2-13067 (Primaster) Agent is calling clean for resource(NMSServer) because the resource became OFFLINE unexpectedly, on its own.
2012/05/21 18:46:49 VCS INFO V-16-2-13068 (Primaster) Resource(NMSServer) - clean completed successfully.
2012/05/21 18:46:51 VCS INFO V-16-1-10307 Resource NMSServer (Owner: Unspecified, Group: AppService) is offline on Primaster (Not initiated by VCS)
 
Causes for errors on the database:
        The size of the database procedure cache was insufficient, which caused that the Sybase database responded slowly to or even did not respond to the database access operations.
Database logs (stored in the /opt/sybase/ASE*/install directory) contain the following information:
04:00000:00007:2012/05/21 18:46:08.87 server  Error: 4201, Severity: 17, State: 2
04:00000:00007:2012/05/21 18:46:08.87 server  DUMP TRANSACTION for database 'tempdb' failed: insufficient memory to allocate backout structure. Raise the value of the configuration parameter 'procedure cache size'.
02:00000:00007:2012/05/21 18:46:08.87 server  Error: 4201, Severity: 17, State: 2
02:00000:00007:2012/05/21 18:46:08.87 server  DUMP TRANSACTION for database 'model' failed: insufficient memory to allocate backout structure. Raise the value of the configuration parameter 'procedure cache size'.
01:00000:00050:2012/05/21 18:46:54.01 kernel  Cannot read, host process disconnected: SYB_BACKUP  spid: 50
00:00000:00966:2012/05/21 18:47:15.33 server  Shutdown started by user 'sa'. SQL text: shutdown with wait
00:00000:00966:2012/05/21 18:47:16.00 kernel  PCI(M0): ASE_PCI: pci_shutdown; PCI Launcher Boss shutdown requested.
00:00000:00966:2012/05/21 18:47:16.00 server  ASE shutdown by request.
00:00000:00966:2012/05/21 18:47:16.00 kernel  ueshutdown: exiting
00:00000:00966:2012/05/21 18:47:16.04 kernel  SySAM: Checked in license for 1 ASE_CORE (2010.08100/permanent/1DF6 911E 1987 9DC7).
 

Suggestions
Install the Sybase patch ASE15.0.3 ESD#4 and the U2000 V100R005 C00CP6133 patch

END