No relevant resource is found in the selected language.

This site uses cookies. By continuing to browse the site you are agreeing to our use of cookies. Read our privacy policy>Search

Reminder

To have a better experience, please upgrade your IE browser.

upgrade

HUAWEI CLOUD Stack 6.5.0 Troubleshooting Guide 02

Rate and give feedback :
Huawei uses machine translation combined with human proofreading to translate this document to different languages in order to help you better understand the content of this document. Note: Even the most advanced machine translation cannot match the quality of professional translators. Huawei shall not bear any responsibility for translation accuracy and it is recommended that you refer to the English document (a link for which has been provided).
Abnormal Master and Slave Database Instances

Abnormal Master and Slave Database Instances

Symptom

The master and slave database instances are both abnormal, and physical database files are lost or damaged. This scenario involves the loss of Gauss, Redis software, or instance directory.

Fault Locating
  1. Use PuTTY to Log in as the dbuser user to the active and standby nodes where database instance resides.

    The default username is paas, and the default password is QAZ2wsx@123!.

  2. Run the following commands to check whether the instance file is lost:

    cd /opt/

    ll

  3. Check whether the Gauss or Redis directory exists. If the directory does not exist, perform operations provided in Troubleshooting step by step. If the directory exists, check whether files in the directory are damaged. If files are not damaged, skip Initial Installation of Database Software, and perform operations provided in Recreating the Master and Slave Database Instances.

Troubleshooting

Initial Installation of Database Software
Procedure
  1. Use PuTTY to log in to the manage_lb1_ip node.

    The default username is paas, and the default password is QAZ2wsx@123!.

  2. Install database software by recreating pod.
  3. Run the following command and enter the password of the root user to switch to the root user:

    su - root

    Default password: QAZ2wsx@123!

  4. Run the following command to obtain the name of the pod corresponding to the IP address of DBAgent.

    kubectl get pod -n fst-manage -owide| grep dbagent

    Information similar to the following is displayed:

    dbagent-fl17s     1/1       Running   0          19h       10.8.41.66   paas-10-8-41-66
    dbagent-ndcqh     1/1       Running   0          20h       10.8.41.73   paas-10-8-41-73

  5. Run the following command to delete the DBAgent services pod of the master and slave database:

    kubectl delete pod dbagent-ndcqh -nfst-manage

    Information similar to the following is displayed:

    pod "dbagent-ndcqh" deleted
    NOTE:

    After the pod is deleted, the container re-establishes a new pod. If you want to confirm the pod, perform 4.

  6. Use PuTTY to log in to the slave database node as paas user.

    The default password is QAZ2wsx@123!.

  7. Run the following command to switch to the root user:

    su - root

    The default password is QAZ2wsx@123!.

  8. Run the following command to check whether the database software is successfully installed:

    ll /opt/gauss

    drwx------ 5 dbuser dbgroup 4096 Dec 14 13:51 app
    -rw------- 1 dbuser dbgroup 1250 Dec 14 13:51 gauss_product.conf
    NOTE:

    If database instance is abnormal because the standby instance directories are lost, run the following command on the slave instance node, and then rectify based on Abnormal Replication State of Slave Database Instance.

    cd /opt/paas/oss/manager/apps/DBAgent/bin

    ./dbsvc_tool -cmd repair-db-node -type gauss

Recreating the Master and Slave Database Instances
Procedure
  1. Use PuTTY to log in as the paas user to the master and slave database instances respectively.

    The default username is paas, and the default password is QAZ2wsx@123!.

  2. Run following commands to recreate the database:

    cd /opt/paas/oss/manager/apps/DBAgent/bin

    ./dbsvc_tool -cmd repair-db-node -type gauss

    Information similar to the following is displayed:

     success
    NOTE:
    • When recreating the database, the database process whose master and slave instance have been lost must be stopped.
    • The master database must be re-created before the slave database.
    • After the instance is successfully created, perform logical recovery on the master database instance.
    • type indicates the database type to be restored. If the database is Redis, run the following command instead:

      ./dbsvc_tool -cmd repair-db-node -type redis

    • success indicates that the database instance is restored successfully.

  3. Restore master database instance on the active node of the database as instructed in section Restoring Gauss of HUAWEI CLOUD Stack 6.5.0 Backup and Restoration Guide.

    NOTE:
    • The restoration only needs to be performed on the active node.
    • If the instance is not backed up before the disk is damaged, the data cannot be restored.
    • If you recreate a Redis database, the database table needs to be recreated.

Verification
  1. Use PuTTY to log in to the manage_db1_ip node.

    The default username is paas, and the default password is QAZ2wsx@123!.

  2. Run the following commands to check information about database:

    cd /opt/paas/oss/manager/apps/DBAgent/bin

    ./dbsvc_adm -cmd query-db-instance | grep gauss

    Information similar to the following is displayed:

    DBInstanceId                                  ClassId  InstNumber                    Tenant          IP             Port   State  DBType  Version            Role    Rpl Status  MasterID                      GuardMode  DataCheckSum  isSSL
    apmdbsvr-10_186_66_155-1@10_186_67_174-1      primary  apmdbsvr-10_186_66_155-1      fst-manage      10.186.66.155  32081  Up     gauss   V100R003C20SPC112  Master  Normal      --                            --         1520954391    off
    apmdbsvr-10_186_66_155-1@10_186_67_174-1      primary  apmdbsvr-10_186_67_174-1      fst-manage      10.186.67.174  32081  Up     gauss   V100R003C20SPC112  Slave   Normal      apmdbsvr-10_186_66_155-1      --         1520954391    off

    You only need to pay attention to the value of Rpl Status.

    If the statuses of Master and Slave are Normal, the database is running properly.

    If the statues are abnormal, contact technical support.

    NOTE:

    Check information about Redis database, run the following commands:

    ./dbsvc_adm -cmd query-db-instance -type redis

    You only need to pay attention to the value of Rpl Status.

    If the statuses of Master and Slave are Normal, the database is running properly.

    If the statues are abnormal, contact technical support.

Translation
Download
Updated: 2019-06-01

Document ID: EDOC1100062375

Views: 1145

Downloads: 12

Average rating:
This Document Applies to these Products
Related Documents
Related Version
Share
Previous Next