No relevant resource is found in the selected language.

This site uses cookies. By continuing to browse the site you are agreeing to our use of cookies. Read our privacy policy>Search

Reminder

To have a better experience, please upgrade your IE browser.

upgrade

FusionCloud 6.3.1.1 Troubleshooting Guide 02

Rate and give feedback :
Huawei uses machine translation combined with human proofreading to translate this document to different languages in order to help you better understand the content of this document. Note: Even the most advanced machine translation cannot match the quality of professional translators. Huawei shall not bear any responsibility for translation accuracy and it is recommended that you refer to the English document (a link for which has been provided).
Node Faults

Node Faults

Troubleshooting for Deploy Nodes

Symptom

The troubleshooting methods provided in this section are used to troubleshoot faults on Deploy nodes only in the following scenarios:

  • The nodes are unavailable because an OS is faulty.
  • The database of the service deployment system is damaged.
    • The replication status of slave database instances is abnormal.
    • The slave database instance is abnormal.
    • The master database instance is abnormal.
  • Files are incorrectly deleted or disks are damaged.
OS Faults
Symptom

If the OS cannot be used, an OS fault occurs. Currently, the fault occurs only when the deployment system is deployed in a two-node cluster and one node is faulty. For example: If the OS cannot be logged in, you can rectify the OS fault using operation instructions.

Prerequisites

You have obtained the OS image files.

Procedure
  1. On the Service OM homepage, choose Computing > ECS > Compute Instances.
  2. Click next to More in the Operation column.
  3. In the displayed drop-down list, click Stop.
  4. In the displayed dialog box, click OK to stop the faulty VM.
  1. Click the faulty VM name and choose Hardware > Disk.
  2. On the Disk page, click Unbind in the Operation column.
  3. In the displayed dialog box, click OK to unbind the data disk from the faulty VM.
  1. On the computing instance page, click next to More in the Operation column.
  2. In the displayed drop-down list, click Delete.
  3. In the displayed dialog box, click OK to delete the faulty VM.
  4. Reinstall the VM (using the IP address same as the faulty node) by referring to section How Do I Create a VM? Change the time zone of the faulty node by referring to section Changing the Time Zone. The date time must be the same as the time of any running ManageOne VM. If the time is inconsistent, run the following command:

    date -s Time of the ManageOne VM that is running properly

    For example: date -s 2018-06-01 12:10:15

  5. Run the following commands to configure the VM service port:

    echo "34001 60999" > /proc/sys/net/ipv4/ip_local_port_range

    sed -i '/^net.ipv4.ip_local_port_range*/d' /etc/sysctl.conf

    echo "net.ipv4.ip_local_port_range = 34001 60999" >> /etc/sysctl.conf

  6. On the Service OM homepage, choose Storage > EVS > Disks.
  7. Click next to More in the Operation column.
  8. In the displayed drop-down list, click Bind VM.
  9. In the displayed dialog box, click OK to bind the old disk to the new VM.
  10. Use PuTTY to log in to the faulty node.

    NOTE:

    You can log in to the ManageOne deployment plane and choose Resource > Server to view the faulty node.

    Default account: sopuser; default password: D4I$awOD7k

  11. Run the following command to switch to the root user:

    su - root

    The default password is Changeme_123.

  12. Run the following commands to mount the old disk to the new VM:

    mount /dev/mapper/oss_vg-opt_vol /opt

    mount /dev/mapper/oss_vg-optlog_vol /opt/log

    vi /etc/fstab

    Add the following content to the fstab file:

    /dev/oss_vg/opt_vol /opt ext4 defaults,usrquota,grpquota,nodev,nosuid 0 2

    /dev/oss_vg/optlog_vol /opt/log ext4 defaults,nodev,nosuid 0 2

  13. Determine the faulty node.

    • If the deploy01 node is faulty, perform 21 through 23 and 25 through 29.
    • If the deploy02 node is faulty, perform 24 through 25 and 29.

  14. Run the following commands to decompress the DeployAgent package to the specified directory:

    cd /opt/pub/software/repository/Services/ServiceDeploy/DeployAgent/

    unzip -o DeployAgent-1.28.65.zip -d baseagent

    NOTE:

    1.28.65 is the version number. Replace it with the actual version number.

  15. Run the following command to copy the check_create_user_group.sh file to the manager directory:

    cp baseagent/tools/shscript/manager/check_create_user_group.sh /opt/oss/manager/agent/tools/shscript/manager/

  16. Run the following commands to create users and user groups for the service deployment system:

    export SOP_ROOT=/opt

    /bin/bash ${SOP_ROOT}/oss/manager/agent/tools/shscript/manager/check_create_user_group.sh
    NOTE:

    SOP_ROOT is the root directory for installing the service deployment system. The /opt is used as an example.

  17. Initialize the faulty node by referring to section Initializing the Faulty Node.
  18. Redeploy the services on the faulty node by referring to section Redeploying Services on the Faulty Node, and restore the database replication status by referring to section Slave Database Instance Is Abnormal.
  19. Run the following commands to configure NTP clock sources for deploy01 node:

    sed -i "/server .* maxpoll 4 minpoll 3/d" /etc/ntp.conf;

    echo "server IP address of the NTP clock source maxpoll 4 minpoll 3 prefer">>/etc/ntp.conf;

    NOTE:

    If there are multiple NTP clock sources, repeat this command to add the clock sources in sequence.

  20. Run the following command to restart the NTP service:

    service ntpq -p restart

  21. Run the following command to check whether the NTP clock source is configured successfully:

    cat /etc/ntp.conf |grep "server NTPIP maxpoll 4 minpoll 3 prefer"

    • If yes, the IP address is prefixed with an asterisk (*).
    • If no, contact technical support for assistance.

  22. After the faults are restored, perform OS security hardening by referring to section Hardening the OS.
  23. Run the following command to log out of the root user:

    exit

Database Faults
Symptom

If the service deployment system becomes abnormal due to a database error, a database fault occurs. For example, database instances are abnormal. You must restore the database to rectify a database fault.

Prerequisites

Service deployment system databases have been periodically backed up. For details about how to back up databases, see "Backing Up ManageOne Data" in FusionCloud 6.3.1.1 Backup and Restoration Guide. If the databases are not periodically backed up, databases cannot be restored in certain scenarios (for example, the master and slave database instances are abnormal).

Procedure

Two-node deployment

  1. Delete MySQL database instances on Deploy nodes except the database instances of the service deployment system.

    1. Log in to the service deployment system, and choose Deployment > Database > RDBMS from the main menu.
    2. In the upper right corner of the RDBMS page, enter the IP address of the Deploy node in the search box and press Enter. Record MySQL database instance names on Deploy nodes.
    3. Use PuTTY to log in to the node deploying ManageOne as the root user.
    4. Run the following commands to delete the specified database instance:

      su - ossadm

      /opt/oss/envs/Product-DataMgmtService/service/bin/dbsvc_adm -cmd delete-db-instance -instid Database instance name

  2. Delete Redis database instances on Deploy nodes except the database instances of the service deployment system.

    1. Log in to the service deployment system, and choose Deployment > Database > Redis from the main menu.
    2. In the upper right corner of the Redis page, enter the IP address of the Deploy node in the search box and press Enter. Record Redis database instance names on the Deploy node.
    3. Use PuTTY to log in to the node deploying ManageOne as the root user.
    4. Run the following commands to query and record database instances in the /opt/redis/data directory:

      cd /opt/redis/data

      ll

    5. Run the following commands to delete the specified database instance: .

      su - ossadm

      /opt/oss/envs/Product-DataMgmtService/service/bin/dbsvc_adm -cmd delete-db-instance -instid Database instance name

  3. On the deployment plane, choose Deployment > Feature Deployment > Services. Search for MOCustom, and click .

    NOTE:

    All services in the search result must be deployed.

  4. Restore ManageOne data by referring to section "Restoring ManageOne Data" in FusionCloud 6.3.1.1 Backup and Restoration Guide.

Single-node deployment

Restore the database data by following instructions provided in Slave Database Instance Is Abnormal.

Troubleshooting for Application Nodes

Scenarios

The troubleshooting methods provided in this section are used to troubleshoot faults on application nodes only in the following scenarios:

  • The nodes are unavailable because an OS is faulty.
  • Files are incorrectly deleted or disks are damaged.
OS Faults
Symptom

If the OS cannot be used, an OS fault occurs. For example, the OS cannot be logged in. You can rectify an OS fault using operation instructions.

Prerequisites

You have obtained the OS image files.

Procedure
  1. On the Service OM homepage, choose Computing > ECS > Compute Instances.
  2. Click next to More in the Operation column.
  3. In the displayed drop-down list, click Stop.
  4. In the displayed dialog box, click OK to stop the faulty VM.
  1. Click the faulty VM name and choose Hardware > Disk.
  2. On the Disk page, click Unbind in the Operation column.
  3. In the displayed dialog box, click OK to unbind the data disk from the faulty VM.
  1. On the computing instance page, click next to More in the Operation column.
  2. In the displayed drop-down list, click Delete.
  3. In the displayed dialog box, click OK to delete the faulty VM.
  4. Reinstall the VM (using the IP address same as the faulty node) by referring to section How Do I Create a VM? Change the time zone of the faulty node by referring to section Changing the Time Zone. The date time must be the same as the time of any running ManageOne VM. If the time is inconsistent, run the following command:

    date -s Time of the ManageOne VM that is running properly

    For example: date -s 2018-06-01 12:10:15

  5. Run the following commands to configure the VM service port:

    echo "34001 60999" > /proc/sys/net/ipv4/ip_local_port_range

    sed -i '/^net.ipv4.ip_local_port_range*/d' /etc/sysctl.conf

    echo "net.ipv4.ip_local_port_range = 34001 60999" >> /etc/sysctl.conf

  6. Initialize the faulty node by referring to section Initializing the Faulty Node.
  7. Run the following command to restart the NTP service:

    service ntpd restart

  8. On the Service OM homepage, choose Storage > EVS > Disks.
  9. Click next to More in the Operation column.
  10. In the displayed drop-down list, click Bind VM.
  11. In the displayed dialog box, click OK to bind the old disk in 7 to the new VM.
  12. Use PuTTY to log in to the faulty node as user sopuser.

    The default password is D4I$awOD7k.

  13. Run the following command to switch to the root user:

    su - root

    The default password is Changeme_123.

  14. Run the following commands to mount the old disk to the new VM:

    mkdir /opt/log

    mount /dev/mapper/oss_vg-opt_vol /opt

    mount /dev/mapper/oss_vg-optlog_vol /opt/log

    vi /etc/fstab

    Add the following content to the fstab file:

    /dev/oss_vg/opt_vol /opt ext4 defaults,usrquota,grpquota,nodev,nosuid 0 2

    /dev/oss_vg/optlog_vol /opt/log ext4 defaults,nodev,nosuid 0 2

  15. Redeploy services on the faulty node. For details, see section Redeploying Services on the Faulty Node.
  16. Run the following command to add a new DNS server:

    echo "nameserver dnsIP" >> /etc/resolv.conf

    NOTE:

    If there are multiple DNS IP addresses, run the command for multiple times.

  17. If the ElasticSearch service is deployed on the faulty node, perform the following substeps:

    1. Go to the bin directory in the ManageOne_init_tools folder.
    2. Use FileZilla to upload the es_modify_system_config.sh file to the /opt directory of the faulty node.
    3. Run the following command to replace the kernel file of the OS:

      bash /opt/es_modify_system_config.sh

  18. Run the following command to set the floating IP address:

    • If both the regionAlias-ManageOne-Service01 and regionAlias-ManageOne-Service02 are to be reinstalled, set the floating IP address of the OM plane on the regionAlias-ManageOne-Service01.
    • If both the regionAlias-ManageOne-Service03 and regionAlias-ManageOne-Service04 are to be reinstalled, set the floating IP address of the operation plane on the regionAlias-ManageOne-Service03.

    ifconfig eth0:0 FloatIP netmask Mast

    NOTE:

    Replace FloatIP and Mast as required.

  19. After the faults are restored, perform OS security hardening by referring to section Hardening the OS.
  20. Run the following command to log out of user root:

    exit

  21. Re-associate the restored node by referring to How Do I Associate a Restored Node on the OM Plane?
Database Faults
Symptom

If the service deployment system becomes abnormal due to a database error, a database fault occurs. For example, database instances are abnormal. You must restore the database to rectify a database fault.

Prerequisites

Service deployment system databases have been periodically backed up. For details about how to back up databases, see "Backing Up ManageOne Data" in FusionCloud 6.3.1.1 Backup and Restoration Guide. If the databases are not periodically backed up, databases cannot be restored in certain scenarios (for example, the database instances are abnormal).

Procedure

For details, see Database Faults.

Destructive Faults
Symptom

If the service deployment system cannot be used due to an irreversible operation, for example, a file is incorrectly deleted or a disk is damaged, you must reinstall DeployAgent and restore data.

Procedure
  1. Clear the environment of the faulty node. For details, see section How Do I Clear the VM Environment?
  2. Initialize the faulty node. For details, see section Initializing the Faulty Node.

Checking whether the OMMHAService has been deployed

NOTE:

When the node status becomes , check whether the OMMHAService service has been deployed on the node.

  1. Use a browser to log in to the ManageOne deployment plane.

    URL: https://Floating IP address of the deployment plane:31943, for example, https://192.168.0.1:31943.

    Default account: admin; default password: Huawei12#$

  2. Choose Resource > Server from the main menu.
  3. Click the name of the node to be restored. Then, check whether OMMHAService exists in the Environment Alias column in the Environment List.

    Use PuTTY to log in to the faulty node as the sopuser user, and run the following commands to ensure that this service can be deployed successfully:

    su - root

    Password:

    cd /opt/oss/Product/etc

    chown -R ossadm:ossgroup cipher/ ssl/

    NOTE:
    • The default password of the root user is Changeme_123.
    • Product indicates a tenant name, for example, SOP or Product_O. Replace it as required.

Deploying the Agent service

  1. Choose Resource > Server from the main menu.
  2. Click the name of the faulty node to view its details. View the Agent services that are not deployed in the Agent service list.
  3. Choose Resource > Agent from the main menu.
  4. Enter RouterAgent/regionAlias in the search box in the upper right corner of the Agent page, and press Enter.

    NOTE:

    regionAlias indicates the alias of a region. Set regionAlias to the value of regionAlias in environment planning. The region alias must contain three uppercase letters, for example, MOC.

  5. Re-associate the server containing the faulty node.

    1. Click in the Operation column of the service to be deployed. The Associate server page is displayed.
    2. Click in the Operation column of the faulty node and click Yes. The server is successfully associated.
    3. Click , select the faulty node, and click Yes. In the displayed dialog box, click Yes. The associated server is added successfully.

  1. Deploying the Agent service

    1. Choose Resource > Agent from the main menu.
    2. In the Operation column of the service to be deployed, click (Deploy/Upgrade Agent).
    3. Click OK.

  2. Deploy the Agent service on BusService/regionAlias, DBAgent/regionAlias, MOICAgent/OMGlobal/regionAlias, and BackupAgent/regionAlias by referring to 8 to 11.
  3. Choose Deployment > Database > Redis from the main menu. On the displayed page, filter instances based on the IP address of the faulty node, and record the Redis instance name. Log in to the Deploy node, and run the following commands to delete the Redis instance:

    NOTE:

    If no Redis instance exists on the faulty node, skip this step.

    su - ossadm

    The default password is ZJE%JLq5qx.

    cd /opt/oss/manager/apps/DataMgmtService/bin

    ./dbsvc_adm -cmd delete-db-instance -instid Database instance name

    On the deployment plane, choose Deployment > Feature Deployment > Services. Search for MOCustom, and click .

    NOTE:

    All services in the search result must be deployed.

Restoring the database

  1. Choose Deployment > Database > RDBMS from the main menu, and filter instances based on the IP address of the faulty node.
  2. If a single node is faulty, rectify the fault by referring to section Slave Database Instance Is Abnormal.
  3. If both the master and slave nodes are faulty, restore the data by referring to section "Restoring ManageOne Data" in FusionCloud 6.3.1.1 Backup and Restoration Guide.

    NOTE:

    If no database instance exists on the faulty node, skip this step.

Deploying the original applications that were deployed on the faulty node
NOTE:

Perform this step after you perform 2 through 16 on all faulty nodes.

  1. Use PuTTY to log in to the Deploy node as the sopuser user.

    NOTE:

    You can log in to the ManageOne deployment plane and choose Resource > Server to view the Deploy node.

    In distributed deployment mode, log in to the active Deploy node. For details, see Determine the Active and Standby Nodes of the Deployment System.

    The default password is D4I$awOD7k.

  2. Run the following command to switch to the ossadm user:

    su - ossadm

    The default password is ZJE%JLq5qx.

  3. Run the following command to export environment variables:

    . /opt/oss/manager/bin/engr_profile.sh

  4. Deploy the original applications that were deployed on the faulty node.

    1. Log in to the service deployment system and choose Resource > Server. Check and record the ID of the faulty node.
    2. Run the following commands to deploy the original applications that were deployed on the faulty node:
      NOTE:

      nodeId indicates the ID of the faulty node obtained in 20.a.

      cd /opt/oss/manager/agent/rtsp/mccommon/tools/shscript

      sh ossrunpy.sh TroubleRecoveryMgmt redeployServices nodeId

      Information similar to the following is displayed:

      [2016-12-16 11:48:01] [121856] stage_ids ['env3d85d92f-6901-4c38-8ecb-01fd8caa333b_stage_53d93f89-707d-449e-a52a-16bb66e627d1'] found for nodeId:1 
      [2016-12-16 11:48:01] [121856] Clear deploy history for nodeId:1 succeed. 
      [2016-12-16 11:48:01] [121856] Env names ['DeployTestAppService'] found for nodeId:1 
      [2016-12-16 11:48:01] [121856] stage of nodeId:1 is Gamma 
      Deploy service for env DeployTestAppService successfully. 
      [2016-12-16 11:48:03] [121856] Deploy services for nodeId:1 succeed. 
      [2016-12-16 11:48:03] [121856] Please switch to the browser and check the status and progress of the task.....

  5. Log in to the service deployment system and choose Deployment > Task to check the deployment task status. If the status is , the node has been restored.
  6. Configure microservice parameters.

    1. Configure application service parameters.

    For details, see Configuring Service Parameters on the OM Plane.

    1. Configure SFTP active/standby synchronization.

      If the regionAlias-ManageOne-Service03 or regionAlias-ManageOne-Service04 node is faulty, reconfigure the SFTP active/standby synchronization by referring to Configuring an SFTP User on the Operation Plane for Internal Transmission.

    1. Modify the sudoers configuration file.

      For details, see Modifying a VM Configuration File.

    1. Record permissions for the files and directories on the server.

      For details, see Recording Permissions for the Files and Directories on Nodes.

    1. Modify the configuration file of the Help Center.

      If MOHelpCenterWebsite is deployed on the faulty node (you can log in to the deployment system to view the node where MOHelpCenterWebsite is deployed), modify the configuration file of the help center by referring to (Optional) Modifying the Copyright Information of the Help Center and Modifying the Cloud Service Scope of the Help Center.

  7. Run the following commands to enhance the VM:

    sed -i 's/PermitRootLogin yes/PermitRootLogin no/g' /etc/ssh/sshd_config

    service sshd restart

  8. Run the following command to check whether the values of userName displayed under Product of the tenant.json and tenants.json files in the command output are the same as those in the following information:

    /opt/oss/manager/var/tenants/Product/tenant.json

    {
        "tenants": {
            "Product": {
                "fromDeployTemplate": "",
                "ossuser": {
                    "userName": "ossuser",
                    "groupName": "ossgroup",
                    "userID": "3004",
                    "groupID": "2000",
                    "homeDir": "/home/ossuser"
                },
                "productVersion": "",
                "language": "zh_CN",
                "productType": "Telco OS 1.0"
            }
        },
        "fileVersion": "1",
        "installPath": "/opt/oss"
    }

    /opt/oss/manager/etc/sysconf/tenants.json

    {
        "tenants": {
            "manager": {
                "fromDeployTemplate": "",
                "ossuser": {
                    "userName": "ossadm",
                    "groupName": "ossgroup",
                    "userID": "3001",
                    "groupID": "2000",
                    "homeDir": "/home/ossadm"
                },
                "productVersion": "",
                "language": "zh_CN",
                "productType": "Telco OS 1.0"
            },
            "Product": {
                "fromDeployTemplate": "",
                "ossuser": {
                    "userName": "ossuser",
                    "groupName": "ossgroup",
                    "userID": "3004",
                    "groupID": "2000",
                    "homeDir": "/home/ossuser"
                },
                "productVersion": "",
                "language": "zh_CN",
                "productType": "Telco OS 1.0"
            }
        },
        "fileVersion": 1,
        "installPath": "/opt/oss"
    }

  9. Verify the troubleshooting results. For details, see Verifying the Result.

Troubleshooting Service Status Exceptions After Node Restart

Symptom
  • The service is not started after the node is restarted.
  • Log in to the deployment plane fails.
Procedure
  1. Use PuTTY to log in to the faulty node using the IP address of the node.

    Default account: sopuser; default password: D4I$awOD7k

  2. Run the following command to switch to the ossadm user:

    su - ossadm

    The default password is ZJE%JLq5qx.

  3. Run the following commands to start the microservices:

    cd /opt/oss/manager/agent/bin

    . engr_profile.sh

    ./ipmc_adm -cmd startnode

Troubleshooting the System That Enters Maintenance Mode Due to Disk Damage

Symptom

If a disk is damaged during disk mounting, the system automatically enters the maintenance mode.

If the following information is displayed, the disk is damaged:

[3.116335]systemd-fsck[597]:/dev/mapper/oss_vg-optlog_vol:Inodes that were part of a corrupted orphan linked list found.
[3.118363]systemd-fsck[597]:/dev/mapper/oss_vg-optlog_vol:UNEXPECTED INCONSISTENCY;RUN fsck MANUALLY.
[3.119977]systemd-fsck[597]:/dev/mapper/oss_vg-optlog_vol:(i.e.,without -a or -p options)
NOTE:

/dev/mapper/oss_vg-optlog_vol indicates the disk name. Replace it with the actual one.

Possible Causes
  • The VM is powered off.
  • The disk array is restarted.
Procedure
  1. After the system enters the maintenance mode, the system automatically logs in as the root user.
  2. Run the following command to unmount the disk:

    umount Disk name

  3. If the following information is displayed, run the fuser -ck /opt command to stop the running processes, and then unmount the disk again:

    [root@EulerOS-BaseTemplate ~]# umount /dev/mapper/oss_vg-optlog_vol
    umount: /opt: target is busy.
    (In some cases useful info about processes that use 
     the device is found by lsof(8) or fuser(1))

    fuser -ck /opt

    NOTE:

    opt is the directory name.

  4. Run the following command to repair the disk:

    fsck.ext4 -f -y Disk name

  5. Run the following command to restart the VM:

    reboot

  6. Run the following command to log out of user root:

    exit

OS Is in Read-Only State

Symptom

The OS state changes to read-only when the command is executed.

Possible Causes
  • The VM is powered off.
  • The disk array is restarted.
  • The OS file or the disk is damaged.
  • The device storage space is full.
Procedure
  1. Restart the OS. The environment may be automatically restored.

    If the following information is displayed during the restoration, a disk attachment error occurs:
    [3.116335]systemd-fsck[597]:/dev/mapper/oss_vg-optlog_vol:Inodes that were part of a corrupted orphan linked list found.
    [3.118363]systemd-fsck[597]:/dev/mapper/oss_vg-optlog_vol:UNEXPECTED INCONSISTENCY;RUN fsck MANUALLY.
    [3.119977]systemd-fsck[597]:/dev/mapper/oss_vg-optlog_vol:(i.e.,without -a or -p options)

  2. Manually restore the OS by referring to section Troubleshooting the System That Enters Maintenance Mode Due to Disk Damage.
  3. If you fail to manually restore the OS, press Ctrl+D to forcibly log in to the OS.

Verifying the Result

Checking the Status of Each Node

Check the status of each node.

Procedure
  1. Choose Resource > Server from the main menu of the service deployment system.

    The Server page is displayed.

  2. Check the node status in the server list.

    • If Status of a node is displayed as , the node status is normal.
    • If Status of a node is displayed as , contact technical support for assistance.

Checking the Process Status

After the fault is rectified, check the running status of VM processes.

Procedure
  1. Log in to the regionAlias-ManageOne-Deploy01 node as the sopuser user in SSH mode.

    The default password is D4I$awOD7k.

  2. Run the following command to switch to the ossadm user:

    su - ossadm

    The default password is ZJE%JLq5qx.

  3. Run the following commands to check the process status:

    . /opt/oss/manager/bin/engr_profile.sh

    ipmc_adm -cmd statusapp

    The following information is displayed:

    Process Name                     Process Type                 App Name                     Tenant Name  Process Mode  IP               PID     Status
    mohawiswebsite-3-0               mohawiswebsite               MOHawisWebsite               Product      cluster       192.168.33.25    11958   RUNNING
    moresourcemgmtservice-3-0        moresourcemgmtservice        MOResourceMgmtService        Product      cluster       192.168.33.25    11901   RUNNING
    apimlb-3-0                       apimlb                       APIMLBService                Product      critical      192.168.33.25    -       STOPPED

    . /opt/oss/manager/bin/engr_profile.sh

    ipmc_adm -cmd statusbus

    The following information is displayed:

    Process Name              Process Type              App Name                  Tenant Name  IP               PID    Status  
    icagent-0-0               icagent                   ICProbeAgent              manager      10.185.191.110   10704  RUNNING 
    ir-0-0                    ir                        BusService                manager      10.185.191.110   8737   STOPPED
    dr-0-0                    dr                        BusService                manager      10.185.191.110   8486   RUNNING 
    talcapp-0-0               talcapp                   
    ......
    NOTE:
    • App Name indicates the microservice name.
    • Status indicates the process running status. RUNNING indicates that the process is running. STOPPED indicates that the process is stopped.
    Check whether the status of all processes is RUNNING.
    • If the running status is normal, repeat the preceding operations to check the process running status on the other nodes.
    • If no, contact technical support for assistance.
    NOTE:
    • For a microservice, if the value of Process Mode is single or critical and the value of Status for one of the active and standby nodes is RUNNING, this service has been restarted successfully. If the value of Process Mode is cluster or multi and the values of Status for all nodes are RUNNING, this service has been restarted successfully.
    • In other scenarios, the service fails to be restarted. In this case, contact technical support for assistance.

Checking the Database Status

This section describes how to check the running status of each database.

Procedure
  1. Choose Deployment > Database > RDBMS from the main menu of the service deployment system.

    The RDBMS page is displayed.

  2. Check the statuses of all database instances.

    • The database instance status is normal if the relationships between Instance Type and Status are as follows:
      • When Instance Type is primary, Status is displayed as .
      • When Instance Type is single, Status is displayed as --.
    • If the database instance status is , restore ManageOne data by referring to "Restoring ManageOne Data" in FusionCloud 6.3.1.1 Backup and Restoration Guide.

Verifying the Troubleshooting Result

After the node faults are rectified, check whether the ManageOne web page can be logged in to.

Procedure
  1. Open a web browser, enter http://Address for accessing the homepage of the ManageOne OM plane:31943 in the address box, and press Enter.

    Address for accessing the homepage of the ManageOne OM plane is the domain name of the following nodes:

    regionAlias-ManageOne-Service01 and regionAlias-ManageOne-Service02 nodes on the OM plane
    NOTE:

    regionAlias-ManageOne-Service01 and regionAlias-ManageOne-Service02 are the O&M service nodes. These two nodes share the same domain name. Replace regionAlias with the region alias planned for the live network.

  2. Enter the username and password, and click Log In.

    NOTE:

    You need to change the initial password as prompted upon first login.

    If you log in using the domain name of the OM plane nodes, enter the username admin.

    If the ManageOne welcome page is displayed, the node fault is rectified.

Checking the Driver Status

After the node fault is rectified, check whether the driver is running properly.

Procedure
  1. Log in to the OM plane as the admin user by using a web browser.

    Open a web browser, enter https://Address for accessing the homepage of the ManageOne OM plane:31943 in the address box, and press Enter.
    NOTE:
    • The address for accessing the homepage of the ManageOne OM plane is the domain name of the O&M service nodes.
    • regionAlias-ManageOne-Service01 and regionAlias-ManageOne-Service02 are the O&M service nodes. These two nodes share the same domain name. Replace regionAlias with the region alias planned for the live network.

  2. Choose System > Platform Configuration > System Access from the main menu. In the navigation pane, choose Driver Life Cycle Management > Driver Management.

    Click the Driver Instances tab.

  3. Check whether the status of each driver is normal.

    • If the Instance Status is RUNNING, the driver status is normal.
    • Otherwise, the driver status is abnormal.

Translation
Download
Updated: 2019-06-10

Document ID: EDOC1100063248

Views: 22904

Downloads: 37

Average rating:
This Document Applies to these Products
Related Documents
Related Version
Share
Previous Next