No relevant resource is found in the selected language.

This site uses cookies. By continuing to browse the site you are agreeing to our use of cookies. Read our privacy policy>Search

Reminder

To have a better experience, please upgrade your IE browser.

upgrade

HUAWEI CLOUD Stack 6.5.0 Alarm and Event Reference 04

Rate and give feedback:
Huawei uses machine translation combined with human proofreading to translate this document to different languages in order to help you better understand the content of this document. Note: Even the most advanced machine translation cannot match the quality of professional translators. Huawei shall not bear any responsibility for translation accuracy and it is recommended that you refer to the English document (a link for which has been provided).
Troubleshooting Guide for Typical Cinder Problems

Troubleshooting Guide for Typical Cinder Problems

Checking FusionSphere OpenStack

NOTE:

In the Region Type I scenario, this section describes how to check cascading FusionSphere OpenStack.

In the Region Type I scenario, Cinder is classified into cascading and cascaded FusionSphere OpenStack systems. If a fault occurs, you need to troubleshoot the fault in both the cascading and cascaded FusionSphere OpenStack systems. Specifically, you can search error logs and handle the fault based on the obtained information.

In Region Type II and Region Type III scenarios, this section describes how to check FusionSphere OpenStack.

  • If Cinder has a fault, you can locate the fault based on the error log.
  • If the environment configuration is incorrect, use the handling method listed in the reference document.
  • If an interconnected module has a fault, send the error log to the O&M personnel for fault locating.

This document focuses on troubleshooting cinder faults.

Querying Volume Information

  1. In the Region Type I scenario, use PuTTY to log in to the cascading FusionSphere OpenStack node using the IP address corresponding to the Cascading-Reverse-Proxy field (searching in the parameter summary file xxx_export_all_EN.xlsm).

    In the Region Type II or Region Type III scenario, use PuTTY to log in to the FusionSphere OpenStack node using the IP address corresponding to the Reverse-Proxy field in the parameter summary file xxx_export_all_EN.xlsm.

    The default username is fsp and the default password is Huawei@CLOUD8.

  2. Run the su root command and enter the password of user root to switch to user root.

    The default password of user root is Huawei@CLOUD8!.

  3. Run the following command to import the environment variables:

    source set_env

    During the import, enter 1, select openstack environment variable (keystone v3). The default password is FusionSphere123.

  4. Run the following command to query the volume details:

    cinder show volume_id
    • If Service Unavailable (HTTP 503) is returned, as shown in Figure 6-1, the cinder-api service is not started. If this happens, perform the following procedure:
      1. Run the following command to start the cinder-api service, as shown in Figure 6-2. The startup takes several seconds.

        cps host-template-instance-operate --action start --service cinder cinder-api

      2. Run the following command to check the service status:

        cps template-instance-list --service cinder cinder-api

      3. If the service is still not started, query /var/log/fusionsphere/component/cinder-api/cinder-api_*.log and /var/log/fusionsphere/component/cinder-apiControl/cinder-apiControl_*.log to locate the fault.
      4. After the service becomes normal, go to 4.
        Figure 6-1 Service Unavailable
        Figure 6-2 Starting the cinder-api service
    • If volume details are displayed, the volume is not deleted.

      Run the following command to query the detailed information about the host, as shown in Figure 6-3.

      cinder show volume_id | zgrep host

      Figure 6-3 Host information
    • If information similar to Figure 6-4 is displayed, the volume has been deleted. Perform the following steps to view host details in the database:
      Figure 6-4 Command output if the volume is deleted
      1. Run the following command to query the IP address of the node where the database is located, that is, the IP address corresponding to omip, as shown in Figure 6-5:

        cps template-instance-list --service gaussdb gaussdb

        Figure 6-5 Database information
      2. Use PuTTY to log in to the active database node using the IP address queried in 4.a. The IP address is in the active state.

        The default username is fsp and the default password is Huawei@CLOUD8.

      3. Run the su root command and enter the password of user root to switch to user root.

        The default password of user root is Huawei@CLOUD8!.

      4. Run the following command to import the environment variables:

        source set_env

        During the import, enter 1 to select environment variable openstack environment variable (keystone v3).

      5. Run the following command to log in to the database node:

        su gaussdba

      6. Run the following command to enter the database:

        gsql -d cinder

        The default password of the database is FusionSphere123.

      7. Run the following command to query the host information of the volume_id, as shown in Figure 6-6.

        select host from volumes where id = 'volume_id';

        Figure 6-6 Host information of the volume

    You can obtain the host information by performing the preceding operations.

Querying cinder-api Logs

  1. In the Region Type I scenario, use PuTTY to log in to the cascading FusionSphere OpenStack node using the IP address corresponding to the Cascading-Reverse-Proxy field (searching in the deployment parameter summary file xxx_export_all_EN.xlsm).

    In the Region Type II or Region Type III scenario, use PuTTY to log in to the FusionSphere OpenStack node using the IP address corresponding to the Reverse-Proxy field in the deployment parameter summary file xxx_export_all_EN.xlsm.

    The default username is fsp and the default password is Huawei@CLOUD8.

  2. Run the su root command and enter the password of user root to switch to user root.

    The default password of user root is Huawei@CLOUD8!.

  3. Run the following command to import the environment variables:

    source set_env

    During the import, enter 1, select openstack environment variable (keystone v3). The default password is FusionSphere123.

  4. Run the following command to query the IP addresses of cinder-api control nodes, that is, all IP addresses corresponding to omip, as shown in Figure 6-7.

    cps template-instance-list --service cinder cinder-api
    Figure 6-7 Controller node information

  5. Use the IP addresses queried in 4 to log in to all cinder-api control nodes one by one.

    The default username is fsp and the default password is Huawei@CLOUD8.

    NOTE:
    • The controller nodes are deployed in distributed mode. Therefore, you need to query each controller node.
    • cinder-api logs are generated on each controller node, and the requests issued from the upper layer may be processed on any controller node.

  6. Run the su root command and enter the password of user root to switch to user root.

    The default password of user root is Huawei@CLOUD8!.

  7. Perform the following steps to search for request id based on volume_id and locate the node that processes the request:

    NOTE:
    • The request is randomly assigned to a controller node for processing. Therefore, you need to obtain request id based on volume_id on each controller node.
    • request id is in "req-xxx" format.
    1. Run the following command to query the cinder-api directory, as shown in Figure 6-8.

      ll /var/log/fusionsphere/component/cinder-api

      Figure 6-8 cinder-api directory
    2. Run the following command, search for the log file based on the volume_id and find the request id. Based on the time when the error occurs, match the found log information to obtain request id, as shown in Figure 6-9.

      zgrep volume_id /var/log/fusionsphere/component/cinder-api/*

      Figure 6-9 Searching for request id
    3. Run the following command to query the log based on the obtained request_id. Query the error information.

      zgrep req_id /var/log/fusionsphere/component/cinder-api/*

      • If you locate the error information, rectify the fault accordingly.
      • If no error information is located, perform either of the following operations:
      Figure 6-10 Error information

Querying cinder-scheduler Log

  1. In the Region Type I scenario, use PuTTY to log in to the cascading FusionSphere OpenStack node using the IP address corresponding to the Cascading-Reverse-Proxy field (searching in the deployment parameter summary file xxx_export_all_EN.xlsm).

    In the Region Type II or Region Type III scenario, use PuTTY to log in to the FusionSphere OpenStack node using the IP address corresponding to the Reverse-Proxy field in the deployment parameter summary file xxx_export_all_EN.xlsm.

    The default username is fsp and the default password is Huawei@CLOUD8.

  2. Run the su root command and enter the password of user root to switch to user root.

    The default password of user root is Huawei@CLOUD8!.

  3. Run the following command to import the environment variables:

    source set_env

    During the import, enter 1, select openstack environment variable (keystone v3). The default password is FusionSphere123.

  4. Run the following command to query all cinder-scheduler controller nodes:

    cps template-instance-list --service cinder cinder-scheduler

  5. Use the IP addresses in 4 to log in to all cinder-scheduler control nodes one by one.

    The default username is fsp and the default password is Huawei@CLOUD8.

  6. Run the su root command and enter the password of user root to switch to user root.

    The default password of user root is Huawei@CLOUD8!.

  7. Run the following command on all cinder-scheduler control nodes to search for the cinder-scheduler log based on the request id obtained in 7. Locate ERROR based on the error occurrence time and view the error information.

    zgrep req_id /var/log/fusionsphere/component/cinder-scheduler/ *

  8. Locate the fault cause based on the error information and handle the fault.

Querying cinder-volume Logs

If the backend storage corresponding to the volume is found in the log, that is, the host field is not None, the cinder-volume component reports an error.

Procedure

  1. In the Region Type I scenario, use PuTTY to log in to the cascading FusionSphere OpenStack node using the IP address corresponding to the Cascading-Reverse-Proxy field (searching in the parameter summary file xxx_export_all_EN.xlsm).

    In the Region Type II or Region Type III scenario, use PuTTY to log in to the FusionSphere OpenStack node using the IP address corresponding to the Reverse-Proxy field (searching in the parameter summary file xxx_export_all_EN.xlsm).

    The default username is fsp and the default password is Huawei@CLOUD8.

  2. Run the su root command and enter the password of user root to switch to user root.

    The default password of user root is Huawei@CLOUD8!.

  3. If the volume is not deleted, run the following command to query the host information in the volume details and determine the back-end storage type, as shown in Figure 6-11.

    cinder show volume_id | zgrep host

    Figure 6-11 Host information in the volume details

  4. If the volume has been deleted, query the host information in the database to determine the backend storage type.

    1. Run the following command to query the IP address of the node where the database is located, that is, the IP address corresponding to omip, as shown in Figure 6-12.

      cps template-instance-list --service gaussdb gaussdb

      Figure 6-12 Database information
    2. Use PuTTY to log in to the active database node using the IP address queried in 4.a. The IP address is in the active state.

      The default username is fsp and the default password is Huawei@CLOUD8.

    3. Run the su root command and enter the password of user root to switch to user root.

      The default password of user root is Huawei@CLOUD8!.

    4. Run the following command to import the environment variables:

      source set_env

      During the import, enter 1 to select environment variable openstack environment variable (keystone v3).

    5. Run the following command to log in to the database node:

      su gaussdba

    6. Run the following command to enter the database:

      gsql -d cinder

      The default password of the database is FusionSphere123.

    7. Run the following command to query the host information of the volume_id, as shown in Figure 6-13.

      select host from volumes where id = 'volume_id';

      Figure 6-13 Host information of volume_id

  5. Query the IP addresses of all cinder-volume control nodes based on the host field information, that is, the IP addresses corresponding to omip, as shown in Figure 6-14.

    cps template-instance-list --service cinder cinder-volume-xxx

    If the backend storage queried in 4.g is cinder-vrm001, the value of cinder-volume-xxx is cinder-volume-vrm001. If cinder-kvm001 is found, the value of cinder-volume-xxx is cinder-volume-kvm001.

    Figure 6-14 Backend storage node

  6. Use the IP addresses queried in 5 to log in to all cinder-volume control nodes one by one.

    The default username is fsp and the default password is Huawei@CLOUD8.

  7. Run the su root command and enter the password of user root to switch to user root.

    The default password of user root is Huawei@CLOUD8!.

  8. Run the following command to import the environment variables:

    source set_env

    During the import, enter 1, select openstack environment variable (keystone v3). The default password is FusionSphere123.

  9. Run the following command to query the cinder-volume-xxx log of each node:

    zgrep req_id /var/log/fusionsphere/component/cinder-volume-xxx/*

    req_id is the value of request id queried in 7.

    cinder-volume-xxx is the same as the configuration in 5.

  10. Locate and handle the fault based on the log.

Troubleshooting Faults in the Cascaded FusionSphere OpenStack System

Querying cinder-api Logs of the Cascaded FusionSphere OpenStack Systems

  1. Use PuTTY to log in to the cascaded FusionSphere OpenStack node using the IP address corresponding to the Cascaded-Reverse-Proxy field (searching in the parameter summary file xxx_export_all_EN.xlsm).

    The default username is fsp and the default password is Huawei@CLOUD8.

  2. Run the su root command and enter the password of user root to switch to user root.

    The default password of user root is Huawei@CLOUD8!.

  3. Run the following command to import the environment variables:

    source set_env

    During the import, enter 1, select openstack environment variable (keystone v3). The default password is FusionSphere123.

  4. Run the following command to query the cascaded OpenStack volume details based on the cascading OpenStack volume_id.

    Generally, the cascading OpenStack volume_id and the cascaded OpenStack volume_id are the same.

    cinder show volume_id of the cascading FusionSphere OpenStack system

    volume_id of the cascading FusionSphere OpenStack system is the disk ID displayed on the ManageOne tenant portal.
    • If the volume details are displayed, run the following command to query the host information:

      cinder show volume_id of the cascading FusionSphere OpenStack system | zgrep host

    • If the volume details are not displayed, volume_id of the cascaded FusionSphere OpenStack systems is different from volume_id of the cascading FusionSphere OpenStack system, or the volume is deleted. In this case, you need to query the database.
      1. Run the following command to query the IP address of the node where the database is located, that is, the IP address corresponding to omip, as shown in Figure 6-16.

        cps template-instance-list --service gaussdb gaussdb

        Figure 6-16 Database information
      2. Use PuTTY to log in to the active database node using the IP address queried in 4.a. The IP address is in the active state.

        The default username is fsp and the default password is Huawei@CLOUD8.

      3. Run the su root command and enter the password of user root to switch to user root.

        The default password of user root is Huawei@CLOUD8!.

      4. Run the following command to import the environment variables:

        source set_env

        During the import, enter 1 to select environment variable openstack environment variable (keystone v3).

      5. Run the following command to log in to the database node:

        su gaussdba

      6. Run the following command to enter the database:

        gsql -d cinder

        The default password of the database is FusionSphere123.

      7. Run the following command to query the volume id of the cascaded FusionSphere OpenStack system, as shown in Figure 6-17:

        select id from volumes where display_name = 'volume@volume id of the cascaded FusionSphere OpenStack system';

        Figure 6-17 Querying the database
      8. Run the following command to query the host information of the volume of the cascaded FusionSphere OpenStack system:

        select host from volumes where id = 'volume id of the cascaded FusionSphere OpenStack system';

  5. Run the following command to query the IP addresses of all cinder-api controller nodes of the cascaded OpenStack system, that is, the IP addresses corresponding to omip:

    cps template-instance-list --service cinder cinder-api

  6. Use the IP addresses queried in 5 to log in to all cinder-api control nodes one by one.

    The default username is fsp and the default password is Huawei@CLOUD8.

  7. Run the su root command and enter the password of user root to switch to user root.

    The default password of user root is Huawei@CLOUD8!.

  8. Run the following command to query cinder-api logs of each cascaded node and obtain the request ID which is in "req-xxx" format:

    zgrep volume_id /var/log/fusionsphere/component/cinder-api/ *

    volume_id is the volume id of the cascaded OpenStack system queried in 4.g.

  9. Run the following command to query the cinder-api logs of the node based on request id and obtain the error information:

    zgrep rep-id /var/log/fusionsphere/component/cinder-api/*

    req_id is the value of request id queried in 7.

    If the fault is not located, go to Querying cinder-scheduler Logs of the Cascaded FusionSphere OpenStack Systems.

Querying cinder-scheduler Logs of the Cascaded FusionSphere OpenStack Systems

When creating a blank volume or creating a volume from an image, you need to query the cinder-scheduler log of the cascaded OpenStack system. This operation is not required by creating a volume from backup or snapshot.

Procedure

  1. Use PuTTY to log in to the cascaded FusionSphere OpenStack node using the IP address corresponding to the Cascaded-Reverse-Proxy field (searching in the parameter summary file xxx_export_all_EN.xlsm).

    The default username is fsp and the default password is Huawei@CLOUD8.

  2. Run the su root command and enter the password of user root to switch to user root.

    The default password of user root is Huawei@CLOUD8!.

  3. Run the following command to import the environment variables:

    source set_env

    During the import, enter 1, select openstack environment variable (keystone v3). The default password is FusionSphere123.

  4. Run the following command to query all cinder-scheduler controller nodes:

    cps template-instance-list --service cinder cinder-scheduler

  5. Use the IP addresses obtained in 4 to log in to all cinder-scheduler control nodes one by one.

    The default username is fsp and the default password is Huawei@CLOUD8.

  6. Run the su root command and enter the password of user root to switch to user root.

    The default password of user root is Huawei@CLOUD8!.

  7. Run the following command to query the logs of each node based on request id:

    zgrep req_id /var/log/fusionsphere/component/cinder-scheduler/*

    req_id is the value of request id queried in 7.

  8. Locate and handle the fault based on the error information in the log.

Querying cinder-volume Logs of the Cascaded FusionSphere OpenStack Systems

If the backend storage corresponding to the volume is found in the log, that is, the host field is not None, the cinder-volume component reports an error.

Procedure

  1. Use PuTTY to log in to the cascaded FusionSphere OpenStack node using the IP address corresponding to the Cascaded-Reverse-Proxy field (searching in the parameter summary file xxx_export_all_EN.xlsm).

    The default username is fsp and the default password is Huawei@CLOUD8.

  2. Run the su root command and enter the password of user root to switch to user root.

    The default password of user root is Huawei@CLOUD8!.

  3. Run the following command to import the environment variables:

    source set_env

    During the import, enter 1, select openstack environment variable (keystone v3). The default password is FusionSphere123.

  4. Query the cinder-volume log of the cascaded OpenStack system to locate the fault by referring to Querying cinder-volume Logs.
Translation
Download
Updated: 2019-08-30

Document ID: EDOC1100062365

Views: 48464

Downloads: 33

Average rating:
This Document Applies to these Products
Related Version
Related Documents
Share
Previous Next