No relevant resource is found in the selected language.

This site uses cookies. By continuing to browse the site you are agreeing to our use of cookies. Read our privacy policy>Search

Reminder

To have a better experience, please upgrade your IE browser.

upgrade

HUAWEI CLOUD Stack 6.5.0 Troubleshooting Guide 02

Rate and give feedback :
Huawei uses machine translation combined with human proofreading to translate this document to different languages in order to help you better understand the content of this document. Note: Even the most advanced machine translation cannot match the quality of professional translators. Huawei shall not bear any responsibility for translation accuracy and it is recommended that you refer to the English document (a link for which has been provided).
Inconsistency Between Docker Cache and Disk Data Due to Insufficient Docker Disk Space, Causing the Container Restarting Failure

Inconsistency Between Docker Cache and Disk Data Due to Insufficient Docker Disk Space, Causing the Container Restarting Failure

Symptom

The Docker disk space is insufficient. The docker ps command output shows that an etcd container is normal but the etcd container cannot be accessed.

Possible Causes

The Docker cache is inconsistent with the disk data.

Troubleshooting Method

  1. Use PuTTY to log in to the manage_db1_ip node.

    The default username is paas, and the default password is QAZ2wsx@123!.

  2. Run the following command and enter the password of the root user to switch to the root user:

    su - root

    Default password: QAZ2wsx@123!

  3. Run the following command to check whether the disk space of /opt and /opt/paas/run is sufficient:

    df -H | head -n 20

    /dev/mapper/opt_vg-vol_opt              64G  62G   0  97%  /opt
    /dev/mapper/etcd_vg-vol_opt_paas_run    53G  51G   0  96%  /opt/paas/run

    Check whether the disk usage reaches 90% or higher.

    • If yes, the disk space is insufficient. Go to Step 4.
    • If no, the disk usage is low. Go to Step 8.

  4. Check whether an etcd container fails to be started due to the inconsistency between the Docker cache and disk data.

    1. Run the following command to query IDs of etcd containers:

      docker ps | grep etcd

      As shown in the preceding figure, the content in the red boxes indicates the IDs of etcd containers.

    1. Run the following command to check the container status:

      docker ps | grep 74dde2f135ba

      In the preceding command, 74dde2f135ba indicates the ID of a container queried in 1.

      As shown in the red box in the preceding figure, the etcd container is in the Up state, indicating that the etcd container is normal.

    2. Run the following command to check the actual status of the container:

      docker inspect 74dde2f135ba | grep Status

      In the preceding command, 74dde2f135ba indicates the ID of a container queried in 1.

      As shown in the preceding figure, the actual status of the etcd container is exited, indicating that the etcd container is in the exit state, which is different from the status queried in 2.

    NOTE:

    If the disk space is insufficient, run the docker ps command to check the container status. If the command output shows that the container is still running properly, run the docker inspect command to check the actual status of the container.

  5. Locate the cause of the insufficient disk space. Clear the disk space or expand the disk capacity to ensure that the disk space is sufficient.
  6. Run the following command to restart the Docker service:

    systemctl restart docker

  7. Run the following command to check the actual status of the container again:

    docker inspect 74dde2f135ba | grep Status

    In the preceding command, 74dde2f135ba indicates the ID of a container queried in 1 in Step 4.

    As shown in the red box in the preceding figure, the actual status of the etcd container is running, indicating that the etcd container is normal.

  8. If the problem persists, contact technical support for assistance.

    NOTE:

    Use PuTTY to log in to the manage_db02 and manage_db03 nodes where the etcd cluster is located in sequence, and repeat Step 1 to Step 8.

Translation
Download
Updated: 2019-06-01

Document ID: EDOC1100062375

Views: 1274

Downloads: 12

Average rating:
This Document Applies to these Products
Related Documents
Related Version
Share
Previous Next