No relevant resource is found in the selected language.

This site uses cookies. By continuing to browse the site you are agreeing to our use of cookies. Read our privacy policy>Search

Reminder

To have a better experience, please upgrade your IE browser.

upgrade

HUAWEI CLOUD Stack 6.5.0 Troubleshooting Guide 02

Rate and give feedback:
Huawei uses machine translation combined with human proofreading to translate this document to different languages in order to help you better understand the content of this document. Note: Even the most advanced machine translation cannot match the quality of professional translators. Huawei shall not bear any responsibility for translation accuracy and it is recommended that you refer to the English document (a link for which has been provided).
Addon Node Exceptions in the Data Zone

Addon Node Exceptions in the Data Zone

This section describes how to restore the addon node.

Symptom

The addon node is abnormal and cannot be restored even after a forcible restart.

Troubleshooting

Prerequisites
  • Clusters are running properly in the FusionStage environment. For details about how to rebuild an addon node, see related documentations of the IaaS layer.
  • The rebuilt addon node uses the same IP address as the original node.
  • The rebuilt addon node has the same specifications as the original addon node.
  • The passwords of all nodes in the management zone are the same.
Deleting a Faulty addon Node
Procedure
  1. Use PuTTY to log in to the manage_lb1_ip node.

    The default username is paas, and the default password is QAZ2wsx@123!.

  2. Run the following command and enter the password of the root user to switch to the root user:

    su - root

    Default password: QAZ2wsx@123!

  3. Delete the faulty node.

    1. Run the following command to query the name of the faulty node from the NAME column:

      kubectl get no

    2. Run the following command to export information about the faulty node:

      kubectl get no addon-dc03026e -oyaml > /tmp/addon.yaml

      • addon-dc03026e indicates the name of the faulty node obtained in a.Run the following comm....
      • addon.yaml indicates the file to which information about the faulty node is exported. The file name is user-defined. It is recommended that the file name should contain the first half of the name of the faulty node.
    3. Run the following command to edit the information of the faulty node:

      kubectl edit node <Name of the faulty node> -n kube-system

      Name of the faulty node indicates the node name obtained in 3.a.

      Add forcedelete: "true" in the annotations field, as shown in the following figure.

      Press Esc to switch to the command mode and run the :wq! command to save the file and exit.

    4. Run the following command to delete the faulty node:

      kubectl delete no <Name of the faulty node> --grace-period=0 --force

Installing an addon Node
Procedure
  1. Use PuTTY to log in to the rebuilt addon node.

    The default username is paas, and the default password is QAZ2wsx@123!.

  2. Run the following command to switch to the root user:

    su - root
    paasword: password of the root user

  3. Check whether there is a log file (that contains the information about mounting disks to the node) whose name ends with .flag or create_vol.log in the /home/paas/create_vol_tool/, /var/log/tools/create_vol/ or /tmp directory. If yes, delete the file. If no, switch to the paas user and mount the disk to the rebuilt node again. For details about how to mount disks, see section "Preparations Before Installation > Configuring Disk Partitions" in FusionStage 6.5.0.SPC100 Product Documentation.
  4. Manage the rebuilt node.

    1. Use PuTTY to log in to the manage_lb1_ip node.

      The default username is paas, and the default password is QAZ2wsx@123!.

    2. Run the following command to modify the configuration file of the rebuilt node:

      vi /tmp/addon.yaml

      • addon.yaml indicates the node configuration file generated in b.Run the following comm....
      • Delete the creationTimestamp, resourceVersion, selfLink, uid, and status fields.
      NOTE:

      Change the value of unschedulable from true to false only if unschedulable: true exists.

      Save the change and exit.

    3. Run the following command and enter the password of the root user to switch to the root user:

      su - root

      Default password: QAZ2wsx@123!

    4. Run the following command to manage the rebuilt node:

      kubectl create -f /tmp/addon.yaml

  5. Check the status of the rebuilt node.

    1. Run the following command to view the status of the rebuilt node:

      kubectl get no

      The node status can be Unknown, NotReady, or Ready.

    2. Log in to the rebuilt node as the paas user. Run the following command to view service status of the node:

      monit summary

      • If all services on the rebuilt node are in the Running state, the rebuilt node has been successfully managed.
      • If any service is always faulty, run the monit restart {Service Name} command to restart the service.

        {Service Name} indicates the name of the faulty service.

  6. Run the following command and enter the password of the root user to switch to the root user:

    su - root

    Default password: QAZ2wsx@123!

  7. Run the following command to check pod status of the rebuilt node:

    kubectl get pods -n kube-system -o wide | grep <Name of the rebuilt node>

    If all pods on the rebuilt node are in the Running state, the node has successfully restored.

    NOTE:
    • If any pod is abnormal or any instance is not in the Running state, delete the pod for the system to restart another pod.
    • To change the host name of the rebuilt node, perform the following steps:
      1. Log in to the node as the paas user, switch to the root user, and run the following command to change the host name temporarily:

        hostname {New host name}

      2. Run the following command to modify the configuration file for the new host name to take effect permanently:

        echo '{New host name}' > /etc/hostname

      3. Log in to the rebuilt node as the paas user for the modification to take effect.

Translation
Download
Updated: 2019-06-01

Document ID: EDOC1100062375

Views: 1901

Downloads: 12

Average rating:
This Document Applies to these Products
Related Documents
Related Version
Share
Previous Next