No relevant resource is found in the selected language.

This site uses cookies. By continuing to browse the site you are agreeing to our use of cookies. Read our privacy policy>Search

Reminder

To have a better experience, please upgrade your IE browser.

upgrade

HUAWEI CLOUD Stack 6.5.0 Troubleshooting Guide 02

Rate and give feedback:
Huawei uses machine translation combined with human proofreading to translate this document to different languages in order to help you better understand the content of this document. Note: Even the most advanced machine translation cannot match the quality of professional translators. Huawei shall not bear any responsibility for translation accuracy and it is recommended that you refer to the English document (a link for which has been provided).
Pod Scheduling Failures

Pod Scheduling Failures

Symptom

The following are the main symptoms of pod scheduling failures:

  • A message indicating that a pod is in the Pending state is displayed in the kubectl describe pod field.
  • The values of IP and Node of the pod in the kubectl describe pod field are both empty.

Possible Causes

  • The nodeSelector field of the pod does not match the label of the node.
  • The node has already had an anti-affinity pod.
  • The disk space of the node is fully occupied.

Troubleshooting Method

  1. Check whether the nodeSelector field of the pod matches the label of the node.

    1. Use PuTTY to log in to the manage_lb1_ip node.

      The default username is paas, and the default password is QAZ2wsx@123!.

    2. Run the following command and enter the password of the root user to switch to the root user:

      su - root

      Default password: QAZ2wsx@123!

    3. Run the following command to query the nodeSelector field of the pod:

      kubectl -n fst-manage get pod {PodName} -o yaml | grep -C 2 "nodeSelector"

      In the preceding command, {PodName} is the name of pod scheduling failures.

    1. Run the following command (containing the -l parameter and values of Key and Value of the label. Multiple key–value pairs can be separated by semicolons (;)) to check the node that meets the nodeSelector field configuration:

      kubectl get node -l {nodeSelector} -n {namespace}

      The value of {nodeSelector} is that of nodeSelector of the pod obtained in 1.c, for example, kubectl get node -l fst-manage.amsaccess=amsaccess -n fst-manage, and {namespace} is the namespace of the pod.

    If there are not enough nodes to meet the nodeSelector field of the pod, check the nodes to be labeled and according to the following operation to label the node.

    1. Log in to FusionStage.

      1. Use a browser to log in to ManageOne Operation Portal (ManageOne Tenant Portal in B2B scenarios) as a VDC administrator or VDC operator.
        Login address:
        • Login address in a non-B2B scenario: https://Address for accessing ManageOne Operation Portal, for example, https://console.demo.com.
        • Login address in the B2B scenario: https://Address for accessing ManageOne Tenant Portal, for example, https://tenant.demo.com
      2. Select your region and then project from the drop-down list on the top menu bar.
      3. Choose Console > Application > FusionStage from the main menu.

    2. Choose Project Management > Resource > Node from the main menu.
    3. In the node list, select the node to which you want to attach labels and click Manage Label. On the Manage Label page, click Add, set Key and Value, and click Save.

  2. Run the following command and enter the password of the root user to switch to the root user:

    su - root

    Default password: QAZ2wsx@123!

  3. Check whether an anti-affinity pod has been deployed on the node.

    Run the following command to check the definition of the anti-affinity pod:

    kubectl -n {namespace} get pod {PodName} -o yaml

    {namespace} indicates the corresponding namespace, and {PodName} indicates the name of the anti-affinity pod.

    ...
    spec:
      affinity:
        podAntiAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
          - labelSelector:
              matchExpressions:
              - key: podaffinity
                operator: In
                values:
                - cse-etcd-backup-pod
            numOfMatchingPods: "1"
            topologyKey: kubernetes.io/hostname
      containers:
      ...

    Run the following command to check whether the anti-affinity pod has been deployed on the node:

    kubectl -n {namespace} get pod -owide | grep {PodName}

    {namespace} indicates the corresponding namespace, and {PodName} indicates the name of the anti-affinity pod.

  4. Check whether the disk space of the node is used up.

    Run the df -h /opt command on the node where pods are to be deployed as the root user to check whether the disk usage reaches 100%.

    If the usage does not reach 100%, the disk space is not used up. Check whether the pod scheduling failure is caused by the other two causes mentioned in Possible Causes. If the usage reaches 100%, contact technical support for assistance.

  5. If the fault persists, contact technical support for assistance.
Translation
Download
Updated: 2019-06-01

Document ID: EDOC1100062375

Views: 1969

Downloads: 12

Average rating:
This Document Applies to these Products
Related Documents
Related Version
Share
Previous Next