No relevant resource is found in the selected language.

This site uses cookies. By continuing to browse the site you are agreeing to our use of cookies. Read our privacy policy>Search

Reminder

To have a better experience, please upgrade your IE browser.

upgrade

HUAWEI CLOUD Stack 6.5.0 Alarm and Event Reference 04

Rate and give feedback:
Huawei uses machine translation combined with human proofreading to translate this document to different languages in order to help you better understand the content of this document. Note: Even the most advanced machine translation cannot match the quality of professional translators. Huawei shall not bear any responsibility for translation accuracy and it is recommended that you refer to the English document (a link for which has been provided).
ALM-2 Failed to Create Pod

ALM-2 Failed to Create Pod

Description

Pods are needed for application deployment. This alarm is reported when containers (containerized applications) or processes (process applications) fail to be created in a pod.

Pod: In Kubernetes, pods are the smallest unit of creation, scheduling, and deployment. A pod is a group of relatively tightly coupled containers. Pods are always co-located and run in a shared application context. Containers within a pod share a namespace, IP address, port space, and volume.

Attribute

Alarm ID

Alarm Severity

Alarm Type

2

Major

Environmental alarm

Parameters

Parameter Name

Parameter Description

kind

Resource type.

namespace

Name of the project to which the resource belongs.

name

Resource name.

uid

Unique ID of the resource.

OriginalEventTime

Event generation time.

EventSource

Name of the component that reports an event.

EventMessage

Supplementary information about an event.

Impact on the System

Related functions are unavailable.

System Actions

The system keeps creating a pod or job.

Possible Causes

  • Containerized application: The Docker service on the node where the pod runs is abnormal. As a result, the containers cannot be created.
  • Process application: The lifecycle configured for the application is incorrect. As a result, the creation fails.

Procedure

  1. Obtain the name of the pod that fails to be created.

    1. Use a browser to log in to the FusionStage OM zone console.
      1. Log in to ManageOne Maintenance Portal.
        • Login address: https://Address for accessing the homepage of ManageOne Maintenance Portal:31943, for example, https://oc.type.com:31943.
        • The default username is admin, and the default password is Huawei12#$.
      2. On the O&M Maps page, click the FusionStage link under Quick Links to go to the FusionStage OM zone console.
    2. Choose Application Operations > Application Operations from the main menu.
    3. In the navigation pane on the left, choose Alarm Center > Alarm List and query the alarm by setting query criteria.
    4. Click to expand the alarm information. Record the values of name and namespace in Location Info, that is, podname and namespace.

  2. Use PuTTY to log in to the manage_lb1_ip node.

    The default username is paas, and the default password is QAZ2wsx@123!.

  3. Run the following command and enter the password of the root user to switch to the root user:

    su - root

    Default password: QAZ2wsx@123!

  4. Run the following command to obtain the IP address of the node on which the pod runs:

    kubectl get pod podname -n namespace -oyaml | grep -i hostExternalIP

    In the preceding command, podname is the instance name obtained in 1, and namespace is the namespace obtained in 1.

  5. Check whether a containerized application is running in the pod.

    Run the following command to obtain the pod template:

    kubectl get pod podname -n namespace -oyaml

    hostname:~/ # kubectl get pod nginx-ctn-1267811602-zmfz3 -n testns -oyaml
    ...
    spec:
      containers:
      - image: */nginx:latest
    ...
    If the pod template contains the following information, a containerized application is running in the pod. Otherwise, a process application is running in the pod.
    • If yes, go to 6.
    • If no, go to 7.

  6. If a containerized application is running in the pod, perform the following operations:

    1. Log in to the node obtained in 4 as the paas user.
    2. Run the following command to switch to the root user:

      su - root

    3. Run the following command to check whether the docker service is normal. The following command output indicates that the docker service is abnormal:
      docker ps
      hostname:~/ # docker ps
      Cannot connect to the Docker daemon. Is the docker daemon running on this host?

      If yes, go to 6.d.

      If no, go to 8.

    4. Run the following command to restart the docker service (evaluate whether the restart affects running services).

      service docker restart

    5. Run the command in 6.c again. After the docker service is restarted, check whether the alarm is cleared.
      • If yes, no further action is required.
      • If no, go to 8.

  7. If a process application is running in the pod, perform the following operations:

    1. Log in to the node obtained in 4 as the paas user.
    2. Run the following commands to view the kubelet log:

      cd /var/paas/sys/log/kubernetes/

      vi kubelet.log

      Press the / key, enter the name of the pod, and then press Enter for search. If Failed or Error is displayed in the command output or if the log line starts with E, the error information may be displayed as shown in the following figure.

    3. Correct the lifecycle script based on the error information. If the following information is displayed, the creation lifecycle fails to be executed. In this case, you need to correct the application configuration.
      hostname:~ # cd /var/paas/sys/log/kubernetes/  
        hostname:/var/paas/sys/log/kubernetes # vi kubelet.log   
        I0610 18:59:15.852078 +0800   24704 plugin.go:612] handle lifecycle for process process1
      I0610 18:59:15.852089 +0800   24704 plugin.go:694] Enter pod(csc-test/xujiapp-596c548587-x5sq8) process(process1) lifecycle handletype=Create
      I0610 18:59:15.852101 +0800   24704 plugin.go:709] Enter pod(csc-test/xujiapp-596c548587-x5sq8) process(process1) lifecycle commands=[]string{"/bin/bash", "-c", "bash installsss.sh"}
      E0610 18:59:15.857215 +0800   24704 plugin.go:803] run process process1 failed: exit status 127.
      E0610 18:59:15.857290 +0800   24704 plugin.go:712] RunCommand failed for pod csc-test/xujiapp-596c548587-x5sq8 in process process1, err is: exit status 127
      E0610 18:59:15.857911 +0800   24704 kubelet_for_process.go:221] Sync pod xujiapp-596c548587-x5sq8 error: failed to "Install" for "process1" with ExecuteCommandFailed: "error info: exit status 127"
    4. Run the following command to delete the abnormal pod to make it to be restarted:

      kubectl delete pod podname -n namespace

      In the preceding command, podname is the instance name obtained in 1, and namespace is the namespace obtained in 1.

    5. Run the following command to check whether the pod is running properly and the alarm is cleared:

      kubectl get pod -n namespace |grep podname

      In the preceding command, podname is the instance name obtained in 1, and namespace is the namespace obtained in 1.

      • If the pod is running properly, no further action is required.
      • If the pod is not running properly, go to 8.

  8. Contact technical support for assistance.

Alarm Clearing

This alarm will be automatically cleared after the fault is rectified.

Related Information

None

Translation
Download
Updated: 2019-08-30

Document ID: EDOC1100062365

Views: 47984

Downloads: 33

Average rating:
This Document Applies to these Products
Related Version
Related Documents
Share
Previous Next