No relevant resource is found in the selected language.

This site uses cookies. By continuing to browse the site you are agreeing to our use of cookies. Read our privacy policy>Search

Reminder

To have a better experience, please upgrade your IE browser.

upgrade
Knowledge Base

Cascading Layer Invoking an Interface at the Cascaded Layer Failed

Publication Date:  2019-05-31  |   Views:  409  |   Downloads:  0  |   Author:  zwx531734  |   Document ID:  EKB1100017080

Contents

Issue Description

A user failed to delete an EVS disk or ECS through the ManageOne operation plane. The user opens the logs collected by the g-ray service and finds that an error code shown in the figure below is displayed for the cinder-volume-cascading service.


The user clicks the error code. A dialog box is displayed, showing the message "Unable to establish connection to https://volume.type1.xian02.dga02.com:443/v2/fabbd5e50b124808a472e3429991849e/volumes?all_tenants=True&limit=600&name=volume%40bd4d7b6a-c47b-4ae8-8f02-38ae8be95330: HTTPSConnectionPool(host='volume.type1.xian02.dga02.com', port=443): Max retries exceeded with url: /v2/fabbd5e50b124808a472e3429991849e/volumes?all_tenants=True&limit=600&name=volume%40bd4d7b6a-c47b-4ae8-8f02-38ae8be95330 (Caused by NewConnectionError('<requests.packages.urllib3.connection.VerifiedHTTPSConnection object at 0x7742310>: Failed to establish a new connection: [Errno 110] ETIMEDOUT',))"



Alarm Information

·       The HAProxy service is abnormal.

Alarm name: Component Fault

Alarm ID: 73203

Alarm source: ID of the controller node at the cascaded layer

Additional information: haproxy.haproxy is included.

Handling Process

The error information indicates that the cinder-volume-cascading service at the cascading layer invoking an interface at the cascaded layer has timed out. Perform the following operations to manually rectify the fault:

1.    Log in to the ManageOne operation plane, chooseAlarms>Current Alarms, and check whether ALM-73203 Component Fault exists. If it exists,haproxy.haproxymust be included in the additional information.

·      If the alarm exists, the HAProxy service at the cascaded layer is abnormal. In this case, rectify the fault according to [Solution 1].

·      If the alarm does not exist, go to the next step.

2.     Contact the administrator to log in to a node at the cascading layer, enter environment variables, and run the following command to obtain the active node of the apacheproxy service:

cps template-instance-list --service apacheproxy apacheproxy



3.      Log in to the active node of the apacheproxy service and run thepingcommand to check whether the IP address of the reverse proxy at the cascaded layer can be pinged. (The IP address of the reverse proxy is obtained from the LLD document, for example,6.6.204.51.)



ping 6.6.204.51




·      If the IP address cannot be pinged, the reverse proxy network between the cascading and cascaded layers is faulty. Check the switch as instructed in the LLD document.

·      If the IP address can be pinged, ask the contact person to check for other reasons.


Solution


[Solution 1: Restart the HAProxy Service]:

1. Use the IP address of the external OM plane to log in to a controller node at the cascaded layer, import environment variables, and run the following command to query the HAProxy service statuses:

cps template-instance-list --service haproxy haproxy


·      If the HAProxy service statuses areactiveandstandby, the HAProxy service is normal. Ask the contact person to check for other reasons.

·      If the statuses arefault, go to the next step.

2. Stop the HAProxy service:



cps host-template-instance-operate --service haproxy haproxy --action stop




3. Start the HAProxy service:



cps host-template-instance-operate --service haproxy haproxy --action start



4. Wait 2 minutes and then perform step 1 to check the HAProxy service statuses. If the HAProxy service is normal, contact the user to apply for the ECS again. If the service is still abnormal, ask the contact person to check for other reasons.


Suggestions

 All services are provisioned at the cascaded layer for FusionCloud Type I. The cascading and cascaded layers communicate with each other through apacheproxy at the cascading layer and HAProxy at the cascaded layer. If either of the services or the network is abnormal, the cascading layer cannot send requests to the cascaded layer. As a result, this error occurs.