HCS IOC Software 2.3.0-S Maintenance Guide 02

HCS IOC Software
2.3.0
Fault Management


Issue 02

Date 2025-03-30


HUAWEI CLOUD COMPUTING TECHNOLOGIES CO., LTD.
Copyright © Huawei Cloud Computing Technologies Co., Ltd. 2025. All rights reserved.
No part of this document may be reproduced or transmitted in any form or by any means without prior
written consent of Huawei Cloud Computing Technologies Co., Ltd.

Trademarks and Permissions

and other Huawei trademarks are trademarks of Huawei Technologies Co., Ltd.
All other trademarks and trade names mentioned in this document are the property of their respective
holders.

Notice
The purchased products, services and features are stipulated by the contract made between Huawei
Cloud and the customer. All or part of the products, services and features described in this document may
not be within the purchase scope or the usage scope. Unless otherwise specified in the contract, all
statements, information, and recommendations in this document are provided "AS IS" without warranties,
guarantees or representations of any kind, either express or implied.
The information in this document is subject to change without notice. Every effort has been made in the
preparation of this document to ensure accuracy of the contents, but all statements, information, and
recommendations in this document do not constitute a warranty of any kind, express or implied.


Huawei Cloud Computing Technologies Co., Ltd.

Address: Huawei Cloud Data Center Jiaoxinggong Road
Qianzhong Avenue
Gui'an New District
Gui Zhou 550029
People's Republic of China

Website: https://www.huaweicloud.com/intl/en-us/


Document version 02 (2025-03-30) Copyright © Huawei Cloud Computing Technologies i
Co., Ltd. All rights reserved.
HCS IOC Software
Fault Management Contents


Contents


1 Fault Management........................................................................................................................ 1
1.1 Summary ........................................................................................................................................................................ 1
1.2 Information gathering.....................................................................................................................................................3
1.3 Service faults ..................................................................................................................................................................4


Document version 02 (2025-03-30) Copyright © Huawei Cloud Computing Technologies ii
Co., Ltd. All rights reserved.
HCS IOC Software
Fault Management 1 Fault Management


1 Fault Management

1.1 Summary
1.1.1 Fault management principle
 Principles of fault analysis, positioning and handling:
 In order to resume business as soon as possible as the principle.
 When locating the fault, the fault data information should be collected in time, and the
collected fault data information should be kept in the mobile storage medium or other
computers in the network.
 When determining the troubleshooting scheme, the impact should be evaluated first and given
priority to ensure the normal delivery of services.
 Third party hardware failure, you can view the relevant information of the third party or call
the service number of the third party company.
 If the fault point cannot be located or the fault cannot be solved according to the manual,
contact technical support to minimize business interruption time.
 Notes before positioning and processing:
 Strictly abide by the operating procedures and industry safety rules to ensure personal safety
and equipment safety.
 The fault phenomenon should be analyzed first, and the cause should be located before
processing. In the case of unknown reasons, blind operation should be avoided, leading to the
expansion of problems.
 Before handling faults, any records of the fault site should be kept, and data or logs cannot be
deleted at will.
 In dealing with faults, in order to ensure the security and privacy of the customer network, if
the relevant fault log is required, please get the consent and authorization of the customer in
advance.
 Before making any modifications, first back up the data through script export, manual backup,
etc.
 Any problems encountered during maintenance, various original information shall be recorded
in detail.


Document version 02 (2025-03-30) Copyright © Huawei Cloud Computing Technologies 1
Co., Ltd. All rights reserved.
HCS IOC Software
Fault Management 1 Fault Management

 All major operations such as restart process shall be recorded and the feasibility before
operation shall be carefully confirmed with proper backup, emergency and safety measures.
 After the system is restored, the operation situation must be observed to confirm that the fault
has been eliminated and the relevant treatment report should be filled in time.
 Use high-risk operations and commands carefully.
 Requirements for maintenance personnel:
 Have the basic knowledge of network equipment, operating system and database, master their
common operation commands, and can skillfully use them for maintenance work.
 Be familiar with the logical structure of the field HCS IOC Software Digital Twin
Visualization Platform service system, the correspondence between the components of the
HCS IOC Software Digital Twin Visualization Platform service system and the field
equipment, and the physical connection relationship between the field equipment.
 Familiar with HCS IOC Software Digital Twin Visualization Platform business process and
system structure, and skilled in operating business related components.
Understand the basic fault-related positioning and handling methods.
1.1.2 Deal with ideas
When the current problem failure occurs, proceed according to the following process.


Document version 02 (2025-03-30) Copyright © Huawei Cloud Computing Technologies 2
Co., Ltd. All rights reserved.
HCS IOC Software
Fault Management 1 Fault Management

1.2 Information gathering
1.2.1 State
Strictly follow the following principles during the information collection operation:
 Any maintenance operation must be authorized by the customer and prohibit any maintenance
operation beyond the customer approval scope.
 Sending the problem location data out to the customer network must be authorized by the
customer.
1.2.2 Basic information collection
Collect bureau point information
After the failure occurs, as the first condition of the problem positioning, through the collection of local
information, let the technical support and research and development personnel.
Quickly understand the situation of the site, at the same time, feedback the contact number of the site
engineer, to ensure the smooth contact channel.need.
The collected local information is shown in the table below.

Table 1-1 Site-point information collection table

Operator or sites Networking Engineer Customer's
enterprise diagram Name/Telephone Name/Telepho
attachment ne

Version - - - -
information

Remote - - - -
maintenance
information

Collect the basic fault information
Through the basic fault information, you can roughly understand the problems on the site, the current state,
and the equipment shape before the failure.
State and possible factors causing failure, the information to be collected is shown in the following table.

Table 1-2 Basic fault information collection table

To collect questions On-site feedback results

Description of fault phenomenon -

Time of failure -

Frequency of the failure occurrence -


Document version 02 (2025-03-30) Copyright © Huawei Cloud Computing Technologies 3
Co., Ltd. All rights reserved.
HCS IOC Software
Fault Management 1 Fault Management

Business impact degree -

Whether the current fault has been -
handled

Is there any system adjustment or any -
operation when the problem occurs?

Operations performed for problems -
arising during maintenance

After the problem appears, whether to -
take what measures to deal with it

To achieve the effect of dealing with -
the problem

There is no obvious alarm information -
on the scene

Whether the scene alarm information -
has been collected

Collect log information
When the above failure occurs, it can first consider checking the corresponding log in the operation log
service of "Operation And Maintenance Control Center > Log Management > Operation Log", mainly
viewing the abnormal information in the "operation log"; or switch to the service status at the back end of
the server to view the service status and view the log of the service startup abnormal service; judge the
reason of the abnormal service startup by viewing the "/opt/sdc/me/webapps/server/me-project-biz/logs"
log.
Account information
Refer to the HCS IOC Software 2.3.0-S ME Account List 01.xlsx.

1.3 Service faults
1.3.1 The service does not exist
Symptom
Click the menu module to access the corresponding service. The interface prompts: "The requested service
does not exist".
Impact on the system
After the service does not exist, the corresponding service function cannot be used.
Possible cause

Document version 02 (2025-03-30) Copyright © Huawei Cloud Computing Technologies 4
Co., Ltd. All rights reserved.
HCS IOC Software
Fault Management 1 Fault Management

An exception of the service, causing the service to close automatically.
Screening ideas

1. Determine that the service has a problem.

2. Check the status of the service through the script.

3. After determining the service, check the log to determine the cause of the problem.

4. According to the cause of the problem, the corresponding solution.
Procedure
Step 1 Log in to the HCS IOC Software Digital Twin Visualization Platform and click the Switch
menu module to confirm whether the corresponding service is available.
In this scenario, the me-workstation service does not exist as an example.


Figure 1-1 A certain service is abnormal
Step 2 Log in to the HCS IOC Software Digital Twin Visualization Platform server back-end run the
following command to switch to the sdc user.
su sdc
Step 3 As sdc user, run the following command to go to the script directory.
cd /opt/sdc/me/
Step 4 Run the following command to check the service status.
./check_health.sh


Document version 02 (2025-03-30) Copyright © Huawei Cloud Computing Technologies 5
Co., Ltd. All rights reserved.
HCS IOC Software
Fault Management 1 Fault Management


Figure 1-2 Service status

Table 1-3 service running status description

Status Instructions

true Started and running

false Not started

Step 5 find a service that is not enabled, try executing the following command as the case may be:
a) Start a service:
/opt/sdc/me/webapps/server/{service_name}/bin/app start
Start me-workstation service:
/opt/sdc/me/webapps/server/me-workstation/bin/app start
b) Stop a service:
/opt/sdc/me/webapps/server/{service_name}/bin/app stop
c) Restart a service:
/opt/sdc/me/webapps/server/{service_name}/bin/app restart

Explain
{service_name} Replace the service that you want to operate.
{service_name} contains me-workstation, me-project-biz, etc.

Step 6 If the restart does not solve the problem, please follow the following log path to check the
corresponding service to analyze the specific reason.

Table 1-4 List of service log paths


Document version 02 (2025-03-30) Copyright © Huawei Cloud Computing Technologies 6
Co., Ltd. All rights reserved.
HCS IOC Software
Fault Management 1 Fault Management

Service name Log path
ae-engine /opt/sdc/ae/ae-engine/logs
ae-manage-biz /opt/sdc/ae/ae-manage/logs
auth-center-biz /opt/sdc/pubapps/auth-center-biz/logs
bms-biz /opt/sdc/pubapps/bms-biz/logs
file-center-biz /opt/sdc/pubapps/file-center-biz/logs
gateway-server /opt/sdc/pubapps/gateway-serve/logs
log-center-biz /opt/sdc/pubapps/log-center-biz/logs
message-center-biz /opt/sdc/pubapps/message-center-biz/logs

gis /opt/sdc/me/webapps/server/gis/logs

me-csm-biz /opt/sdc/me/webapps/server/me-csm-biz/logs

me-data-server /opt/sdc/me/webapps/server/me-data-server/logs

me-media-server /opt/sdc/me/webapps/server/me-media-server/logs

me_resource-biz /opt/sdc/me/webapps/server/me_resource-biz/logs

me-simulation-biz /opt/sdc/me/webapps/server/me-simulation-biz/logs

me-workstation /opt/sdc/me/webapps/server/me-workstation/logs

me-cloud-render-biz /opt/sdc/me/webapps/server/me-cloud-render-biz/logs

me-csm-probe /opt/sdc/me/webapps/server/me-csm-probe/logs

me-ei-biz /opt/sdc/me/webapps/server/me-ei-biz/logs

me-project-biz /opt/sdc/me/webapps/server/me-project-biz/logs

me-scene-biz /opt/sdc/me/webapps/server/me-scene-biz/logs

me-video-analysis-biz /opt/sdc/me/webapps/server/me-video-analysis-
biz/logs

1.3.2 The authorization file verification error
Symptom
The HCS IOC Software Digital Twin Visualization Platform displays a message indicating an
authorization file signature verification error. Upload the license file as soon as possible.


Document version 02 (2025-03-30) Copyright © Huawei Cloud Computing Technologies 7
Co., Ltd. All rights reserved.
HCS IOC Software
Fault Management 1 Fault Management

Impact on the system
The system is not available.
Possible cause
License The authorization is expired, resulting in unable to use.
Screening ideas
troubleshoot service error log.
Procedure
Step 1 Log in to the HCS IOC Software Digital Twin Visualization Platform and switch the menu to
view module permissions.


Figure 1-3 Error interface
Step 2 Log in to the HCS IOC Software Digital Twin Visualization Platform server back-end as the
root user.
Step 3 Run the following command to go to the script directory as user root and check whether
license.xml exists.
cd /usr/local/secx/etc
Step 4 If the license does not exist, apply for it. If the license does exist, check whether the time limit
in license.xml has expired. If the license has expired, submit the application again.


Document version 02 (2025-03-30) Copyright © Huawei Cloud Computing Technologies 8
Co., Ltd. All rights reserved.
HCS IOC Software
Fault Management 1 Fault Management


Figure 1-4 License file display
Step 5 (Optional) Check the serial number.
Run the following command to generate the license serial number.
/usr/local/secx/bin/secx-serialnum
After successful execution, retrieve the serial number from the "red box" in Figure 1.3.2-3and
compare it with the serial number in the license.xml file:
 If the serial numbers do not match, you need to submit a new license application using the
new serial number and upload it to the digital twin visualization platform.
 If the serial numbers match, check if the format of the license.xml is correct.


Figure 1-5 license code
1.3.3 The visualization page cannot be accessed
Symptom Description
The HCS IOC Software Digital Twin Visualization Platform cannot be accessed through the browser.
Impact on the system
Page is not accessible and business is not available.
Possible cause
The agent had a problem and could not return to the front-end static page.
Screening ideas

Document version 02 (2025-03-30) Copyright © Huawei Cloud Computing Technologies 9
Co., Ltd. All rights reserved.
HCS IOC Software
Fault Management 1 Fault Management

Check that the agent service is normal.
Procedure
Step 1 Access a web browser and log in to the web page of the HCS IOC Software Digital Twin
Visualization Platform to check whether you can log in to the system.
Note: Default access address is https://{application_data_server ip}:18550/me/portal/


Figure 1-6 Platform login error
Step 2 Log in to the HCS IOC Software Digital Twin Visualization Platform application server back-
end based on the account information in the attachment and switch to the sdc user.
su sdc
Step 3 Run the following command to go to the script directory as user sdc.
cd /opt/sdc/me/
Step 4 Run the following command to check the nginx service status.
./check_health.sh


Figure 1-7 Service status


Document version 02 (2025-03-30) Copyright © Huawei Cloud Computing Technologies 10
Co., Ltd. All rights reserved.
HCS IOC Software
Fault Management 1 Fault Management

Step 5 When nginx is offline, run the following command to start the nginx service.
nginx
Step 6 After the nginx service starts, you can access the digital twin visualization platform.


Figure 1-8 Login interface
1.3.4 The visualization page response efficiency is very low
Symptom Description
HCS IOC Software Digital Twin Visualization Platform can be used normally, but the response efficiency
is very low.
Impact on the system
Running state page access is very slow and page load times are very long.
Possible cause
The server resources are tight, resulting in a slow overall business processing.
Screening ideas
Check the memory and disk usage of the server.
Procedure
Step 1 Log in to the HCS IOC Software Digital Twin Visualization Platform application or database
server.
Step 2 Check the disk status of the server.
Run the command:
df -h


Document version 02 (2025-03-30) Copyright © Huawei Cloud Computing Technologies 11
Co., Ltd. All rights reserved.
HCS IOC Software
Fault Management 1 Fault Management


Figure 1-9 Disk situation
Step 3 (Optional) If the disk space in the/root directory is insufficient, expand the disk capacity.
Step 4 Check the server memory status.
Run the command:
free -h


Figure 1-10 Memory situation
Step 5 (Optional) If the server memory is insufficient, expand the capacity of the server.
Step 6 Execute command to release memory:
echo 1 > /proc/sys/vm/drop_caches
echo 2 > /proc/sys/vm/drop_caches
echo 3 > /proc/sys/vm/drop_caches


Document version 02 (2025-03-30) Copyright © Huawei Cloud Computing Technologies 12
Co., Ltd. All rights reserved.