No relevant resource is found in the selected language.

This site uses cookies. By continuing to browse the site you are agreeing to our use of cookies. Read our privacy policy>Search

Reminder

To have a better experience, please upgrade your IE browser.

upgrade

FusionCloud 6.3.1.1 Troubleshooting Guide 02

Rate and give feedback :
Huawei uses machine translation combined with human proofreading to translate this document to different languages in order to help you better understand the content of this document. Note: Even the most advanced machine translation cannot match the quality of professional translators. Huawei shall not bear any responsibility for translation accuracy and it is recommended that you refer to the English document (a link for which has been provided).
FAQs

FAQs

How Do I Configure the NTP Clock Source for the Deployment System Node?

Based on the NTP requirements of users, all services must share the same clock source to ensure time consistency. If there is a time difference between services, an exception occurs when services call each other. By default, the clock source of the deployment system node is set to LOCAL. If users want to configure their own NTP servers, parameters in the /etc/ntp.conf file need to be set.

Context

You need to configure a synchronization policy file /etc/ntp.conf for the deployment system node so that the other nodes can synchronize and validate the configuration policy during the initialization.

Procedure
  1. Use PuTTY to log in to the deployment system node regionAlias-OM-Global-Deploy01 as the root user.
  2. Run the following command to edit the NTP service configuration file of the OS:

    vim /etc/ntp.conf

  3. Add the NTP clock sources to be modified to the end of the command line in the configuration file. The example is as follows:

    server IP address of NTP clock source 1 maxpoll 4 minpoll 3 prefer
    server IP address of NTP clock source 2 maxpoll 4 minpoll 3 prefer
    NOTE:

    prefer indicates that the time is preferentially synchronized from IP address of NTP clock source 1.

  4. Press Esc, enter :wq, and then press Enter to save and exit the vi editor.

  5. Run the following command to restart the NTP service:

    service ntpd restart

    The NTP clock source is configured successfully when the following information is displayed.

    Shutting down network time protocol daemon (NTPD)            done
    Starting network time protocol daemon (NTPD)                 done

How Do I Upload and Scan the Software Package?

This section describes how to upload a software package to the deployment system node and scan the package.

Prerequisites

You have obtained the software package. For details about how to obtain the software package, see Obtaining Required Software Packages and Documents.

Procedure
  1. Use FileZilla to upload the ManageOne-6.3.2.zip software package and the ManageOne-6.3.2.zip.asc digital signature file to the /opt/pub/software/solutions directory of the deployment node regionAlias-ManageOne-Deploy01 as the root user.

    NOTE:

    In this document, software package names are examples only. Replace them with actual package names.

  2. Run the following commands to set the owner group of the software package and its read and write permissions:

    cd /opt/pub/software/

    chown -R ossadm:ossgroup *

    find /opt/pub/software/ -type f | xargs chmod 640

    find /opt/pub/software/ -type d | xargs chmod 750

  3. Open a web browser, enter https://Floating IP address of the ManageOne deployment plane :31943 in the address box, and press Enter. Log in to the ManageOne deployment plane as the admin user.
  4. Scan the software packages.

    1. Choose Deployment > Feature Deployment > Software from the main menu.
    2. On the Software page, click Scan. In the displayed dialog box, click Yes to start the scanning.
    3. On the displayed Tasks page, view the scanning progress and status.

      The software scanning is successful if Progress shows 100% and Status shows Successful.

    4. Confirm the software package scanning results.
      1. Choose Deployment > Feature Deployment > Software from the main menu.
      2. On the Software page, check whether ManageOne-6.3.2 exists. If it does, the scanning is successful. Otherwise, contact technical support for assistance.
        NOTE:
        • The service deployment system automatically checks the validity and integrity of software packages. Invalid software packages are not displayed in the software package list.
        • In this document, software package names are examples only. Replace them with actual package names.

What Do I Do If a Batch Processing File Failed to Be Launched?

An error message is displayed when a user launches the .bat batch processing file because the DLL file of the OS on the local PC is lost.

Procedure
  1. Go to the /check/3rdTools directory of the ManageOne_init_tools folder on your PC and execute vcredist_x86.exe.
  2. In the displayed dialog box, select the I have read and accept the license terms check box and click Install.

    When the "Installation Is Complete" message is displayed, click finish to close the dialog box.

  3. On your local PC, double-click the .bat batch processing file in the root directory of the ManageOne_init_tools folder and check whether an error message is displayed.

    • If the error message is displayed, go to 4.
    • If the error message no longer displays, the problem is resolved.

  4. Obtain the related DLL file and store the file in the corresponding directory on your local PC as listed in Table 26-24.

    Table 26-24 Directory for storing the DLL file

    If...

    Then...

    The OS version is Windows series 64-bit

    Store the DLL file in the C:\Windows\SysWOW64 and C:\Windows\System32 directories.

    The OS version is Windows series 32-bit

    Store the DLL file in the C:\Windows\System32 directory.

    If the error message is still displayed after you double-click the .bat batch processing file, contact technical support.

What Do I Do When Services Fail to be Installed?

This section describes how to install a service again after it fails to be installed.

How Do I Reinstall a Service Instance Failed to be Installed?

This section uses the MOVDC/Global/regionAlias service instance as an example to describe how to reinstall a service instance that failed to be installed.

Procedure
  1. View the microservice that failed to be deployed.

    1. Choose Deployment > Feature Deployment > Services from the main menu.
    2. In the Instance Name column, click MOVDC/Global/regionAlias that failed to be deployed.
    3. In the Deployment History List area, click the Failed status in the Deployment Status column.
    4. In the Resource Deploy Status area, view and record MOVDCService that failed to be deployed.

  2. Query the node where the microservice failed to be deployed.

    1. Choose Deployment > Microservice Deployment > Environments.
    2. In the upper right corner, select All and Product from the search area, and enter MOVDCService in the search box.
    3. Click MOVDCService/Global/regionAlias in the Environment column.
    4. In the Node List area, view the node whose Deployment Status is Failed.

  3. Query and restart the parent microservice of MOVDCService.

    1. Use PuTTY to log in to the node obtained in 2.d as user sopuser.

      The default password of the sopuser user is D4I$awOD7k.

    2. Run the following command to switch to user ossuser:

      su - ossuser

      Enter the password of user ossuser as prompted. The default password of the ossuser user is 79y52unx@R.

    3. Run the following commands to query the parent microservice of MOVDCService:

      cd /opt/oss/envs

      find -name start.sh 2>/dev/null| xargs grep MOVDCService | awk -F'-' '{print $2}' | awk -F'/' '{print $1}'

      Replace MOVDCService in the preceding command with the actual microservice name. Information similar to the following is displayed:

      MOVDCService
      MOLoginAgentService
      • If the microservice in the command and the other microservice are included in the displayed information, the other microservice is the parent microservice of the microservice in the command. Go to 3.d to restart the parent microservice.

        For example, MOLoginAgentService included in the previous displayed information is the parent microservice of MOVDCService.

      • If only the microservice in the command is included in the displayed information, the microservice is the parent microservice. Go to 4.
    4. Run the following command to switch to the ossadm user:

      su - ossadm

      Enter the password of the ossadm user as prompted. The default password of the ossadm user is ZJE%JLq5qx.

    5. Run the following commands to stop MOVDCService and restart MOLoginAgentService:

      . /opt/oss/manager/bin/engr_profile.sh

      ipmc_adm -cmd stopapp -app MOVDCService

      ipmc_adm -cmd restartapp -app MOLoginAgentService

  4. Reinstall the service instance.

    1. Choose Deployment > Feature Deployment > Services from the main menu.
    2. In the Instance Name column, click MOVDC/Global/regionAlias that failed to be deployed.
    3. In the Deployment History List area, locate the row that contains the Failed deployment status, and click Retry.
    4. On the Service page, check the instance deployment status.
      • If the status is displayed as , the service instance has been installed successfully and the service process has been started.
      • If the status is displayed as , the service instance has failed to be installed. In this case, uninstall the service instance and install it again by following the instructions provided in How Do I Uninstall the Service Instances and Install the Service Again?

How Do I Uninstall the Service Instances and Install the Service Again?

This section describes how to uninstall the service instances and install the service again.

Procedure

The Deploy feature is used as an example in this section.

  1. Log in to the service deployment system.
  2. Choose Deployment > Feature Deployment > Software from the main menu.
  3. In the search box in the upper right corner of the Software page, select All, enter ManageOne, and press Enter.

    NOTE:
    • If no software package is displayed, click Scan in the upper left corner of the Software page and then search for the software package again. If the problem persists, upload the software package by following the instructions provided in How Do I Upload and Scan the Software Package? and scan it again.
    • If the digital signature file ManageOne-6.3.X.zip.asc is not uploaded during software package uploading, the software package cannot be found during package scanning in the service deployment system.

  4. Click in the Operation column that contains the ManageOne-6.3.X software package.

    NOTE:

    In this document, software package names are examples only. Replace them with actual package names.

  5. Set parameters on the Configure deployment info tab page as described in Table 26-25.

    Table 26-25 Parameter description

    Parameter

    Value

    Owner

    system

    Region

    Set this parameter to the value of RegionID in environment planning in Table 26-4. For example, the value can be cn-global-1.

    Stage

    Product

    Region alias

    Set this parameter to the value of regionAlias in environment planning in Table 26-4. For example, the value can be MOC.

    Feature set

    Select Manager-HA-Deploy.

  6. Select the Deploy feature.
  7. Click to set parameters automatically.

    Table 26-26 describes these parameters.
    Table 26-26 Parameter description

    No.

    Parameter

    Description

    Example Value

    1

    MGGlobalAccessLBAddress

    Indicates the floating IP address of the Deploy nodes.

    10.31.32.53

  8. Click Next.
  9. All the Deploy instances are displayed, as shown in Figure 26-1.

    Figure 26-1 Global Deploy instances

  10. Uninstall the Manager instances one by one.

    1. Choose Deployment > Feature Deployment > Services from the main menu.
    2. Click in the row that contains the Manager instance to be uninstalled.
    3. Click OK and wait until the task is complete.

  11. After uninstalling all the Manager instances, perform 1 through 9 and click OK to deploy Deploy in the Global zone again.

How Do I Create a VM?

This section describes how to create a VM in the OpenStack+KVM scenario. Operations in this section are also applicable to other scenarios.

This section assumes that 500 VMs are managed in the FusionCloud solution. Table 26-27 describes information required for creating a ManageOne VM.

For details about the VM hardware configuration requirements for other management scales in the FusionCloud solution, see ManageOne 6.3.1 Installation Planning Guide.

If the log center service is not installed, the log center VMs regionAlias-ManageOne-LOG01 and regionAlias-ManageOne-LOG02 do not need to be created.

If the automatic O&M service is not installed, the automatic O&M VMs regionAlias-ManageOne-AutoOps01/02 do not need to be created.

Table 26-27 VM creation planning information

VM Node

Availability Zone

Image

Disk Type

Flavor

Number of vCPUs

Memory (MB)

System Disk (GB)

Data Disk (GB)

Network Plan

IP Address

regionAlias-ManageOne-Deploy01/02

manage-az

Use the image registered in Registering an Image.

Cloud Disk

mo.om.small2

4

14336

8

60

Public_Service network plane

IP address specified in the manually configured environment plan

regionAlias-ManageOne-Service01/02/03/04

mo.small3

6

22528

8

130

regionAlias-ManageOne-DB01/02

mo.small2

4

12288

8

200

regionAlias-ManageOne-IAMService01/02

mo.iam.small

4

6144

8

30

regionAlias-ManageOne-LOG01/02

mo.log.small

4

10240

8

500 + 300 x (Number of regions – 1)

regionAlias-ManageOne-AutoOps01/02

mo.auto.small

2

4096

8

80

  • VM IP addresses can be automatically allocated or manually specified. Take the requirements of the upper-layer solution as final if any.
  • The data disk specifications in the log center depend on the number of planned regions.

    Size (GB) of the data disk in the log center = Basic disk specifications in the management scale + (Number of planned regions – 1) x 300 (GB)

    For example: If two regions are to be installed, the size of the data disk in the log center is 800 GB (500+ (2 – 1) x 300).

Registering an Image
Procedure
  1. Decompress the VM template file ManageOne_OSImg_EulerOS2.2_basetemplate_qcow2.zip to your local PC.
  2. On the Service OM homepage, choose Computing > IMS > Images and click Register.
  3. Set parameters as described in Table 26-28 and click Register.

    Table 26-28 Parameters for registering the image

    Parameter

    Description

    Example Value

    Provided As Service

    Not supported

    Not supported

    Type

    Type of the image, which includes FusionCompute, KVM, VMware, and Ironic.

    KVM

    Image Name

    Name of the image, which consists of 1 to 255 characters.

    ManageOne

    Applicable OS

    Applicable OS, which must be the same as that in the image file.

    Linux

    OS Version

    Applicable OS, which must be the same as that in the image file.

    EulerOS 2.2 64bit

    Min Disk (GB)

    Minimum disk capacity of the VM created using the image, which must be greater than or equal to the VM disk capacity used for creating the image.

    8

    Min Memory (MB)

    Minimum memory of the VM created using the image.

    4096

    Disk Device Type

    Bus type used when the system disk communicates with the host for reading and writing data. The value can be scsi or virtio for registering Euler or SUSE Linux images. You are advised to set it to virtio.

    virtio

    Upload Mode

    • HTTPS: The image file is uploaded from a local PC.
    • NFS: The image file is uploaded from the Network File System (NFS) server.

    HTTPS

    Image File

    VM image file stored on your local PC.

    ManageOne_OSImg_EulerOS2.2_basetemplate.qcow2

    Disk Format

    Disk format, which must be the same as that in the image file.

    qcow2

    NOTE:

    In addition to the parameters listed in the table, retain the default values of other parameters in the Register Image dialog box.

Creating Flavors
Procedure
  1. On the Service OM homepage, choose Computing > ECS > Flavors and click Create Flavor.
  2. Configure the parameters as described in Table 26-29. Confirm the information and click OK to create the ManageOne VM flavor.

    If the log center service is not installed, you do not need to create flavor mo.log.small.

    If the automatic O&M service is not installed, you do not need to create flavor mo.auto.small.

    Table 26-29 Parameters for creating flavors

    Parameter

    Description

    mo.om.small2

    mo.small3

    mo.small2

    mo.iam.small

    mo.log.small

    mo.auto.small

    Type

    Type of the computing instance created using this flavor

    VM

    VM

    VM

    VM

    VM

    VM

    Boot Device

    • Local Disk: The local disk of the host is used. Compared with EVS disks, local disks feature stable I/O performance and high throughput, but they are not limited by the VDC quota and the usage statistics cannot be collected. In addition, the performance of local disks varies with load and single point of failure (SPOF) may be involved. Therefore, use local disks as boot device for systems that run for a short period of time with a relatively low requirement on stability and reliability. You are advised to synchronize key data to other instances or back up the data to a cloud disk.
    • Cloud Disk: Shared storage is used. EVS disks feature high reliability and storage performance and support hot migration and disk upgrade and degrade. Therefore, use cloud disks as boot device for systems that run for a long period of time with a high requirement on stability and reliability.

    Cloud Disk

    Cloud Disk

    Cloud Disk

    Cloud Disk

    Cloud Disk

    Cloud Disk

    Name

    Name of the flavor. The name consists of 1 to 255 characters. Only uppercase letters, lowercase letters, digits, hyphens (-), underscores (_), and periods (.) are allowed.

    mo.om.small2

    mo.small3

    mo.small2

    mo.iam.small

    mo.log.small

    mo.log.small

    vCPUs

    Number of vCPUs, which is an integer from 1 to 256.

    4

    6

    4

    4

    4

    2

    Memory (MB)

    Memory size, which is an integer from 1 to 6291456.

    14336

    22528

    12288

    6144

    10240

    4096

    System Volume (GB)

    Size of the computing instance disk created using this flavor, which must be greater than or equal to the system disk size of the image.

    8

    8

    8

    8

    8

    8

    vCPU Bound to Physical Thread

    • Enabled: Bind the vCPU of the VM to the physical thread to reduce CPU competition and improve performance.
    • Disabled: The vCPU of the VM is not bound to the physical thread.

    Disabled

    Disabled

    Disabled

    Disabled

    Disabled

    Disabled

    NUMA Affinity

    NUMA affinity. It specifies whether physical resources occupied by the VM vCPU and memory are allocated from the same NUMA node. The default value is Enabled in the following conditions: Huge page memory is set, vCPU Bound to Physical Thread is enabled, or real-time VMs are used. If you set this parameter to Disabled in the preceding conditions, the setting will not apply.

    Disabled

    Disabled

    Disabled

    Disabled

    Disabled

    Disabled

  3. Repeat 1 and 2 to create VMs in other flavors on ManageOne.
Creating a VM Group
Context

A VM group allows you to configure the policy used for scheduling VMs in the group. During VM creation, you can specify a VM group for the VM. Based on the VM group policy, the system can define the policy used for scheduling VMs in the VM group. Set the affinity group policy to anti-affinity to prevent VMs of the same service type from being deployed on the same physical host.

Table 26-30 describes the VM groups to be created for ManageOne VMs. Replace the VM group names with actual ones.

In addition to the VMs listed in Table 26-30, you need to manually select different dedicated servers for service VMs and IAM VMs when creating VM compute instances to ensure that reliability rules are met. For details, see Table 26-32 in Creating a VM Compute Instance.

If the log center service is not installed, you do not need to create VM group LOG_VM_GROUP.

If the automatic O&M service is not installed, you do not need to create VM group AUTOOPS_VM_GROUP.

Table 26-30 ManageOne VM groups

VM Name

VM Group

regionAlias-ManageOne-Deploy01

regionAlias-ManageOne-Deploy02

MO_Deploy_VM_GROUP

regionAlias-ManageOne-DB01

regionAlias-ManageOne-DB02

DB_VM_GROUP

regionAlias-ManageOne-LOG01

regionAlias-ManageOne-LOG02

LOG_VM_GROUP

regionAlias-ManageOne-AutoOps01

regionAlias-ManageOne-AutoOps02

AUTOOPS_VM_GROUP

Procedure
  1. On the Service OM homepage, choose Computing > ECS > VM Groups and click Create.
  2. Set Name to MO_Deploy_VM_GROUP and Policy to Anti-affinity, and then click OK.

    NOTE:

    Anti-affinity indicates that VMs in the same VM group must run on different hosts. When you create a VM in a specified anti-affinity VM group, the VM cannot be created on the same host where the VMs in the VM group are running if the VM group already contains VMs. Otherwise, VM creation fails. After the VM is created, the anti-affinity policy becomes invalid when VM migration is performed.

  3. Repeat 1 and 2 to create VM groups DB_VM_GROUP and AUTOOPS_VM_GROUP in sequence.
Creating a VM Compute Instance
Procedure
  1. On the Service OM homepage, select Computing > ECS > Compute Instances and click Create. regionAlias-ManageOne-Deploy01 is used as an example.
  2. Select manage-az for Availability Zones from the drop-down list, and click Next.
  3. Select ManageOne in the Select Image area, select Cloud Disk in the Select System Disk Storage Type area, and click Next.
  4. Select mo.om.small2 for Flavor, and click Next.
  5. Set parameters as described in Table 26-31 and keep default settings for other parameters. Click Next.

    Table 26-31 Parameters for creating a VM on the ManageOne Deploy node

    Parameter

    Description

    Example Value

    Port Type

    Retain the default value.

    Software switch

    Network

    • Shared Network: used for service IP addresses.
    • UnShared Network: used for management IP addresses.

    UnShared Network

    Project

    Set this parameter as required.

    Projects must be on the Public_Service network plane.

    dc_system_pool (ID:xxx)

    Public_Service (VLAN ID:xxx)

    IP

    • Automatic allocation: IP addresses are automatically allocated by the system.
    • Specified: Set the IP address based on the IP address segment and as planned.

      VM IP addresses can be automatically allocated or manually specified. Take the requirements of the upper-layer solution as final if any.

    IP addresses planned in Table 26-4 if specified

    NOTE:

    In this step, you do not need to configure information about NICs and floating IP addresses.

  6. Set VM Name to MOC-ManageOne-Deploy01 and Computer Name to Deploy01.
  7. Select the host where the MOC-ManageOne-Deploy01 VM is to be deployed.

    NOTE:

    Different VMs must be deployed on at least three dedicated servers to improve service reliability. This avoids a single point failure due to service interruption caused by dedicated server faults.

    Select VM groups created in How Do I Create a VM? for VMs on the Deploy nodes, database nodes, log center nodes, and automatic O&M nodes, respectively.

    Select dedicated servers listed in Table 26-32 for VMs on the O&M nodes, operation nodes, and IAM nodes to ensure high reliability of ManageOne.
    Table 26-32 Mapping between VMs and dedicated servers

    Dedicated Server

    VM

    Dedicated server 1

    regionAlias-ManageOne-Service01

    regionAlias-ManageOne-IamService01

    Dedicated server 2

    regionAlias-ManageOne-Service02

    regionAlias-ManageOne-Service04

    Dedicated server 3

    regionAlias-ManageOne-Service03

    regionAlias-ManageOne-IamService02

  8. Click Finish. Creating a VM takes about 15 minutes.
  9. Choose Resources > Computing > Compute Instances from the main menu. Record IDs of the created VMs.
  10. Repeat 1 through 9 to create other VMs of ManageOne. Table 26-27 describes required parameters.

    If the log center service is not installed, the log center VMs regionAlias-ManageOne-LOG01 and regionAlias-ManageOne-LOG02 do not need to be created.

    If the AutoOps service is not installed, the O&M automation VMs regionAlias-ManageOne-AutoOps01 and regionAlias-ManageOne-AutoOps02 do not need to be created.
    NOTE:

    After a VM is created, the default user name is root and password is Changeme_123.

Creating and Binding a Disk
Procedure
  1. On the Service OM homepage, choose Storage > EVS > Disks and click Create.
  2. Set disk parameters. regionAlias-ManageOne-Deploy01 is used as an example.

    For details about the disk capacity specifications, see Table 26-27.
    Table 26-33 Parameter description

    Parameter

    Description

    Example Value

    Name

    Indicates the disk name.

    MOC-ManageOne-Deploy01

    Availability Zone

    Indicates the AZ.

    manage-az

    Capacity (GB)

    Indicates the disk capacity.

    60

    Disk Type

    Indicates the type of the disk to be created.

    manager.fsa

  3. Click OK.
  4. Choose Storage > EVS > Disks.
  5. Locate the target disk MOC-ManageOne-Deploy01, click More, and select Bind VM.
  6. Select the target VM MOC-ManageOne-Deploy01 and click OK.
  7. Repeat 1 to 6 to create and bind disks to other VMs of ManageOne.

How Do I Manually Initialize Data Disks?

You need to manually initialize the data disks that fail to be initialized during data disk initialization.

Procedure
  1. Go to the /bin directory of the ManageOne_init_tools folder on your local PC and locate the create_vol_euler.sh file.
  2. Use FileZilla to upload the create_vol_euler.sh file to the /opt/ directory of each VM node whose data disk needs to be initialized, as the root user.
  3. Use PuTTY to log in to each VM node whose data disk needs to be initialized as user root.
  4. On each VM node, run the following command to query the name of the new data disk, and record the name:

    fdisk -l

    Information similar to the following is displayed:

    Disk /dev/xvda: 75.2 GB, 75161927680 bytes
    255 heads, 63 sectors/track, 9137 cylinders, total 146800640 sectors
    Units = sectors of 1 * 512 = 512 bytes
    Sector size (logical/physical): 512 bytes / 512 bytes
    I/O size (minimum/optimal): 512 bytes / 512 bytes
    Disk identifier: 0x00082be6
    
        Device Boot      Start         End      Blocks   Id  System
    /dev/xvda1   *        2048     2088959     1043456   83  Linux
    /dev/xvda2         2088960    35633151    16772096   83  Linux
    /dev/xvda3        35633152    71280639    17823744   83  Linux
    /dev/xvda4        71280640   146800639    37760000    f  W95 Ext'd (LBA)
    /dev/xvda5        71282688    73369599     1043456   83  Linux
    /dev/xvda6        73371648    83843071     5235712   83  Linux
    /dev/xvda7        83845120    90124287     3139584   83  Linux
    /dev/xvda8        90126336   132055039    20964352   83  Linux
    /dev/xvda9       132057088   136247295     2095104   83  Linux
    /dev/xvda10      136249344   146800639     5275648   83  Linux
    
    Disk /dev/xvde: 569.1 GB, 569083166720 bytes
    255 heads, 63 sectors/track, 69187 cylinders, total 1111490560 sectors
    Units = sectors of 1 * 512 = 512 bytes
    Sector size (logical/physical): 512 bytes / 512 bytes
    I/O size (minimum/optimal): 512 bytes / 512 bytes
    Disk identifier: 0x00000000
    
    Disk /dev/xvde doesn't contain a valid partition table
    NOTE:

    /dev/xvda indicates the system disk name and /dev/xvde indicates the data disk name. Data disk names start from /dev/xvde and increase in ascending order, for example, /dev/xvde, /dev/xvdf, and /dev/xvdg. /dev/xvda and /dev/xvde are examples only. Replace them with actual disk names as required.

  5. Run the following commands to initialize the data disk:

    cd /opt

    bash create_vol_euler.sh Data disk name

    It takes about 2 minutes to log in to a single VM and initialize the data disk of the VM. If the following information is displayed, the data disk of the VM has been initialized successfully:

    Execute create_vol_euler.sh successfully

How Do I Forcibly Restart a VM?

Symptom

The VM cannot be accessed or the VM does not respond for a long time.

Procedure

Contact the lab administrator to forcibly restart the VM.

Follow-up Procedure

Verify the troubleshooting results. For details, see Verifying the Result.

How Do I Avoid Incorrect VM Time Due to Node Power-Off?

Symptom

No external NTP clock source is configured for management nodes of ManageOne. When a management node is powered off unexpectedly, the time on the node is inconsistent with the local time.

Procedure
  1. Use PuTTY to log in to the regionAlias-ManageOne-Deploy01 node as the root user.

    The default password is Changeme_123.

  2. Run the following command to edit the NTP service configuration file of the OS:

    vim /etc/ntp.conf

  3. Append the NTP clock sources to the command line. Example commands:

    server IP address of NTP clock source 1 maxpoll 4 minpoll 3 prefer
    server IP address of NTP clock source 2 maxpoll 4 minpoll 3
    NOTE:

    prefer indicates that the time is preferentially synchronized from IP address of NTP clock source 1.

  4. Press Esc. Then, run the following command to save the configuration and exit:

    :wq!

  5. After the patch packages are installed, run the following command to restart the NTP service:

    service ntp restart

  6. Run the following command to query the status of the NTP service:

    ntpq -p

    Information similar to the following is displayed:

       remote           refid      st t when poll reach   delay   offset  jitter 
    ============================================================================== 
    *192.168.8.12    192.168.8.11    1 u   29   64  177    0.240    0.093   1.222 

    192.168.8.12 indicates the IP address of the NTP server clock source. * indicates that the NTP service status is normal, which is displayed after 5 minutes.

Clearing Disk Space

Symptom

Operations such as logging in to the system or installing software packages sometimes fail.

Possible Causes

The system has been running for a long time but the disk space has not been cleared. As a result, the disk space is insufficient.

Procedure

Deleted files cannot be restored. Exercise caution when performing the following operations.

  1. Use PuTTY to log in to the regionAlias-ManageOne-Deploy01 node as the root user.

    The default password is Changeme_123.

  2. Run the following command to verify the usage of each partition:

    df -h

    In the command output, if the usage of a partition exceeds 80%, clear the space.

    Filesystem                     Size  Used Avail Use% Mounted on
    /dev/xvda3                      17G  2.5G   14G  16% /
    devtmpfs                       7.8G  152K  7.8G   1% /dev
    tmpfs                          7.8G     0  7.8G   0% /dev/shm
    /dev/xvda1                    1003M   50M  903M   6% /boot
    /dev/xvda5                    1003M   18M  935M   2% /home
    /dev/xvda10                    5.0G  915M  3.9G  19% /usr
    /dev/xvda6                     3.0G  176M  2.7G   7% /var
    /dev/xvda7                     5.0G  3.0G  1.7G  64% /var/log
    /dev/xvda8                    1003M   18M  935M   2% /var/log/audit
    /dev/xvda9                    1003M   18M  935M   2% /var/tmp
    /dev/mapper/oss_vg-opt_vol      89G   76G   13G  85% /opt
    /dev/mapper/oss_vg-optlog_vol   30G  178M   28G   1% /opt/log

  3. For example, to clear the /opt directory, run the following commands to go to the /opt directory and sort the occupied space size of directories in descending order (unit: MB).

    cd /opt

    du -sm * |sort -rn

    The following command output shows that the pub directory occupies the largest space, that is, about 12.5 GB.

    12492    pub
    5887     tools
    2598     mysql
    1092     oss
    762      log
    96       share
    15       redis
    1        sudobin2
    1        lost+found
    1        aquota.user
    1        aquota.group

  4. Run the following commands to go to the pub directory and sort the directories according to the directory sizes (in MB) in descending order:

    cd pub

    du -sm * |sort -rn

    The following command output shows that the software directory occupies the largest space.

    12492    software
    557      upload
    1        manager
    1        backup_local

  5. Go to the /software directory and find unnecessary files that occupy large space by referring to commands described in 4.
  6. Run the following command to delete the file:

    rm -r xxx

    NOTE:

    xxx: files to be deleted.

  7. Repeat the preceding operations to clear the space of directories whose usage exceeds 80% if any. Otherwise, skip this step.
Follow-up Procedure

Check and clear the disk space periodically.

How Do I Configure a Floating IP Address?

Context

Before installing a service, a temporary floating IP address is configured to meet service requirements. The server OS is unexpectedly started before services of the temporary floating IP address are taken over. As a result, the temporary floating IP address becomes invalid and the services cannot be installed. Therefore, the temporary floating IP address needs to be reconfigured.

Prerequisites
  • You have obtained the passwords for the sopuser and root users.
  • You have obtained the floating IP address of the server.
Procedure
  1. Use PuTTY to log in to the operating system of the server as the sopuser user.
  2. Run the following command to switch to the root user:

    su - root

    Default password: Changeme_123

  3. Run the following command to configure the floating IP address:

    ifconfig eth0:0 <Floating IP address of the server>

  4. Run the following command to check whether the floating IP address is configured successfully:

    ifconfig eth0:0

How Do I Detect IP Address Conflicts?

Symptom

The network connection is unstable or is intermittently disconnected.

Procedure

The following assumes that node IP address is 192.168.1.100 and the NIC name of the node is eth0. Replace the IP address and NIC name with the actual ones.

  1. Use PuTTY to log in to that node as the root user.
  2. Run the following commands to check whether IP address conflicts exist:

    arping -D -I eth0 -c 2 192.168.1.100

    • If command output similar to the following is displayed, there is no IP addresses conflicts.
      ARPING 192.168.1.100 from 0.0.0.0 eth0
      Sent 2 probes (2 broadcast(s))
      Received 0 response (s)
    • If command output similar to the following is displayed, there are IP addresses conflicts.
      ARPING 192.168.1.100 from 0.0.0.0 eth0
      Unicast reply from 192.168.1.100 [20:0B:C7:A0:32:31] for 192.168.1.100 [20:0B:C7:A0:32:31] 0.810ms
      Sent 1 probes (1 broadcast(s))
      Received 1 response (s)

How Do I Power On the System After Abnormal System Is Powered Off?

Symptom

The system is powered on after it is abnormally powered off. Services need to be recovered.

NOTE:
Impact of the Operation

The service deployment system can automatically start the services after all the server OSs in the system are powered on (you can power on the servers in any sequence).

Procedure

Power on all servers and there is no requirement on the power-on sequence of the servers.

How Do I Rectify the Service Faults on the Active and Standby Nodes Caused by Power Failure?

Symptom

When a power failure occurs, the active/standby switchover of ManageOne may be abnormal due to file exceptions. As a result, the services on the active and standby nodes are faulty.

Procedure
  1. Use a browser to log in to the ManageOne deployment plane.

    URL: https://Floating IP address of the deployment plane:31943, for example, https://192.168.0.1:31943.

    Default account: admin; default password: Huawei12#$

  2. Choose Deployment > Microservice Deployment > Environments from the main menu.
  3. If the operation plane is faulty, enter ERService/Global in the search box and press Enter.
  4. If the OM plane is faulty, enter ERService/OMGlobal in the search box and press Enter.
  5. In the search result list, click the environment name whose the environment alias is ERService.
  6. On the displayed page, click Deploy under Deployment Actions in the right pane.
  7. On the displayed microservice deployment page, click OK and wait until the task is complete.
  8. Choose Deployment > Microservice Deployment > Tasks from the main menu.
  9. On the Tasks page, check the deployment status of installed microservices.

    • If the status of a microservice is displayed as , the microservice has been installed successfully and the service process has been started.
    • If the status of a microservice is displayed as , uninstall the microservice and deploy it again.

How Do I Handle Disk Exceptions When VMs Are Powered On and Off Abnormally?

Symptom

VM disk exceptions occur due to abnormal power-off, typically, forcible VM stop/restart and unexpected storage disconnection.

Description

Perform operations provided in this section if information similar to the following is displayed during the startup:

systemd-fsck[605]: /dev/sda2: UNEXPECTED INCONSISTENCY; RUN fsck MANUALLY.  
systemd-fsck[605]: (i.e., without -a or -p options)  
[ 13.652068] systemd-fsck[605]: fsck failed with error code 4.  
Welcome to emergency mode. Use "systemctl default" or ^D to activate default  mode.  
Give root password for maintenance  
(or type Control-D to continue): 
Procedure
  1. Enter the password of user root and switch to the emergency mode to repair the faulty disk.
  2. Run the following command to rectify the fault:

    fsck -y Disk name

    For example, if the name of the faulty disk is /dev/sdb2, run the fsck -y /dev/sdb2 command.

    The disk is damaged if information similar to the following is displayed:

    fsck.ext4 /dev/sda2  
    e2fsck 1.42.7 (21-Jan-2013)  
    /dev/sda2 is mounted.  
    e2fsck: Cannot continue, aborting. 

    The information indicates that disk /dev/sda2 is being attached and cannot be repaired by running the fsck command. In this case, run the mount -on remount,ro /dev/sda2 command to set the disk to read only, and then run the fsck /dev/sda2 command to repair the disk. After the repair is complete, run the exit command.

  3. Confirm the password of user root for logging in to EulerOS.

    • If the password is forgotten, go to 4.
    • If the password is known, go to 5.

  4. Perform the following steps to restore the password of user root:

    1. On the grub menu, press e to enter the editing mode and type the grub account and password, root and Huawei#12, respectively.
    2. Add init=/bin/bash to the end of linux16 /boot/vmlinuz-****.
    3. Press Ctrl+X to start the shell and enter the single-user mode.
    4. Run the following command to change the mounted file system to the writable mode:

      # mount -no remount,rw /

    5. Run the passwd command and change the password of user root as prompted.

      If SELinux is enabled for the system, run the following command:

      # touch /.autorelabel

      Otherwise, the system cannot be started normally.

      After running the exec /sbin/init command to start the system or running the exec /sbin/reboot command to enter the single-user mode, run the following commands to check files in all /etc/fstab directories:

      mount -no remount,ro /

      umount -a

      /usr/sbin/fsck -AsCy

      exit

  5. If the fault persists, enter the emergency mode and run the systemctl default command to enter the system.

    After logging in to the system, run the dmesg|grep -i error command to check the startup logs. If the error logs still exist, the storage may be abnormal. In this case, Contact technical support for assistance.

How Do I Associate a Restored Node on the OM Plane?

After a node is restored, Service Monitoring does not report data about the node. You need to re-associate it.

Procedure
  1. Choose System > Platform Configuration > Service Monitoring from the main menu.
  2. Choose Monitoring Panel > Service Monitoring.

    On the displayed page, you can view the created ManageOne monitoring card, as shown in Figure 26-2.

    Figure 26-2 Service Monitoring
    NOTE:

    Click to filter the created service monitoring cards by region and service type.

  3. Click in the upper right corner of the ManageOne monitoring card to edit basic information.
  4. Click Next. On the displayed 2.Node Information tab page, select the ManageOne nodes in the Optional node column.

    The selected nodes are displayed in the Selected nodes area.

  5. Click Next. On the displayed 3.Monitoring Template tab page, select the monitoring template of the nodes and associate the nodes with the template.

What Can I Do If Kafka Is Abnormal When MessagingBrokeService Is Restarted After the VM Is Powered Off and On?

Symptom

Kafka is abnormal when MessagingBrokeService is restarted after the VM is powered off and powered on.

Procedure
  1. Use a browser to log in to the ManageOne deployment plane.

    URL: https://Floating IP address of the deployment plane:31943, for example, https://192.168.0.1:31943

    Default username: admin; default password: Huawei12#$

  2. Choose Deployment > Services.
  3. In the search box on the right, enter the service name MessagingBrokeService and click .
  4. Click a search result.

    On the displayed service details page, view all deployed nodes in the server list.

  5. Perform the following steps on all nodes in sequence:

    1. Log in to a node where MessagingBrokeService resides as user sopuser.

      The default password is D4I$awOD7k.

    2. Run the following command to switch to the root user:

      su - root

      The default password is Changeme_123.

    3. Run the following command to check the running status of MessagingBrokeService:

      su ossadm -c ". /opt/oss/manager/bin/engr_profile.sh; ipmc_adm -cmd statusapp -app MessagingBrokeService"

    The following information is displayed:

     msgbrksrv-2-0 msgbrksrv MessagingBrokeService Product cluster 192.166.191.125 11006 RUNNING
    NOTE:

    RUNNING indicates that the running status is normal, and STOPPED indicates that the running status is abnormal.

  6. Run the following command to stop MessagingBrokeService on all nodes that are running properly:

    su ossadm -c ". /opt/oss/manager/bin/engr_profile.sh; ipmc_adm -cmd stopapp -app MessagingBrokeService"

  7. Run the following commands to modify the MessagingBrokeService configuration file on all nodes in sequence:

    sed -i 's/unclean.leader.election.enable=false/unclean.leader.election.enable=true/g' /opt/oss/Product/apps/MessagingBrokeService/kafka/bin/GenerateConfig/Kafka/server.properties

  8. Run the following command to start MessagingBrokeService from abnormal to normal on all nodes in sequence:

    su ossadm -c ". /opt/oss/manager/bin/engr_profile.sh; ipmc_adm -cmd startapp -app MessagingBrokeService"

    NOTE:

    Start the nodes according to the status queried in 5. During the startup, you must start another node after a node is started.

  9. Run the following command to check whether MessagingBrokeService is normal:

    su ossadm -c ". /opt/oss/manager/bin/engr_profile.sh; ipmc_adm -cmd statusapp -app MessagingBrokeService"

    • If yes, run the following command to modify the configuration file of MessagingBrokeService after all services are started:

    sed -i 's/unclean.leader.election.enable=true/unclean.leader.election.enable=false/g' /opt/oss/Product/apps/MessagingBrokeService/kafka/bin/GenerateConfig/Kafka/server.properties

    • If no, contact technical support for assistance.

  10. Run the following command to stop MessagingBrokeService on all nodes in sequence:

    su ossadm -c ". /opt/oss/manager/bin/engr_profile.sh; ipmc_adm -cmd stopapp -app MessagingBrokeService"

  11. Run the following command to start MessagingBrokeService on all nodes in sequence:

    su ossadm -c ". /opt/oss/manager/bin/engr_profile.sh; ipmc_adm -cmd startapp -app MessagingBrokeService"

  12. Run the following command to log out of the root user:

    exit

What Do I Do If Service Parameters Are Abnormal After MOLogStorageService Is Manually Upgraded?

Symptom

After MOLogStorageService is manually upgraded, the service parameter esHeapSize is restored to the default value and is not adjusted based on the environment specifications.

Procedure
  1. Use a browser to log in to the ManageOne deployment plane.

    URL: https://Floating IP address of the deployment plane:31943, for example, https://192.168.0.1:31943

    Default account: admin; default password: Huawei12#$

  2. Choose Deployment > Feature Deployment > Services from the main menu.
  3. In the search box on the right, enter the service name MOLogStorageService and click .
  4. In the search result, click the service name.

    On the service details page that is displayed, view all deployed nodes in the server list.

  5. Perform the following steps to modify the memory of the log center node:

    1. Use PuTTY to log in to the node where MOLogStorageService is deployed.

      Default account: sopuser; default password: D4I$awOD7k

    2. Run the following command to switch to the root user:

      su - root

      The default password is Changeme_123.

    3. Run the following command to switch to the /opt/oss/Product/etc directory:

      cd /opt/oss/Product/etc

    4. Run the following command to create a moLogMem.conf file or open the file:

      vi moLogMem.conf

    5. Press i to check whether the file contains the following content:
      MOSCALE="MO32G"

      Replace 32 as required. For details about the memory of each log center node, see Table 26-34.

      • If the content exists, change 32 to the actual memory of the log center node.
      • If the content dos not exist, insert all content in the file and change 32 to the actual memory of the log center node.
    6. Press Esc and run the following command to save the file and exit:

      :wq!

    7. Check whether the permission and group of the moLogMem.conf file are as follows. If they are not, perform 5.h through 5.i to modify the group and permission.
      - rwxr - x - - - ... ossadm ossgroup ... moLogMem.conf
    8. Run the following command to modify the group:

      chown ossadm:ossgroup moLogMem.conf

    9. Run the following command to modify the file permission:

      chmod 750 moLogMem.conf

    10. If multiple log center nodes need to be modified, repeat 5.a through 5.g on each node.
    11. Run the following command to log out of user root:

      exit

  6. Perform the following operations to check and modify the configuration items:

    1. Use a browser to log in to the ManageOne Deployment Portal.

      URL: https://Floating IP address of the Deployment Portal:31945, for example, https://192.168.0.1:31945

      Default account: admin; default password: Huawei12#$

    2. Choose Deployment > Microservice Deployment > Environments from the main menu.
    3. Enter MOLogStorageService in the search box and press Enter.
    4. Click the displayed environment name and check whether the following configuration item exists.

      • If configuration item exists, check whether the value is correct based on information provided in Table 26-34. If the value is correct, no further action is required. If the value is incorrect, click Deploy in the Deployment Actions area on the upper right of the page and change the value of esHeapSize in the Configuration Item List area.
      • If the configuration item does not exist, click Deploy in the Deployment Actions area on the upper right of the page, click in the upper right corner of the Configuration Item List area, enter esHeapSize in the Name text box, set its value based on information provided in Table 26-34, and click OK.
      Table 26-34 Mapping between the memory and the value of esHeapSize

      Memory (GB)

      Value (MB)

      8

      4096

      16

      7168

      32

      15360

How Do I Handle the LogCenter Log Service Exception?

Symptom

The Run Logs and Management Operation Logs report that the log service is abnormal and cannot be used to search for logs.

Procedure
  1. Use a browser to log in to the ManageOne deployment plane.

    URL: https://Floating IP address of the deployment plane:31943, for example, https://192.168.0.1:31943

    Default account: admin; default password: Huawei12#$

  2. Choose Deployment > Microservices from the main menu. In the search box on the right, enter the MOLogAccessService and click .
  3. Click the MOLogAccessService service and then Deactivate on the right of the service page to stop the MOLogAccessService service.
  4. In the navigation pane, choose Microservices. In the search box on the right, enter MOLogStorageService and click .
  5. Click the MOLogStorageService service and then Deactivate. After the task is complete, click Activate to restart the MOLogStorageService service.
  6. After MOLogStorageService is restarted, wait for 10 to 20 minutes.
  7. Log in to the ManageOne OM plane using a browser.

    Login address: https://Address for accessing the homepage of the ManageOne OM plane:31943, for example, https://oc.type.com:31943.

    Default account: admin; default password: Huawei12#$

  8. Choose System > Management Operation Logs from the main menu. On the displayed page, check whether the log service is abnormal.

    If yes, contact technical support for assistance.

    If no, perform 9.

  9. Choose System > Logs > Run Logs > Cluster Status from the main menu and check the Health in the index. If all the statuses are displayed in green or yellow, perform 10 through 11. If any status is displayed in red, check the status half a day or one day later. If the status is displayed in red after one day, contact technical support for assistance.
  10. Perform 1 through 2.
  11. Click the MOLogAccessService service and then Activate on the right of the service page to restart the MOLogAccessService service.

How Do I Solve the Problem that Services Are Unavailable Due to Excessive Logs Received by the LogCenter?

Context

Services are unavailable due to the LogCenter receives too many run logs of FusionSphere OpenStack or ManageOne. You can set the switch to filter unnecessary run logs.

Procedure
  1. Access the es_indices_mgnt folder in the root directory of the ManageOne_init_tools folder obtained after the installation tool package is decompressed.
  2. Use Notepad++ to open the log_indices_mgnt.conf configuration file, and change the value of action in the runlog_confs field based on whether to receive logs. Save the modification and exit.

    Change the parameter values in bold in the following information. You can set the run log collection switch of ManageOne or FusionSphere OpenStack as required.

    • If the value of action is org, run logs are collected.
    • If the value of action is clean, run logs are not collected.
          "runlog_confs": {
              "mo": {
      	"action": "clean",
      	"org": "templates/org/mo.json",
      	"clean": "templates/without_runlog/mo.json",
      	"dest": "/opt/oss/Product/apps/MOLogCenterService/etc/cslogconfig/"
              },
              "openstack": {
      	"action": "clean",
      	"org": "templates/org/openstack.json",
      	"clean": "templates/without_runlog/openstack.json",
      	"dest": "/opt/oss/Product/apps/MOLogCenterService/etc/cslogconfig/"
              }
          },

  3. Run the run.bat script in the es_indices_mgnt folder and enter the following information as prompted to disable or enable the function of collecting run logs.

    1. If "Please input float_ip for <deploy portal>:" is displayed in the command output, enter the floating IP address of the deployment plane and replace it as required.
    2. If "Please input node password for <sopuser>:" is displayed in the command output, enter the password of the sopuser user. The default password is D4I$awOD7k.
    3. If "Please input node password for <root>:" is displayed in the command output, enter the password of the root user. The default password is Changeme_123.
    4. If "Please input db password for <dbuser>:" is displayed in the command output, enter the password of the dbuser user. The default password is Admin@123.
    5. If "Please choose operation:" is displayed in the command output, enter 0.
    6. If "confirm?(y/n)" is displayed in the command output, enter y.

  4. If information similar to the following is displayed, and the MOLogCenterService service is restarted on all log nodes, the execution is successful:

    Start action get deploy float ip
    ...
    ****exec**** 192.168.126.1 /bin/chown sopuser:wheel /home/sopuser/nodelists.json
    
    get </home/sopuser/nodelists.json> to <nodelists.json>
    ...
    Start action mgnt service
    ********Login 192.168.126.14 Successfully********
    ****exec**** 192.168.126.14 su - root
    ****exec**** 192.168.126.14 su - ossadm -c"/opt/oss/manager/agent/bin/ipmc_adm -
    cmd restartapp -app MOLogCenterService"
    ...
    Start action mgnt service
    ********Login 192.168.126.15 Successfully********
    ****exec**** 192.168.126.15 su - root
    ****exec**** 192.168.126.15 su - ossadm -c"/opt/oss/manager/agent/bin/ipmc_adm -
    cmd restartapp -app MOLogCenterService"

  5. Wait for one hour, log in to the ManageOne OM plane, choose System > Logs > Run Logs > Log Search, and check whether the log collection function of ManageOne or FusionSphere OpenStack takes effect in the run logs list.

    • If the log collection function is disabled, ManageOne or FusionSphere OpenStack logs generated within one hour cannot be found.
    • If the log collection function is enabled, ManageOne or FusionSphere OpenStack logs generated within one hour can be found.

Translation
Download
Updated: 2019-06-10

Document ID: EDOC1100063248

Views: 22650

Downloads: 37

Average rating:
This Document Applies to these Products
Related Documents
Related Version
Share
Previous Next