No relevant resource is found in the selected language.

This site uses cookies. By continuing to browse the site you are agreeing to our use of cookies. Read our privacy policy>Search

Reminder

To have a better experience, please upgrade your IE browser.

upgrade

FusionInsight HD 6.5.0 Software Installation 02

Rate and give feedback:
Huawei uses machine translation combined with human proofreading to translate this document to different languages in order to help you better understand the content of this document. Note: Even the most advanced machine translation cannot match the quality of professional translators. Huawei shall not bear any responsibility for translation accuracy and it is recommended that you refer to the English document (a link for which has been provided).
Preparing the Environment

Preparing the Environment

Manually Creating a Partition

Scenario

If multiple RAID levels exist on a node except the OS disk (for example, when the NameNode and DataNode are deployed on the same server, RAID 1 for medata disks and non-RAID for data disks exist on a node at the same time), using preinstall to format partitions may make medata partitions be configured on the disk with non RAID.

This topic describes how to manually create a partition when multiple RAID levels exist and run preinstall to install the diskmgt process for disk monitoring after creating a partition and install the RPMs.

Prerequisites

The image of OS has been mounted on active management node.

Procedure
  1. Create a directory to which a disk is mounted.

    1. Run the following command to create a directory to which a disk is mounted:

      mkdir -p directory to which the disk is mounted

      For example, create a NameNode partition and ensure that data is stored in /srv/BigData:

      mkdir -p /srv/BigData/namenode

    2. Run the following command to change the directory permission:

      chmod 000 directory to which the disk is mounted

      For example,

      chmod 000 /srv/BigData/namenode

    3. Run the following command to change the directory owner to root:root:

      chown root:root directory to which the disk is mounted

      For example,

      chown root:root /srv/BigData/namenode

  2. Format a planned disk.

    1. Partition a specified disk (the GPT format that supports disks larger than 2 TB is recommended). /dev/sdX indicates the block device name of the disk, for example, /dev/sdc and /dev/sdd.

      parted -s /dev/sdX mklabel gpt

    2. Create a partition.

      parted -s /dev/sdX mkpart logic xxM 100%

      xxM indicates the start position of the partition. It is recommended that the partition size starts from 100 MB. 100% indicates that all free space of the disk is allocated to the partition. Example:

      parted -s /dev/sdX mkpart logic 100M 100%

      If multiple partitions are required, plan the partitions based on the disk.

    3. Format a disk. /dev/sdXx indicates the partition name of disk /dev/sdX, for example, /dev/sdc1, /dev/sdc2, and /dev/sdd1.
    NOTE:

    If you need to manually format disk partitions, choose a correct file system format based on your OS as follows:

    • OS disk file system of the SUSE 12.X: btrfs
    • Data disk of the SUSE 12.X: xfs
    • Data directory of the Red Hat 7.X/CentOS 7.X/EulerOS 2.X OS: xfs
    • Data directory of the SUSE 11.X OS: ext3
    • Data directory of the Red Hat 6.X/CentOS 6.X: ext4

    For SUSE 11.X series OSs, run the following command:

    mkfs.ext3 /dev/sdXx

    For Red Hat 6.X/CentOS 6.X series OSs, run the following command:

    mkfs.ext4 /dev/sdXx

  3. Mount a disk.

    1. Run the following command to refresh the OS partition table:

      partprobe

    2. Run the following command to obtain the UUID of the new partition:

      blkid /dev/sdXx

    3. Modify /etc/fstab and add the following statement to /etc/fstab (SUSE: ext3; Red Hat and CentOS: ext4):
      UUID=XXXXXXXXXXXXXXXXXXXXXXX directory to which the disk is mounted ext4 defaults,noatime,nodiratime 1 0
    4. Run the following command to mount a disk:

      mount -a

    5. Run the following command to change the directory owner to xxx:xxxxx:

      chown 2000:wheel directory to which the disk is mounted

  4. Log in to the active management node as user root and run the preinstall script.

    1. Modify the following parameters. For details, see Configuring and Checking the Installation Environment.
      • Ensure that g_hosts retains only nodes with multiple RAID levels. For example, multiple RAID levels exist on hosts 2 and 3.
      • Set g_parted to 3.
      • Ensure that g_parted_conf retains only nodes with multiple RAID levels. For example, multiple RAID levels exist on hosts 2 and 3.
      Example:
      g_hosts="192.168.10.[12,13]" 
      g_user_name="root" 
      g_port=22 
      g_parted=3g_parted_conf="192.168.10.12:host2.ini;192.168.10.13:host3.ini;" 
      g_add_pkg=1 
      g_pkgs_dir="redhat-6.4:/media/" 
      g_log_file="/tmp/fi-preinstall.log" 
      g_debug=0 
      g_hostname_conf="192.168.10.10:192.168.20.10:host0;192.168.10.11:192.168.20.11:host1;192.168.10.12:192.168.20.12:host2;192.168.10.13:192.168.20.13:host3;192.168.10.14:192.168.20.14:host4;192.168.10.15:192.168.20.15:host5;192.168.10.16:192.168.20.16:host6;192.168.10.17:192.168.20.17:host7;192.168.10.18:192.168.20.18:host8;192.168.10.19:192.168.20.19:host9;192.168.10.20:192.168.20.20:host10;" 
      g_swap_off=1
    2. Run the preinstall script.

Manually Installing the Disk Management Service

Scenario

When preparing the OS manually, the installation engineer may forget to execute preinstall. As a result, the diskmgt process (disk management service) is not installed or uninstalled. To manage node disks, you must manually install the process.

After you manually install the disk management service, the service can identify and manage only the metadata disk and data disk partitions of FusionInsight.

Impact on the System

The diskmgt process is installed on the node OS and the disk management configuration file is generated. The process starts after the installation.

Prerequisites
  • The password of user root for logging in to the current node has been obtained.
  • Logical Volume Manager (LVM), software RAID, and encrypted partitions are not supported.
  • The name of a data directory must comply with the data directory plan of the product.

    For example, if the root directory of the data directory is /srv/BigData, the name of the NameNode data directory must be /srv/BigData/namenode.

  • If multiple data directories share a disk, all partitions of the disk must be used by the FusionInsight data directory or metadata directory.

    For example, disk /dev/sdd has three partitions /dev/sdd1, /dev/sdd2, and /dev/sdd3. It is not supported that /dev/sdd1 is mounted to /srv/BigData/namenode, /dev/sdd2 is mounted to /srv/BigData/zookeeper, and /dev/sdd3 is mounted to /opt.

Procedure
  1. Obtain the FusionInsight SetupTool software package, upload it to the node where the service is to be installed, and decompress the package.

    For details about how to obtain the software package, see Preparing Tools and Software.

  2. Log in to the node where the service is to be installed as user root.
  3. Go to the directory where the Setup software package is decompressed, for example, /opt/FusionInsight_SetupTool.

    cd /opt/FusionInsight_SetupTool

  4. Run the following command to install the diskmgt process:

    sh preinstall/tools/genConfig/genConfigAndInstallDiskmgt.sh

    The installation is successful if the following information is displayed:

    [INFO] [none:795] execute diskmgt.sh successfully. 
    [INFO] [main:608] Execute genConfigAndInstallDiskmgt.sh successfully.
    NOTE:
    • By default, the process can be installed on a single node. For details about how to install the process on multiple nodes, see How to Run Commands or Access Files on Multiple Nodes in a Cluster.
    • FusionInsight SetupTool 6.5.0 allows diskmgt installation in clusters of historical versions. After the installation, the diskmgt process can work correctly in the clusters.

How to Configure the Capacity of RAID1

When you create the RAID1, restrict the capacity of RAID1 to minimize the re-establishment duration of RAID1. It is recommended that the capacity of RAID1 be restricted to 600 GB. With this restriction, the re-establishment duration of RAID1 when a fault occurs is reduced to four hours.

If an LSI SAS 2208 RAID card is used, set the Select Size parameter when you create a virtual disk.

If a command line tool is used to create the RAID1, specify the capacity of RAID1 using a parameter contained in the RAID creation command. For example, if LSI SAS 2308 RAID card is used, use the size parameter to set the capacity of RAID1 when running the sas2ircu create command.

How to Configure Disk Partitions When the Disk Capacity Is Insufficient

This section describes how to configure disk partitions when the disk capacity (300 GB) of some nodes in a cluster is smaller than the standard capacity (600 GB).

It is intended for the OS disk and metadata disk of each node in the cluster, excluding the Hadoop data disk, Solr data disk, Kafka data disk, Storm data disk, and Redis data disk.

When a cluster contains more than 100 nodes, each metadata disk must have at least 600 GB space.

Requirements for OS Disk Partitions
  • OS Partitions on Active/Standby Management Nodes
    Table 7-12 OS partitions on active/standby management nodes

    Partition

    Minimum Capacity Requirement

    Impact

    /

    20 GB

    None

    /tmp

    10 GB

    None

    /var

    10 GB

    None

    /var/log

    50 GB

    None

    /srv/BigData

    60 GB

    None

    /opt

    150 GB

    If the backup data of the NameNode is stored in /srv/BigData/LocalBackup, its amount will not be affected.

    If the backup data of the NameNode is stored in /opt/huawei/Bigdata/LocalBaukup, its amount will be affected. The installation of FusionInsight requires at least 63 GB space. After a single HDFS metadata file is backed up, the file is compressed and dumped in /opt of the active and standby management nodes. The file size is about 1.5 GB (mapping to 100 million files). If 24 backup files are reserved, 36 GB disk space is used by default, and the earliest backup files are overwritten after the space is used up. The number of backup files can be adjusted to prevent full disk occupation.

    Adjustment scheme:

    After the cluster is installed, choose O&M > Backup and Restore > Backup Management, click Config in the default queue, and adjust Maximum number of backup copies in Configuration > Metadata > NameNode.

  • OS Partitions on Non-Active/Standby Management Nodes
    Table 7-13 OS partitions on non-active/standby management nodes

    Partition

    Minimum Capacity Requirement

    Impact

    /

    20 GB

    None

    /tmp

    10 GB

    None

    /var

    10 GB

    None

    /var/log

    50 GB

    The following rules apply to control nodes (include active/standby nodes) where the ResourceManager and NameNode roles reside. Calculate the required capacity (at least 50 GB) proportionally based on the cluster scale and the log retention duration.

    To retain logs of 15 days for a cluster of 200 nodes, configure at least 200 GB capacity.

    To retain logs of 30 days for a cluster of 200 nodes, configure at least 400 GB capacity.

    NOTE:

    If the number of cluster nodes is overlarge, and the log retention duration is long, configure a disk with larger capacity or mount the /var/log partition independently.

    /srv/BigData

    60 GB

    None

    /opt

    150 GB

    None

Requirements for Metadata Disk Partitions

Metadata disk partitions can be generated automatically by running the preinstall command. For details, see Configuring and Checking the Installation Environment.

  • Metadata Disk Partitions on Management Nodes
    Table 7-14 Metadata disk partitions on management nodes

    Partition

    Minimum Capacity Requirement

    Impact

    /srv/BigData/dbdata_om

    300 GB

    It is recommended that the partition occupies a disk independently.

    The monitoring data increases linearly as the number of nodes increases. When the space usage exceeds the threshold, data files are overwritten circularly.

    The capacity calculation formula is as follows:

    Disk capacity x 80% = 2.5 GB x Number of nodes x Retention duration

    The disk usage threshold is 80%; The unit of the retention duration is month; The amount of monitoring data generated in one month will reach 80% of the peak monitoring data. If 500 nodes are deployed, a 300 GB disk is sufficient for storing data generated in six days only.

    /srv/BigData/LocalBackup

    300 GB

    It is recommended that the partition occupies a disk independently.

    The number of NameNode metadata backups will be affected if the partition capacity decreases. In full configuration (the size of a metadata backup file after compression is about 1.5 GB, mapping to 100 million files), files are backed up once every 30 minutes. By default, 36 GB disk space is required for storing 24 backup files. The earliest backup files are overwritten automatically after the space is used up.

  • Metadata Disk Partitions of Control Nodes
    Table 7-15 Metadata disks of control nodes

    Partition

    Minimum Capacity Requirement

    Impact

    /srv/BigData/zookeeper

    50 GB

    It is recommended that the partition occupies a disk independently.

    None

    /srv/BigData/dbdata_service

    50 GB

    It is recommended that the partition occupies a disk independently.

    None

    /srv/BigData/journalnode

    50 GB

    It is recommended that the partition occupies a disk independently.

    None

    /srv/BigData/namenode

    300 GB

    It is recommended that the partition occupies a disk independently.

    None

    /srv/BigData/storm

    100 GB

    It is recommended that the partition occupies a disk independently.

    None

Configuring the preinstall Script When Some Disks of a Node Are SSDs

Symptom

If some disks of a node are SSDs, you are advised to first mount the metadata partitions, such as /srv/BigData/dbdata_om, /srv/BigData/namenode, and /srv/BigData/zookeeper. (The data storage paths can be customized. Default value /srv/BigData is used as an example.) To ensure correct partition mounting, before executing preinstall, manually configure the configuration file of node partitions.

NOTE:

If all the disks of the node are SSDs, skip this section.

Prerequisites

The preinstall configuration file has been generated using the Configuration Planning Tool based on the planned data and has been uploaded to the specified location.

Procedure
  1. Go to the directory where the software installation script tool package is decompressed, for example, /opt/FusionInsight_SetupTool.
  2. Configure and modify configuration files of relative partitions.

    For example, if node host1 has three disks except the system disks (one of the three disks is SSD and the disk identifier is /dev/hioa), mount the SSD disks to the /srv/BigData/zookeeper partition, and mount the other two disks to the /srv/BigData/journalnode and /srv/BigData/hadoop/data1 partitions.

    Edit the /opt/FusionInsight_SetupTool/preinstall/partition/ini/host1.ini file as follows:

    #mount                required  care  condition 
    journalnode.conf      y         y     n 
    zookeeper.conf        y         y     zookeeper_condition.sh 
    datanode1.conf        y         y     n

    Edit the /opt/FusionInsight_SetupTool/preinstall/partition/ini-plugin/condition/zookeeper_condition.sh file to the specify the mounting partition.

    The following uses the Huawei ES3000 V2 PCIe SSD card to describe how to configure the XXX_condition.sh file:

    NOTE:

    Different SDD cards use different drive commands and have different names, configure configuration files based on actual conditions. The following example is used for reference only.

    1. The ES3000 V2 PCIe SSD driver enables the hio_info command to query the SSD card information.

      Run the hio_info command. The system displays the following:

      hioa            Serial number:              
                      Size(GB):              3200 
                      Max size(GB):          3200 
                      Hardware version:      2.0 
                      Firmware version:      615 
                      Driver version:        2.1.0.6 
                      Work mode:             MLC 
                      Run time (sec.):       357749 
                      Total  read(MB):       40388700 
                      Total write(MB):       40389258 
                      Lifetime remaining:    98.202% 
                      Max bad block rate:    0.113% 
                      Health:                OK 
                      Comment:               NA
    2. Based on the preceding information, edit the following content of the zookeeper_condition.sh file.
      #!/bin/bash 
       
      DEV="$1" 
      If [ -z "${DEV}" ]; then 
               exit 1 
      fi 
       
      which hio_info > /dev/null 2>&1 
      if [ $? -ne 0 ]; then 
               echo "can not find command hio_info." 
               exit 1 
      fi 
       
      hio_info | grep -q "^${ DEV }" 
      if [ $? -ne 0 ]; then 
               #not SSD 
               exit 1 
      fi 
       
      #find SSD 
      exit 0
      • Input parameter

        $1: specifies the disk identifier, for example sda, sdb, and hioa.

      • Returned value
        • 0 indicates that the file is configured successfully.
        • Other values indicate that the file fails to be configured.

  3. Execute the preinstall script and configure and check the installation environment.

    For details, see section Configuring and Checking the Installation Environment.

Download
Updated: 2019-05-17

Document ID: EDOC1100074555

Views: 6667

Downloads: 6

Average rating:
This Document Applies to these Products
Related Version
Related Documents
Share
Previous Next