No relevant resource is found in the selected language.

This site uses cookies. By continuing to browse the site you are agreeing to our use of cookies. Read our privacy policy>Search

Reminder

To have a better experience, please upgrade your IE browser.

upgrade

FusionInsight HD 6.5.0 Software Installation 02

Rate and give feedback:
Huawei uses machine translation combined with human proofreading to translate this document to different languages in order to help you better understand the content of this document. Note: Even the most advanced machine translation cannot match the quality of professional translators. Huawei shall not bear any responsibility for translation accuracy and it is recommended that you refer to the English document (a link for which has been provided).
Preparing OS

Preparing OS

OS Version Requirements

Ensure that the OS specified by Table 2-8 is installed on each server. To obtain the software, visit the official website. The OS requirements for each FusionInsight HD node are as follows:

  • All nodes use the same type of OS. For details about supported OSs, see Table 2-8. (You are advised to install the recommended OS.)
  • The OS is newly installed on each node. (Minimal OS is not supported.)
  • The password of user root must be consistent on each node.
  • The kernel version of the SUSE OS cannot be 3.0.101-0.40-default (you can run the uname -r command to view the version).
  • FusionInsight HD uses cgroup to implement resource isolation. If a user's application uses cgroup and a subsystem is mounted, ensure that user omm has read and write permission on the directory where the subsystem is mounted. Currently, SUSE 11.1 does not support resource isolation using blkio cgroup, you are advised to install the recommended OS.
  • The OS supports NTP 4.2.8p11 or earlier, except 4.2.8p3 and 4.2.8p4. For the SUSE OS, NTP 4.2.6 is not supported (you can run the rpm -qa | grep ntp command to view the version).
  • SSH v1.x has security risks. You are advised to configure SSH v2 or later (In the command output of cat /etc/ssh/sshd_config | grep Protocol, the value of Protocol indicates the SSH version supported by the OS. If the value is 2, the OS supports SSH v2.)
  • If the OS OpenLDAP version is earlier than 2.4.40 (you can run the rpm -qa | grep openldap command to view the version), LDAP cannot execute TLSv1.1, TLSv1.2, and secure cipher suites. The OS has security risks. You are advised to install the recommended OS.
  • If the OS OpenSSL version is earlier than 1.0.1 (you can run the rpm -qa | grep openssl command to view the version), LDAP cannot execute TLSv1.1, TLSv1.2, and secure cipher suites. The OS has security risks. You are advised to install the recommended OS.
  • Use the default port No. (22) for the SSH port of each node. If the port No. is modified, functions, such as cluster creation, service/instance addition, host addition, or host reinstallation, cannot be used.
Table 2-8 FusionInsight HD-depended OS

Server Type

OS Software

Supported Version

Universal x86 servers

SUSE

  • Recommended: SUSE Linux Enterprise Server 11 SP4 (SUSE 11.4)
  • Recommended: SUSE Linux Enterprise Server 12 SP1 (SUSE 12.1) (If you use this OS version, install the SUSE-SLE-SERVER-12-SP1-2016-930 patch provided by SUSE. Otherwise, NTP waiting times out and FusionInsight Manager installation fails.)
  • Available: SUSE Linux Enterprise Server 11 SP1 (SUSE 11.1)
  • Available: SUSE Linux Enterprise Server 11 SP2 (SUSE 11.2)
  • Available: SUSE Linux Enterprise Server 11 SP3 (SUSE 11.3)
  • Available: SUSE Linux Enterprise Server 12 (SUSE 12) (If you use this OS version, install the SUSE-SLE-SERVER-12-2016-933 patch provided by SUSE. Otherwise, the time synchronization between the active management node and other nodes fails.)
  • Available: SUSE Linux Enterprise Server 12 SP2 (SUSE 12.2)
  • Available: SUSE Linux Enterprise Server 12 SP3 (SUSE 12.3)

Red Hat

  • Recommended: RedHat-6.6-x86_64 (Red Hat 6.6)
  • Recommended: RedHat-7.2-x86_64 (Red Hat7.2)
  • Available: RedHat-6.4-x86_64 (Red Hat 6.4)
  • Available: RedHat-6.5-x86_64 (Red Hat 6.5)
  • Available: RedHat-6.7-x86_64 (Red Hat 6.7)
  • Available: RedHat-6.8-x86_64 (Red Hat 6.8)
  • Available: RedHat-6.9-x86_64 (Red Hat 6.9)
  • Available: RedHat-7.0-x86_64 (Red Hat 7.0)
  • Available: RedHat-7.1-x86_64 (Red Hat 7.1)
  • Available: RedHat-7.3-x86_64 (Red Hat 7.3)
  • Available: RedHat-7.4-x86_64 (Red Hat 7.4) (If you use this OS version, install systemd-219-57.el7 provided by Red Hat. Otherwise, systemd may fail to provide services, causing system resources unavailable.)
  • Available: RedHat-7.5-x86_64 (Red Hat 7.5)
    NOTE:

    Elk does not support the Red Hat 7.5.

CentOS

  • Available: CentOS-6.4 (CentOS 6.4)
  • Available: CentOS-6.5 (CentOS 6.5)
  • Available: CentOS-6.6 (CentOS 6.6)
  • Available: CentOS-6.7 (CentOS 6.7)
  • Available: CentOS-6.8 (CentOS 6.8)
  • Available: CentOS-6.9 (CentOS 6.9)
  • Available: CentOS-7.0 (CentOS 7.0)
  • Available: CentOS-7.1 (CentOS 7.1)
  • Available: CentOS-7.2 (CentOS 7.2)
  • Available: CentOS-7.3 (CentOS 7.3)
  • Available: CentOS-7.4 (CentOS 7.4)
  • Available: CentOS-7.5 (CentOS 7.5)
    NOTE:

    Elk does not support the CentOS 7.5.

Euler

  • Available: EulerOS V2.0SP1 (EulerOS2.1)
  • Available: EulerOS V2.0SP2 (EulerOS2.2)
  • Available: EulerOS V2.0SP3 (EulerOS2.3)
  • Available: EulerOS V2.0SP5 (EulerOS2.5)
    NOTE:

    Elk only support the EulerOS V2.0SP3 system.

NeoKylin

  • Available: NeoKylin-6.9
  • Available: NeoKylin-7.2
  • Available: NeoKylin-7.4
  • Available: NeoKylin-7.5
    NOTE:

    Elk does not support the NeoKylin system.

Oracle

  • Available: Oracle Linux 7.4
  • Available: Oracle Linux 7.5
  • Available: Oracle Linux 7.6
    NOTE:

    Elk does not support the Oracle system.

Huawei TaiShan ARM server

NOTE:

Elk only support the RedHat-7.5-aarch64 system.

Red Hat

  • Available: RedHat-7.4-aarch64 (RedHat7.4) (If you use this OS version, install systemd-219-57.el7 provided by Red Hat. Otherwise, systemd may fail to provide services, causing system resources unavailable.)
  • Available: RedHat-7.5-aarch64 (RedHat7.5)

CentOS

  • Available: CentOS-7.4-aarch64 (CentOS7.4) (If you use this OS version, install systemd-219-57.el7 provided by Red Hat. Otherwise, systemd may fail to provide services, causing system resources unavailable.)
  • Available: CentOS-7.5-aarch64 (CentOS7.5)

Euler

Available: EulerOS V2.0SP3 (EulerOS2.3)

Hard Disk Partition Requirements

In the FusionInsight HD system, disks are classified into the following types:

  • OS disk: OS disks locate on all nodes and are used to store the OS of each node. OS disks have fixed partition requirements.
  • Metadata disk: Metadata disks include the metadata disks of management nodes and control nodes, which are located on management nodes and control nodes. Metadata disks are used to store metadata of FusionInsight HD, including management process data and control process data.
  • Hadoop data disk: Hadoop data disks are located on data nodes and used to store Hadoop data of FusionInsight HD.
  • Solr data disk: Solr data disks are located on data nodes and used to store Solr metadata and index data.
  • Elasticsearch data disk: Elasticsearch data disks are located on data nodes and used to store Elasticsearch metadata and index data.
  • Kafka data disk: Kafka data disks are located on data nodes and used to store Kafka data.
  • Storm data disk: Storm data disks are located on data nodes and used to store NameNode Supervisor and workers data.
  • Redis data disk: Redis data disks are located on data nodes and used to store Redis database data.
NOTE:

If you need to manually format disk partitions, choose a correct file system format based on your OS as follows:

  • OS disk file system of the SUSE 12.X: btrfs, data disk of the SUSE 12.X: xfs
  • Data directory of the Red Hat 7.X/CentOS 7.X/EulerOS 2.X OS: xfs
  • Data directory of the SUSE 11.X OS: ext3
  • Data directory of the Red Hat 6.X/CentOS 6.X: ext4

If some disks of a node are SSDs, you are advised to first mount the metadata partitions, such as /srv/BigData/dbdata_om, /srv/BigData/namenode, and /srv/BigData/zookeeper and perform the configuration. For details, see Configuring the preinstall Script When Some Disks of a Node Are SSDs.

Because the actual services and maintenance scenarios in the deployment environment are different, the minimum disk partition capacity needs to be increased in a timely manner to avoid insufficient disk space. For example, to capture dump information, you need to increase the /var space.

Partition requirements of all types of disks are as follows:

  • OS disk partition requirements

    When installing the OS, partition the OS disk (of which the drive letter is generally sda) based on Table 2-9. Other hard disks (with the drive letter sdN, and N can be any letter except a) are in Free status.

    NOTE:

    You can refer to the following disk partition information when the disk capacity is 600 GB or larger. When the disk capacity is smaller than 600 GB (300 GB), configure the disk partitions based on How to Configure Disk Partitions When the Disk Capacity Is Insufficient.

    Table 2-9 OS disk partition information

    Type of Node

    Partition Directory

    Minimum Capacity Requirements

    Usage

    Management node, control node, and data node

    OS

    /

    20 GB

    Root partition of the OS, which contains all directories (excluding the subsequent specified directories).

    /tmp

    10 GB

    Directory for storing temporary files

    /var

    10 GB

    Directory containing files to which the OS writes data during operation

    /var/log

    To customize the log retention duration for clusters of different scale, you need to modify the parameter configuration items of the HDFS, HBase, and Yarn services in the cluster. For details, see the FusionInsight HD 6.5.0 Parameter Configuration Description.

    The following uses a cluster of 200 nodes as an example.

    • If a cluster with 200 nodes has the requirements of 15-day log retention, it is recommended that the partition capacity be greater than or equal to 200 GB.
    • If a cluster with 200 nodes has the requirements of 30-day log retention, it is recommended that the partition capacity be greater than or equal to 400 GB.
    NOTE:
    • Calculate the capacity that is required by the ratio in the preceding example based on the cluster scale and log retention duration. (The capacity must be greater than or equal to 130 GB.)
    • If the volume of the logs to be reserved exceeds 200 GB (for example, the volume of 15-day logs to be reserved on 200 nodes is 200 GB), you are advised to mount the /var/log partition to an independent disk (configure RAID 1 for the disk).
    • The configuration rule mainly applies to control nodes (including the active and standby nodes) where the ResourceManager and NameNode roles exist. For other nodes, the partition capacity must be greater than or equal to 130 GB.

    Directory for storing logs

    When the remaining space of the directory on the active management node is less than 10 GB, audit logs generated on other nodes cannot be automatically backed up.

    Partition where the data storage directory locates.

    For example, /srv/BigData

    60 GB

    Data directory of FusionInsight Manager that also provides mount points for component data directories.

    NOTE:
    • You can change this directory to another one, for example, /srv/fi. After this directory is changed, the parent directories of other metadata and data partitions will be changed.
    • You can specify a new directory in Basic Configuration > Data path in the FusionInsight Configuration Planning Tool.
    • The path can contains only slash (/), letters (a to z and A to Z), digits (0 to 9), hyphens (-). and underscodre (_). The path can contain at most 128 characters.
    • Soft links are not mandatory when you customizing data storage path.

    Partition where the software installation directory locates.

    For example, /opt

    All remaining space of the OS disk is allocated to the /opt partition. It is recommended that the total OS disk capacity be greater than or equal to 600 GB and the /opt partition capacity be greater than or equal to 300 GB.

    NOTE:

    If the /srv/BigData/LocalBackup partition is planned for the active and standby management nodes, metadata file backups of HDFS are stored in this partition. Otherwise, the metadata file backups are stored under the /opt/huawei/Bigdata/LocalBaukup directory. The size of a backup file after compression is about 1.5 GB (mapping to 100 million files). By default, 24 backup files are retained, which uses 36 GB disk space.

    Directory for storing programs (such as the FusionInsight HDsoftware installation package and the OS ISO file).

    NOTE:
    • FusionInsight HD is installed in the /opt/huawei/Bigdata directory by default. You can change the installation directory.
    • If a user-defined installation path is used but the installation path is not under /opt, create a partition for the installation path and ensure that the partition capacity is not less than 300 GB. If a hard disk is mounted to the partition, RAID 1 must be configured.
    • You can specify a new directory in Basic Configuration > Software installation path in the FusionInsight Configuration Planning Tool.
    • The path can contains only slash (/), letters (a to z and A to Z), digits (0 to 9), hyphens (-). and underscodre (_). The path can contain at most 128 characters. If Elk needs to be installed, the path can contain a maximum of 60 characters.
    • The customized installation directory must be configured with an independent partition, and the directory must be empty. Do not copy any file to this directory manually after the cluster is installed.
    • Soft links are not mandatory when you customizing software installation path.
  • Metadata disk partition requirements
    NOTE:

    Metadata disk partitions will be automatically generated during Configuring and Checking the Installation Environment. Configure RAID for management and control nodes according to partition requirements and ensure that each node has sufficient disks.

    Table 2-10 Metadata disk partition requirements

    Type of Node

    Partition Directory

    Minimum Capacity Requirements

    Usage

    Management node

    Manager DB

    /srv/BigData/dbdata_om

    Uses a disk drive exclusively. If there are more than 1000 nodes in a cluster, use SSDs with the NVMe or PCIe interface. If there are less than 2000 nodes, the capacity must be greater than or equal to 600 GB. If there are more than 2000 nodes, the capacity must be greater than or equal to 1 TB.

    Directory for storing service data of the PG database on the management node.

    If Manager is to be installed on two nodes, at least two nodes contain this partition directory.

    LocalBackup

    /srv/BigData/LocalBackup

    Uses a disk drive exclusively. If there are less than 2000 nodes, the capacity must be greater than or equal to 600 GB. If there are more than 2000 nodes, the capacity must be greater than or equal to 1 TB.

    If the cluster data is backed up in LocalDir, the backup data is stored the same directory by default.

    If Manager is to be installed on two nodes, at least two nodes contain this partition directory.

    Control node

    DBService

    /srv/BigData/dbdata_service

    Uses a disk drive exclusively. If there are more than 1000 nodes in a cluster, use SSDs with the NVMe or PCIe interface. If there are less than 2000 nodes, the capacity must be greater than or equal to 600 GB. If there are more than 2000 nodes, the capacity must be greater than or equal to 1 TB.

    Directory for storing service data of the PG database of the component.

    At least two nodes contain this partition directory.

    Zookeeper

    /srv/BigData/zookeeper

    Uses a disk drive exclusively. The recommended capacity is greater than or equal to 600 GB.

    Directory for storing ZooKeeper data.

    At least three nodes contain this partition directory. If you want to add nodes, ensure that the quantity is an odd number.

    Journalnode

    /srv/BigData/journalnode

    Uses a disk drive exclusively. If there are more than 100 million files in HDFS, use SSDs with the NVMe or PCIe interface. The recommended capacity is greater than or equal to 600 GB.

    Directory for storing JournalNode data of the Hadoop Distributed File System (HDFS)

    At least three nodes contain this partition directory. If you want to add nodes, ensure that the quantity is an odd number.

    Namenode

    /srv/BigData/namenode

    Uses a disk drive exclusively. If there are more than 100 million files in HDFS, use SSDs with the NVMe or PCIe interface. The recommended capacity is greater than or equal to 600 GB.

    Directory for storing NameNode data.

    At least two nodes contain this partition directory.

    Storm

    /srv/BigData/storm

    Uses a disk drive exclusively. If there are less than 2000 nodes, it is recommended that the capacity be greater than or equal to 600 GB. If there are more than 2000 nodes, it is recommended that the capacity be greater than or equal to 1 TB. SSDs are recommended for clusters with more than 1000 nodes.

    Directory for storing Storm Nimbus metadata.

    At least two nodes contain this partition directory.

  • Hadoop data disk partition requirements
    NOTE:
    • Hadoop data disk partitions will be automatically generated during Configuring and Checking the Installation Environment. Configure RAID for data nodes according to partition requirements and ensure that each node has sufficient disks.
    • If the remaining capacity of a dataN partition is smaller than 10 GB, this partition will not be selected by HDFS for data write.

    The data node Hadoop partition information is listed in Table 2-11. Each data node has at least one data disk drive. Configure the partition for each data node based on the actual requirements.

    Table 2-11 Hadoop data disk partition information

    Type of Node

    Partition Directory

    Minimum Capacity Requirements

    Usage

    Data node

    Data1 to N

    /srv/BigData/hadoop/data1

    Uses a disk drive exclusively. The recommended capacity is greater than or equal to 600 GB.

    Directory for storing DataNode data and intermediate MapReduce data.

    At least three nodes contain this partition directory.

    ...

    ...

    ...

    /srv/BigData/hadoop/dataN (N≥1)

    Uses a disk drive exclusively. The recommended capacity is greater than or equal to 600 GB.

    Directory for storing DataNode data and intermediate MapReduce data.

    At least three nodes contain this partition directory.

    NOTE:

    To install Elk in FusionInsight HD, plan space for Elk backup data and xlog files based on the following formula:

    Space = Original service volume x Number of copies (3 by default) x 2.

  • Solr data disk partition requirements
    NOTE:

    Solr data disk partitions will be automatically generated during Configuring and Checking the Installation Environment. Configure RAID for data nodes according to partition requirements and ensure that each node has sufficient disks.

    Table 2-12 describes Solr data disk partition information of data nodes. Configure the actual number according to user requirements.

    Table 2-12 Solr data disk partition information

    Node Type

    Partition Directory

    Minimum Capacity Requirements

    Usage

    Data node

    SolrServerAdmin

    /srv/Bigdata/solr/solrserveradmin

    Uses a disk drive exclusively. The recommended capacity is greater than or equal to 600 GB.

    Directory for storing Solr metadata and index data.

    Two nodes contain this partition directory.

    SolrServer1

    /srv/BigData/solr/solrserver1

    Uses a disk drive exclusively. The recommended capacity is greater than or equal to 600 GB.

    Directory for storing Solr metadata and index data.

    Plan partitions according to instance deployment. For details, see Service Deployment Scheme.

    SolrServer2

    /srv/BigData/solr/solrserver2

    Uses a disk drive exclusively. The recommended capacity is greater than or equal to 600 GB.

    SolrServer3

    /srv/BigData/solr/solrserver3

    Uses a disk drive exclusively. The recommended capacity is greater than or equal to 600 GB.

    SolrServer4

    /srv/BigData/solr/solrserver4

    Uses a disk drive exclusively. The recommended capacity is greater than or equal to 600 GB.

    SolrServer5

    /srv/BigData/solr/solrserver5

    Uses a disk drive exclusively. The recommended capacity is greater than or equal to 600 GB.

    When index data exists, do not change the Solr data storage directory. Otherwise, index data will be lost.

  • Elasticsearch data disk partition requirements
    NOTE:

    Elasticsearch data disk partitions will be automatically generated during Configuring and Checking the Installation Environment. Configure RAID for data nodes according to partition requirements and ensure that each node has sufficient disks.

    Table 2-13 describes Elasticsearch data disk partition information of data nodes. Configure the actual number according to user requirements.

    Table 2-13 Elasticsearch data disk partition information

    Node Type

    Partition Directory

    Minimum Capacity Requirements

    Usage

    Controller node

    EsMaster

    /srv/BigData/elasticsearch/esmaster

    Uses an exclusive drive. Recommended capacity: ≥ 600 GB

    Stores the metadata of Elasticsearch.

    At least three nodes contain this partition directory.

    Data node

    EsNode1-9

    /srv/BigData/elasticsearch/esnode1-9

    Uses an exclusive drive. Recommended capacity: ≥ 600 GB

    Stores the index data of the Elasticsearch.

    Disk partition depends on actual instance deployment. For details, see Service Deployment Scheme.

    When index data exists, do not change the Elasticsearch data storage directory. Otherwise, index data will be lost.

  • Kafka data disk partition requirements
    NOTE:

    Kafka data disk partitions will be automatically generated during Configuring and Checking the Installation Environment. Configure RAID for data nodes according to partition requirements and ensure that the disk quantity and capacity configured for each node are the same.

    Table 2-14 describes Kafka data disk partition information of data nodes. Configure the actual number according to user requirements.

    Table 2-14 Kafka data disk partition information

    Node Type

    Partition Directory

    Minimum Capacity Requirements

    Usage

    Data node

    Kafka data1 to N

    /srv/BigData/kafka/data1

    Uses a disk drive exclusively. The recommended capacity is greater than or equal to 600 GB.

    Directory for storing Kafka Broker data.

    Two nodes contain this partition directory.

    ...

    Uses a disk drive exclusively. The recommended capacity is greater than or equal to 600 GB.

    /srv/BigData/kafka/dataN

    Uses a disk drive exclusively. The recommended capacity is greater than or equal to 600 GB.

  • Storm data disk partition requirements
    NOTE:

    Storm data disk partitions will be automatically generated during Configuring and Checking the Installation Environment. Configure RAID for data nodes according to partition requirements and ensure that each node has sufficient disks.

    Table 2-15 describes Storm data disk partition information of data nodes. Configure the actual number according to user requirements.

    Table 2-15 Storm data disk partition information

    Node Type

    Partition Directory

    Minimum Capacity Requirements

    Usage

    Data node

    Storm data

    /srv/BigData/storm_data

    Uses a disk drive exclusively. The recommended capacity is greater than or equal to 600 GB.

    Directory for storing Storm Supervisor and workers data.

    At least one node contains this partition directory.

  • Redis data disk partition requirements
    NOTE:

    Redis data disk partitions will be automatically generated during Configuring and Checking the Installation Environment. Configure RAID for data nodes according to partition requirements and ensure that each node has sufficient disks.

    Table 2-16 describes Kafka data disk partition information of data nodes. Configure the actual number according to user requirements.

    Table 2-16 Redis data disk partition information

    Node Type

    Partition Directory

    Minimum Capacity Requirements

    Usage

    Data node

    Redis

    /srv/BigData/redis/Redis_1

    You can divide the capacity of a single disk by the number of partitions on each disk to calculate the partition capacity.

    It is used to store Redis database data.

    If a Redis cluster is created, at least three nodes contain this partition directory.

    NOTE:

    The number of Redis partitions on each node is determined based on the number of Redis instances. Each Redis instance corresponds to a disk partition. (The number of Redis instances is the number of CPU cores minus 2.) It is recommended that the number of Redis partitions allocated on each disk be equal to or less than five to ensure cluster performance.

    ...

    ...

    /srv/BigData/redis/Redis_N

    You can divide the capacity of a single disk by the number of partitions on each disk to calculate the partition capacity.

Hard Disk Partition Encryption

In a Linux operating system (OS), you can use the dm-crypt module to create physical or virtual encrypted disk partitions. If you use the dm-crypt module to encrypt disk partitions, all encryption is automatically completed by the Linux kernel. When you create an encrypted partition for the first time, the OS automatically generates a data key to encrypt or decrypt data during data saving. When the OS accesses the encrypted disk partition, the partition is automatically opened using the data key and displayed as a common partition. All directories and files in the partition can be directly accessed.

FusionInsight installation script tool employs the Linux Unified Key Setup (LUKS) partition encryption solution. This solution generates an access key on each node of a cluster when encrypting partitions. The access key is used to encrypt data keys to improve data key security.

When users are demanding on disk data security and want to prevent unencrypted data from being accessed by the third party directly if FusionInsight disks are lost, they can enable the disk partition encryption feature to format disks and encrypt them before installing big data clusters.

Table 2-17 Key concepts

Concept

Description

dm-crypt

dm-crypt is a transparent disk encryption subsystem in Linux kernel versions. It is part of the device mapper infrastructure.

dm-crypt is implemented as a device mapper target and can be stacked on top of other device mapper transformations. It can thus encrypt whole disks, partitions, logical volumes, as well as files.

Data Key

When encrypting a disk partition, OS will automatically generate a data key, that is, the master key in the LUKS solution. The encrypted information is stored at the bottom layer of a file system by default and the upper-layer file system cannot be directly accessed. When accessing the disk partition, the OS uses a plaintext data key for decryption so that applications can directly access plaintext directories or files.

Access Key

The OS uses an access key to encrypt a data key. All nodes in a cluster use the same access key. By default, both a primary key and a secondary key are generated to improve access key security.

Encryption Algorithms

The aes-xts-plain64 or aes-cbc-plain encryption algorithm is used to encrypt data by default. Other third-part encryption algorithms that are integrated with and supported by the OS kernel can also be used to encrypt data.

Restrictions on partition encryption are as follows:

  • The OS and CPU must support the Intel Advanced Encryption Standard New Instructions (AES-NI).
  • Only partitions of data disks of Hadoop, Solr, Storm, Kafka, and Redis on cluster nodes can be encrypted.
  • Data in the encrypted disk partition will not be decrypted after the partition encryption is manually canceled. You can only obtain its plaintext data by reading data and writing it to other storage devices that are not encrypted.
  • If the encrypted disk is damaged or its partitions are lost, the data key will be lost.
  • Disk encryption may deteriorate data read/write performance.
  • Access key loss may cause attachment failure of the encrypted disks during system power-on and power-off.
  • The access key cannot be backed up.
  • The access key and partition data will be deleted during uninstallation of the diskmgt process.
  • Different encrypted disk partitions of each node share the same data key.
  • The installation script tool cannot encrypt disk partitions on SUSE 11.1/11.2/11.3/11.4, Red Hat 6.9, and CentOS 6.9.
  • For FusionInsight Manager, the access key will not be automatically deleted together with a node. You must manually delete fi_disk_key_1 and fi_disk_key_2 in /usr/local/diskmgt/ini-plugin/keys/ to prevent an accidental key leakage.

Disk RAID Requirements

To ensure reliable running and performance of the cluster, comply with the following disk RAID configuration rules.

The RAID configuration requirements of each node are described using the scenario where eight nodes are deployed separately as an example.

  • Management node RAID requirements

    Figure 2-6 describes the OS disk and metadata disk partition requirements of a management node.

    Figure 2-6 Management node RAID requirements
  • Control node RAID requirements

    Figure 2-7 describes the OS disk and metadata disk partition requirements of a control node.

    Figure 2-7 Control node RAID requirements
  • Data node RAID requirements
    • Figure 2-8 describes the OS disk and data disk RAID requirements of Hadoop data node.
      Figure 2-8 Hadoop data node RAID requirements
    • Figure 2-9 describes the OS disk and data disk RAID requirements of Solr data node.
      Figure 2-9 Solr data node RAID requirements
    • Figure 2-10 describes the OS disk and data disk RAID requirements of Elasticsearch data node.
      Figure 2-10 Elasticsearch data node RAID requirements
    • Figure 2-11 describes the OS disk and data disk RAID requirements of Kafka data node.
      Figure 2-11 Kafka data node RAID requirements
    • Figure 2-12 describes the OS disk and data disk RAID requirements of Storm data node.
      Figure 2-12 Storm data node RAID requirements
    • Figure 2-13 describes the OS disk and data disk RAID requirements of Redis data node.
      Figure 2-13 Redis data node RAID requirements
Download
Updated: 2019-05-17

Document ID: EDOC1100074555

Views: 6122

Downloads: 6

Average rating:
This Document Applies to these Products
Related Version
Related Documents
Share
Previous Next