No relevant resource is found in the selected language.

This site uses cookies. By continuing to browse the site you are agreeing to our use of cookies. Read our privacy policy>Search

Reminder

To have a better experience, please upgrade your IE browser.

upgrade

FusionInsight HD 6.5.0 Administrator Guide 02

Rate and give feedback:
Huawei uses machine translation combined with human proofreading to translate this document to different languages in order to help you better understand the content of this document. Note: Even the most advanced machine translation cannot match the quality of professional translators. Huawei shall not bear any responsibility for translation accuracy and it is recommended that you refer to the English document (a link for which has been provided).
Backing Up Elasticsearch Service Data

Backing Up Elasticsearch Service Data

Scenario

To ensure Elasticsearch routine data security or before and after a critical operation (such as upgrade or migration) on Elasticsearch, Elasticsearch service data needs to be backed up. The backup data can be used to recover the system in time if an exception occurs or the operation has not achieved the expected result, minimizing the adverse impact on services.

This section describes how to create an Elasticsearch data backup task and back up data on FusionInsight Manager. Both automatic backup tasks and manual backup tasks are supported.

  • During the snapshot creation, the search and query functions are not affected. After the snapshot creation process starts, new data is not recorded in the snapshot. Only one snapshot can be created at a time.
  • When a backup task is created, only the indexes that have been opened in the cluster are displayed as backup objects. The closed indexes are not displayed on the GUI. This way, the closed indexes are not backed up.
  • The Elasticsearch service data backup needs to invoke the snapshot interface through the EsNode1 instance. Therefore, ensure that all EsNode1 instances in the cluster are in good health status and can receive requests normally.

Prerequisites

  • A standby cluster for backing up data has been created. For details, see Software Installation.
  • Cross-cluster mutual trust has been configured for the active and standby clusters if data needs to be backed up to the standby cluster and the active cluster is deployed in security mode. For details, see Configuring Cross-Cluster Mutual Trust Relationships. If the active cluster is deployed in normal mode, cross-cluster mutual trust is not required.
  • Time of the active and standby clusters is consistent, and the Network Time Protocol (NTP) service in the active and standby clusters use the same time source.
  • The HDFS in the standby cluster has sufficient space. You are advised to save backup files in the user-defined directory.
  • Perform the following steps when Elasticsearch is configured in security mode and Elasticsearch service data is backed up to RemoteHDFS. If Elasticsearch is configured in no-security mode, skip the steps. Create the repository-hdfs directory in the $BIGDATA_HOME/FusionInsight_Elasticsearch_6.5.0/install/FusionInsight-Elasticsearch-6.1.3/cluster/instance directory/config directory of each instance of the local Elasticsearch service, upload the elasticsearch.keytab file in $BIGDATA_HOME/components/FusionInsight_Elasticsearch_6.5.0/Elasticsearch/ to the repository-hdfs directory, and rename the file krb5.keytab.

    For details about how to run the commands in batches, see section "How Do I Run Commands or Access Files on Multiple Nodes in a Cluster" in FusionInsight HD Administrator Guide by choosing Appendix > Administrator FAQs.

    1. To create the repository-hdfs directory in the instance directory, run the following commands:

      ./clustercmd.sh "mkdir /opt/huawei/Bigdata/FusionInsight_Elasticsearch_6.5.0/install/FusionInsight-Elasticsearch-6.1.3/cluster/EsMaster/config/repository-hdfs"

      ./clustercmd.sh "mkdir /opt/huawei/Bigdata/FusionInsight_Elasticsearch_6.5.0/install/FusionInsight-Elasticsearch-6.1.3/cluster/EsNode1/config/repository-hdfs"

      NOTE:

      If EsNode2-9 and EsClient instances are deployed, create the repository-hdfs directory in all instance directories by running the preceding commands.

    2. Rename elasticsearch.keytab as krb5.keytab.

      cp /opt/huawei/Bigdata/components/FusionInsight_Elasticsearch_6.5.0/Elasticsearch/elasticsearch.keytab /home/omm/krb5.keytab

    3. To upload the krb5.keytab file to the instance directory, run the following commands:

      ./clusterscp.sh put /home/omm/krb5.keytab /opt/huawei/Bigdata/FusionInsight_Elasticsearch_6.5.0/install/FusionInsight-Elasticsearch-6.1.3/cluster/EsMaster/config/repository-hdfs

      ./clusterscp.sh put /home/omm/krb5.keytab /opt/huawei/Bigdata/FusionInsight_Elasticsearch_6.5.0/install/FusionInsight-Elasticsearch-6.1.3/cluster/EsNode1/config/repository-hdfs

Procedure

  1. Log in to FusionInsight Manager, and choose O&M > Backup and Restoration > Backup Management.
  2. Click Create.
  3. On the displayed page, set Name to the name of a backup task.
  4. Set Mode to Periodic or Manual based on site requirement. Periodic indicates that a backup task is periodically executed, and Manual indicates that a backup task is manually executed.

    To create a periodic task, set the following parameters:

    • Started: indicates the time when the task is started for the first time.
    • Period: indicates the interval for executing the backup task. The unit can be hour or day.
    • Backup Policy: indicates the volume of data to be backed up in each task execution. Only Full backup at the first time and incremental backup subsequently can be selected.

  5. In the Configuration area, select Elasticsearch under Service Data.
  6. In the displayed Elasticsearch area, set Path Type to the backup directory type.

    The following backup directory types are supported:

    • RemoteHDFS: indicates that the backup files are stored in the HDFS directory of the standby cluster. If this parameter is selected, set the following parameters:
      • Destination Active NameNode IP Address: indicates the service plane IP address of the active NameNode in the destination cluster.
      • Destination Standby NameNode IP Address: indicates the service plane IP address of the standby NameNode in the destination cluster.
      • Destination NameNode RPC Port: indicates the value of dfs.namenode.rpc.port in the HDFS basic configuration of the destination cluster.
      • Target Path: indicates the HDFS directory for storing destination cluster backup data. The path cannot be an HDFS hidden directory, such as snapshot or recycle bin directory, or a default system directory.

  7. Set Maximum Number of Recovery Points to any value from 1 to 1000 because this parameter is not used by Elasticsearch.
  8. Select one or more indexes to be backed up for Backup Object.

    The following methods are supported to select backup data:

    • Select directly

      Click to obtain all indexes and select the specified indexes for backup.

    • Select using regular expressions
      1. Click Query Regular Expression.
      2. Enter a regular expression in the second text box. Standard regular expressions are supported. For example, to filter indexes that contain ES, enter .*es*. To filter indexes starting with es, enter es*. To enter indexes ending with es, enter .*es*.
      3. Click Refresh and view the filtered tables in Directory Name.
      4. Click Synchronize to save the result.
      NOTE:
      • When entering a regular expression, you can click or to add or delete an expression.
      • If the selected table or directory is incorrect, click Clear Selected Node to deselect it.

  9. Click Verify to check whether the backup task is configured correctly.

    The possible causes of check failure are as follows:

    • The destination active or standby NameNode IP address or port No. is incorrectly configured.
    • The name of the index to be backed up does not exist in the Elasticsearch cluster.

  10. Click OK to save the configurations.
  11. In the backup task list, locate the row where the created task resides, and choose More > Back Up Now in the Operation column.

    After the backup task is executed, the system automatically creates a subdirectory for each backup task in the backup directory. The format of the subdirectory name is backup task name_data source_task creation time, and the subdirectory is used to save latest data source backup files. All the backup file sets are saved to the related snapshot directories.

Download
Updated: 2019-05-17

Document ID: EDOC1100074522

Views: 7239

Downloads: 12

Average rating:
This Document Applies to these Products
Related Documents
Related Version
Share
Previous Next