No relevant resource is found in the selected language.

This site uses cookies. By continuing to browse the site you are agreeing to our use of cookies. Read our privacy policy>Search

Reminder

To have a better experience, please upgrade your IE browser.

upgrade

FusionInsight HD 6.5.0 Product Description 02

Rate and give feedback:
Huawei uses machine translation combined with human proofreading to translate this document to different languages in order to help you better understand the content of this document. Note: Even the most advanced machine translation cannot match the quality of professional translators. Huawei shall not bear any responsibility for translation accuracy and it is recommended that you refer to the English document (a link for which has been provided).
ZooKeeper

ZooKeeper

Basic Concept

Overview

ZooKeeper is a distributed, highly available coordination service. ZooKeeper provides two functions:

  • Prevents the system from single point of failures (SPOFs) and provides reliable services for applications.
  • Provides distributed coordination services and manages configuration information.
Architecture

Nodes in a ZooKeeper cluster have three roles: Leader, Follower, and Observer. Figure 2-118 shows the ZooKeeper architecture. Generally, an odd number (2N+1) of ZooKeeper servers are configured. At least (N+1) vote majority is required to successfully perform write operation.

Figure 2-118 ZooKeeper architecture

Table 2-26 describes the functions of each module shown in Figure 2-118.

Table 2-26 ZooKeeper modules

Module

Description

Leader

Only one node serves as the Leader in a ZooKeeper cluster. The Leader, elected by Followers using the ZooKeeper Atomic Broadcast (ZAB) protocol, receives and coordinates all write requests and synchronizes written information to Followers and Observers.

Follower

Follower has two functions:

  • Prevents SPOF. A new Leader is elected from Followers when the Leader is faulty.
  • Processes read requests and interacts with the Leader to process write requests.

Observer

The Observer does not take part in voting for election and write requests. It only processes read requests and forwards write requests to the Leader, increasing system processing efficiency.

Client

Reads and writes data from or to the ZooKeeper cluster. For example, HBase can serve as a ZooKeeper client and use the arbitration function of the ZooKeeper cluster to control the active/standby status of the HMaster.

If security services are enabled in the cluster, authentication is required during the connection to ZooKeeper. The authentication modes are as follows:

  • keytab mode: Obtain a human-machine user from the administrator for login to the FusionInsight HD platform and authentication, and obtain the keytab file of the user.
  • Ticket mode: Obtain a human-machine user from the administrator for subsequent secure login, enable the renewable and forwardable functions of the Kerberos service, set the ticket update interval, and restart Kerberos and related components.
    NOTE:
    • For details about how to obtain a human-machine user and the keytab file of the user, see the Administrator Guide.
    • The default validity period of a user password is 90 days. Therefore, the validity period of the obtained keytab file is 90 days. To prolong the validity period of the keytab file, modify the user password policy and obtain the keytab file again. For details, see the Administrator Guide.
    • The parameters for enabling the renewable and forwardable functions and setting the ticket update interval are on the System tab of the Kerberos service configuration page. The ticket update interval can be set to kdc_renew_lifetime or kdc_max_renewable_life based on the actual situation.
Principle
  • Write Request
    1. After the Follower or Observer receives a write request, the Follower or Observer sends the request to the Leader.
    2. The Leader coordinates Followers to determine whether to accept the write request by voting.
    3. If more than half of voters return a write success message, the Leader submits the write request and returns a success message. Otherwise, a failure message is returned.
    4. The Follower or Observer returns the processing results.
  • Read Request

    The client directly reads data from the Leader, Follower, or Observer.

Relationship with Other Components

Relationship Between ZooKeeper and HDFS

Figure 2-119 describes the relationship between ZooKeeper and HDFS.

Figure 2-119 Relationship between ZooKeeper and HDFS

As the client of a ZooKeeper cluster, ZKFailoverController (ZKFC) monitors the state information of NameNode. The ZKFC process is deployed only in the node where NameNode is deployed. The ZKFC process is deployed in both the active node and standby node of HDFS NameNode.

  1. The ZKFC of HDFS NameNode connects to ZooKeeper and save information such as host names to ZooKeeper, under the znode directory say /hadoop. NameNode that creates the znode first is considered as the active node, and the other is the standby node. HDFS NameNode Standby watches the active NameNode information through the ZooKeeper watcher.
  2. When the process of the active node is ended abnormally, HDFS NameNode Standby detects that some changes occur in the /hadoop znode directory using ZooKeeper, and then NameNode performs the active/standby switchover.
Relationship Between ZooKeeper and YARN

Figure 2-120 describes the relationship between ZooKeeper and YARN.

Figure 2-120 Relationship between ZooKeeper and YARN
  1. ResourceManager tries to write the election information into ZooKeeper when the system starts. The ResourceManager which writes the information into ZooKeeper successfully first is elected as the active ResourceManager. The other one is elected as the standby ResourceManager. The standby ResourceManager monitors the election information of the active ResourceManager through ZooKeeper at regular time.
  2. The active ResourceManager creates a Statestore directory in ZooKeeper to store Application information. The standby ResourceManager obtains Application information from the Statestore directory and restores the data when the active ResourceManager is faulty.
Relationship Between ZooKeeper and HBase

Figure 2-121 describes the relationship between ZooKeeper and HBase.

Figure 2-121 Relationship between ZooKeeper and HBase
  1. HRegionServer registers itself to ZooKeeper in Ephemeral node. ZooKeeper stores the HBase information, including the HBase metadata and HBase address.
  2. HMaster detects the health status of each HRegionServer using ZooKeeper, and monitors them.
  3. HBase can deploy multiple HMasters (like HDFS NameNode). When the active HMatser node is faulty, the standby HMaster node obtains the state information of the entire cluster using ZooKeeper, which means that HBase single point faults can be avoided using ZooKeeper.
Relationship Between SmallFS and ZooKeeper

FGCService is deployed in high availability (HA) mode. The HA mode is used to ensure service availability when a single-node fault occurs. FGCService depends on ZooKeeper to support the HA mode.

Figure 2-122 Relationship between ZooKeeper and ZooKeeper
Download
Updated: 2019-05-17

Document ID: EDOC1100074548

Views: 4003

Downloads: 37

Average rating:
This Document Applies to these Products
Related Documents
Related Version
Share
Previous Next