No relevant resource is found in the selected language.

This site uses cookies. By continuing to browse the site you are agreeing to our use of cookies. Read our privacy policy>Search

Reminder

To have a better experience, please upgrade your IE browser.

upgrade

OceanStor 9000 V300R006C00 File System Feature Guide 12

Rate and give feedback:
Huawei uses machine translation combined with human proofreading to translate this document to different languages in order to help you better understand the content of this document. Note: Even the most advanced machine translation cannot match the quality of professional translators. Huawei shall not bear any responsibility for translation accuracy and it is recommended that you refer to the English document (a link for which has been provided).
Principle Description

Principle Description

This section introduces principles for InfoTier, including storage tier composition, restriping, watermark policy and file pool policy.

Storage Tier Composition

InfoTier stores a file on a specific storage tier according to file properties. One or multiple node pools form a storage tier. A node pool consists of multiple nodes and can be divided into multiple DiskPools. Each DiskPool includes multiple partitions.

Node Pool

A node pool is the basic unit of a storage tier. A node pool consists of multiple nodes.

OceanStor 9000 physical servers consist of the following node types:

  • P12, P12E, P25, P25E, P36 and P36E nodes are high-performance storage nodes and oriented towards OPS-intensive applications.

  • C36, C36E and C72 nodes are large-capacity archiving nodes and oriented towards capacity applications.

Nodes with different characteristics compose different node pools. These node pools can be divided into different performance tiers for efficient usage. Note the following when creating node pools:

  • A node pool can contain nodes of only one type. For example, if a node pool contains P12 nodes, you can install or add only P12 nodes, but not nodes of other types, into this node pool.
  • If a node pool consists of P12, P12E, P25, P25E, P36, P36E, C36 or C36E nodes, the node pool can contain 3 to 20 nodes. If a node pool consists of C72 nodes, the node pool can contain 2 to 10 nodes.
  • After a node pool has been successfully deployed, node pool nodes can no longer be modified. If you want to change the owning node pool of a certain node, delete the node from its owning node pool first, and then add it into another node pool.

After node pools are created, they can be migrated from one storage tier to another one without restriping data in the node pool.

DiskPool

Disks of all the nodes in each node pool form DiskPools based on disk types. The DiskPool formed by SSDs is used to store data of small files if SSD acceleration configuration is enabled. The DiskPool formed by HDDs is used to store data and metadata.

SSDs in slot 2 to n (n indicates the number of SSDs) form a DiskPool. HDDs form another DiskPool.

NOTE:
DiskPool division is related to disk configurations of nodes. Each node has only one SSD at most in typical configurations. The SSD is inserted into the first slot of the node and used by the underlying file system. No DiskPool composed of SSDs is available. To fully utilize advantages of OceanStor 9000 in reading and writing small files, you need to configure multiple SSDs.
Storage Tier
After system deployment, an administrator can set storage tiers based on service requirements and define the relationship between node pools and storage tiers. There is a default storage tier in the system. If no new storage tier is configured, all node pools belong to the default one. To better use InfoTier, you are advised to configure multiple storage tiers and file pool policies.
  • You are advised to associate a storage tier with a node pool that is created by high-performance servers to process hot data. This accelerates the system's response to hot data and improves the overall storage performance.
  • You are advised to associate a storage tier with a node pool that is created by slow-response but large-capacity servers to process cold data. This fully utilizes the advantages of different storage servers and effectively reduces deployment and maintenance costs.

It is recommended that one storage tier consists of node pools of the same type. Users can arrange the type of node pools in a storage tier based on site requirements.

Relationship Between a Node Pool, DiskPool, and Storage Tier

Figure 3-1 shows the relationship between a node pool, DiskPool, and storage tier. The number of SSDs and HDDs in this figure is an example for your reference. The actual node configurations prevail. If you want a DiskPool composed of SSDs, configure multiple SSDs.

Figure 3-1  Relationship between a node pool, DiskPool, and storage tier

Restriping

Restriping means migrating data that has been stored.

The system periodically scans for metadata and determines whether to restripe files that have been stored in the storage system based on the file pool policy. If files need to be restriped, send a restriping task.

Restriping is used to update data storage location based on the file pool policy, as shown in Figure 3-2.

Figure 3-2  Restriping

Before restriping, check whether the used capacity of the node pool in a specific storage tier is higher than the read-only watermark. If the capacity is lower than the read-only watermark, restriping can be performed. If the capacity is not lower than the read-only watermark, restriping cannot be performed. During restriping, if the used capacities of all node pools in a specific storage tier are higher than the read-only watermark, stop restriping to the storage tier.

Do not interrupt user access during restriping. If a user wants to modify the data that is being restriped, stop restriping, perform a rollback, delete the data that has been restriped to a new node pool, and wait for the next restriping task.

A new restriping task cannot be started until the existing task has been completed. For example, when files are being restriped from storage tier 1 to storage tier 2, the files have to be restriped to storage tier 3 to adapt to the changed file pool policy. In this case, files must be restriped to storage tier 2 before being restriped to storage tier 3.

Watermark Policy

InfoTier uses watermark policies to monitor the storage capacities of node pools. InfoTier determines where to store new data based on available node pool capacity.

Basic Concepts

System watermarks include a high watermark and a read-only watermark, indicating the percentage of the used disk capacity to the valid disk capacity. The valid disk capacity is the same as the minimum disk capacity in the DiskPool. The OceanStor 9000 restricts file storage and migration by setting watermark thresholds. The system can use the Spillover function to define when a node pool reaches its read-only watermark and whether data to the pool can be written to another node pool.

Figure 3-3 shows that data written to the node pool is under the control of watermark policies.

Figure 3-3  Data writing controlled by water policies

High Watermark

The default high watermark indicates that the used disk capacity occupies 85% of the valid disk capacity. When the used capacity of a node pool reaches the high watermark, the system reports a capacity insufficiency alarm, but will still allows data to be written.

By default, the recovery threshold of the high watermark is 5% lower than the alarm threshold of the high watermark. That is, when the used capacity is recovered to lower than 80%, the capacity insufficiency alarm is cleared. The alarm threshold and recovery threshold of the high watermark are self-definable.

Read-Only Watermark

The default read-only watermark indicates that the used disk capacity occupies 90% of the valid disk capacity. When the used capacity of a node pool reaches the read-only watermark, the system reports a read-only alarm, and refuses data to be written to the node pool.

By default, the recovery threshold of the read-only watermark is 5% lower than the alarm threshold of the read-only watermark. That is, when the used capacity is recovered to lower than 85%, the read-only alarm is cleared, and the system allows data to be written. The alarm threshold and recovery threshold of the read-only watermark are both configurable.

Spillover Function

When the used capacity of a node pool reaches the read-only watermark and the Spillover function is enabled, data cannot be written to the node pool but is automatically written to any other node pool or the node pool specified by Spillover. If the used capacity of a node pool specified by Spillover exceeds the read-only watermark, data cannot be written to the node pool.

When the used capacity of a node pool reaches the read-only watermark and the Spillover function is disabled, data can neither be written to the node pool nor to any other node pool. The data writing therefore fails.

Setting Rules of Watermark Thresholds
When setting attributes for a node pool, set a watermark threshold according to specific watermark setting rules.
Table 3-3  Setting rules of watermark thresholds

Watermark Threshold

Range (%)

Remarks

Read-only watermark alarm threshold(%)

85 to 100
NOTE:
To maintain system performance and reliability, you are advised to set the Read-only watermark alarm threshold within the range from 85% to 90%.
  • The read-only watermark alarm threshold must be higher than the high watermark alarm threshold.
  • The read-only watermark recovery threshold must be higher than the high watermark recovery threshold.
  • The read-only watermark alarm threshold must be higher than the read-only watermark recovery threshold.
  • The high watermark alarm threshold must be higher than the high watermark recovery threshold.

Read-only watermark recovery threshold(%)

80 to 95

High watermark alarm threshold(%)

80 to 95

High watermark recovery threshold(%)

75 to 90

File Pool Policy

Administrators can create file pool policies to determine initial storage locations and storage tiers to which files are restriped to.

Immediately after InfoTier is enabled, the OceanStor 9000 matches a file pool policy and uses the file pool policy to migrate files.

A file pool policy can be configured to be a combination of multiple parameters. A file pool policy can be matched only when the file properties match all parameters of the file pool policy.

Table 3-4 lists the parameters of a file pool policy.

Table 3-4  File pool policy parameters

File Parameter

Time Parameter

Performance Parameter

Storage Location

File name

Created time

I/O count

Specific storage tier

File path

Modified time

I/O popularity

Any storage tier

File size

Lastly accessed time

SSD acceleration enabled or not

Specific Node Poola

Users/Groups

N/A

N/A

N/A

a: You can specify a node pool by setting a file path, users/groups, and file name in a file pool policy. If you set other conditions, the storage location can either be a specific storage tier or any storage tier.

Administrators can configure a maximum of 128 file pool policies. When multiple policies are matched, use the policy with the highest priority to store or restripe files. There is a default policy in the system. If no other policies are matched, a file is stored by following the default policy. The default policy has the lowest priority and only reacts when determining the initial storage location of the file. You are allowed to modify the default policy but not to delete it.

The process of storing files to the OceanStor 9000 consists of initial write and restriping after storage. The following example details the storage process.

Assume that node pools composed of P25 nodes form storage tier A, and those composed of C72 nodes form storage tier B. Assume that the file pool policy is configured as Table 3-5 and the storage tier with the maximum free space is storage tier B.

Table 3-5  File pool policy

File Pool Policy Name

File Pool Policy

Destination Storage Tier

File Pool Policy Priority

Policy 1

File size < 100 MB

I/O count > 20

Tier A

High

Policy 2

File size ≥ 100 MB

I/O count ≤ 20

Tier B

Medium

Policy 3

File path: /example

For VIP users

Tier A

Low

Initial Write

When a file is about to be stored on the OceanStor 9000, the OceanStor 9000 will match the file pool policy according to the file path, file name, user name, and user group name of the file and determine the initial storage location of the file.
  • If a file pool policy is matched and the used space of the node pool does not reach its read-only watermark, the file is stored on the storage tier specified in the policy.
  • If a file pool policy is matched, the used space of the node pool reaches its read-only watermark, and the Spillover function is enabled, the file is migrated to a random node pool or a specified node pool according to the Spillover setting.
  • If a file pool policy is matched, the used space of the node pool reaches its read-only watermark, and the Spillover function is disabled, the file fails to be written onto any storage tier.
  • If no file pool policy is matched, the file is stored on the storage tier with the largest available space.
Table 3-6 describes the matching result of the changed file property against the file pool policy.
Table 3-6  Initial write properties

File

File Property

Matching Result

A

File A is created by a VIP user and is placed under the \example directory.

If the file matches policy 3, the file is stored on storage tier A.

B

File B is placed under the \abc directory.

If the file does not match any policy, the file is stored on storage tier B (the storage tier with the largest available capacity).

C

File C is placed under the \abc directory.

If the file does not match any policy, the file is stored on storage tier B (the storage tier with the largest available capacity).

Figure 3-4 shows the storage tiers where file A, file B, and file C are initially stored.
Figure 3-4  File storage tiers

Restriping of Stored Files

For stored data, the OceanStor 9000 monitors file property changes and automatically migrates files to a corresponding storage tier in accordance with the preset file restriping policy for optimized performance.
  • If a file pool policy is matched and the used space of the node pool does not reach its read-only watermark, the file is migrated to the specified storage tier.
  • If a file pool policy is matched, the used space of the node pool reaches its read-only watermark, and the Spillover function is enabled, the file is migrated to a random storage tier or a specified storage tier according to the Spillover setting.
  • If a file pool policy is matched, the used space of the node pool reaches its read-only watermark, and the Spillover function is disabled, the file is not migrated.
  • If no policy is matched, the file is not restriped.
  • If the used capacities of all node pools of a specific storage tier reach its read-only watermark during restriping, restriping to the storage tier is stopped.
Table 3-7 describes the matching result of the changed file property against the file pool policy.
Table 3-7  File property changes

File

File Size

I/O Count

Matching Result

A

150 MB

15

If file A matches file policies 2 and 3, and file policy 2 has a higher priority than file policy 3, the matching result is file policy 2. That is, file A is migrated from storage tier A to storage tier B.

B

50 MB

23

If file B matches file policy 1, the file is migrated from storage tier B to storage tier A.

C

160 MB

15

If file C matches file policy 2, the file is still stored on storage tier B.

Based on the previous matching results, the destination storage tiers are specified as shown in Figure 3-5.
Figure 3-5  File migration

Translation
Download
Updated: 2019-06-27

Document ID: EDOC1000122519

Views: 84610

Downloads: 151

Average rating:
This Document Applies to these Products
Related Documents
Related Version
Share
Previous Next