No relevant resource is found in the selected language.

This site uses cookies. By continuing to browse the site you are agreeing to our use of cookies. Read our privacy policy>Search

Reminder

To have a better experience, please upgrade your IE browser.

upgrade

OceanStor 9000 V300R006C00 File System Feature Guide 12

Rate and give feedback:
Huawei uses machine translation combined with human proofreading to translate this document to different languages in order to help you better understand the content of this document. Note: Even the most advanced machine translation cannot match the quality of professional translators. Huawei shall not bear any responsibility for translation accuracy and it is recommended that you refer to the English document (a link for which has been provided).
Restrictions

Restrictions

This section describes the restrictions on the HDFS feature.

Function Restrictions on FusionInsight Hadoop

This section describes the function restrictions on FusionInsight Hadoop after FusionInsight Hadoop is interconnected with the OceanStor 9000.

Table 14-6 shows FusionInsight Hadoop components supported by HDFS Plugin and component instances.

Table 14-6  FusionInsight Hadoop components supported by HDFS Plugin and component instances

Component

Instance

HDFS

DataNode

JournalNode

NameNode

Zkfc

Hive

HiveServer

MetaStore

WebHCat

HBase

HMaster

RegionServer

ThriftServer

Mapreduce

JobHistoryServer

Yarn

NodeManager

ResourceManager

Spark

JDBCServer

JobHistory

SparkResource

Solr

SolrServer

SolrServerAdmin

Table 14-7 shows the supported HDFS Shell commands after the Hadoop cluster is interconnected with the OceanStor 9000.

NOTE:

For details on the HDFS Shell commands, visit http://hadoop.apache.org/.

Table 14-7  Support for HDFS Shell commands

Command

Support or Not

appendToFile

Yes.

cat

Yes. Because Erasure Code is used to store data, the OceanStor 9000 does not perform Cyclic Redundancy Check (CRC) no matter whether the ignorCrc parameter is added or not.

checksum

No.

chgrp

Yes.

chmod

Yes.

chown

Yes.

copyFromLocal

Yes.

copyToLocal

Yes. Because Erasure Code is used to store data, the OceanStor 9000 does not perform Cyclic Redundancy Check (CRC) no matter whether the ignorCrc parameter is added or not.

count

Yes.

cp

Yes.

createSnapshot

No.

deleteSnapshot

No.

df

Yes.

du

Yes.

dus

Yes.

expunge

Yes.

find

Yes.

get

Yes. Because Erasure Code is used to store data, the OceanStor 9000 does not perform Cyclic Redundancy Check (CRC) no matter whether the ignorCrc parameter is added or not.

getfacl

No.

getfattr

No.

getmerge

No.

help

Yes.

ls

Yes.

lsr

Yes.

mkdir

Yes.

moveFromLocal

Yes.

moveToLocal

Yes.

mv

Yes.

put

Yes.

renameSnapshot

No.

rm

Yes.

rmdir

Yes.

rmr

Yes.

setfacl

No.

setfattr

No.

setrep

No.

stat

Yes.

tail

Yes.

test

Yes.

text

Yes. Because Erasure Code is used to store data, the OceanStor 9000 does not perform Cyclic Redundancy Check (CRC) no matter whether the ignorCrc parameter is added or not.

touchz

Yes.

truncate

No.

usage

Yes.

HDFS Plugin supports Shell and Java APIs and does not support C and HTTP REST APIs.

The following restrictions are placed on data migration and capacity expansion:

  • If there is data in the HBase and Hive tables before the OceanStor 9000 is interconnected with the FusionInsight Hadoop cluster, manually process the data after the interconnection.

    If the data needs to be retained, migrate the data to save it. If the data does not need to be retained, delete it and then restart the FusionInsight Hadoop cluster. For details, see (Optional) Migrating Service Data.

  • During HDFS data migration, metadata in NameNode, including directory permissions and owning groups of files, cannot be migrated.

    For a Kerberos-authenticated FusionInsight Hadoop cluster, the original permission of data becomes invalid after the data is being migrated and is uniformly changed to 777 (that is, rwxrwxrwx).

  • A newly-added Hadoop node cannot immediately take effect. For details on how to make it take effect, see Adding a Hadoop Node.

  • After OceanStor 9000 is interconnected with the FusionInsight Hadoop cluster, when using Hive to create tables, an error message will display, which does not affect normal use of Hive.

Function Restrictions on Cloudera Hadoop

This section describes the function restrictions on Cloudera Hadoop after Cloudera Hadoop is interconnected with the OceanStor 9000.

Table 14-8 shows Cloudera Hadoop components supported by HDFS Plugin and component instances.

Table 14-8  Cloudera Hadoop components supported by HDFS Plugin and component instances

Component

Instance

HDFS

DataNode

NameNode

SeconaryNameNode

Hive

Hive Metastore Server

HiveServer

WebHCat Server

HBase

HBase Thrift Server

RegionServer

Master

MapReduce

JobTracker

TaskTracker

Yarn

JobHistory Server

ResourceManager

NodeManager

Spark

History Server

Solr

Solr Server

Table 14-9 shows the supported HDFS Shell commands after the Hadoop cluster is interconnected with the OceanStor 9000.

NOTE:

For details on the HDFS Shell commands, visit http://hadoop.apache.org/.

Table 14-9  Support for HDFS Shell commands

Command

Support or Not

appendToFile

Yes.

cat

Yes. Because Erasure Code is used to store data, the OceanStor 9000 does not perform Cyclic Redundancy Check (CRC) no matter whether the ignorCrc parameter is added or not.

chgrp

Yes.

chmod

Yes.

chown

Yes.

copyFromLocal

Yes.

copyToLocal

Yes. Because Erasure Code is used to store data, the OceanStor 9000 does not perform Cyclic Redundancy Check (CRC) no matter whether the ignorCrc parameter is added or not.

count

Yes.

cp

Yes.

du

Yes.

dus

Yes.

expunge

Yes.

get

Yes. Because Erasure Code is used to store data, the OceanStor 9000 does not perform Cyclic Redundancy Check (CRC) no matter whether the ignorCrc parameter is added or not.

getfacl

No.

getfattr

No.

getmerge

No.

ls

Yes.

lsr

Yes.

mkdir

Yes.

moveFromLocal

Yes.

moveToLocal

Yes.

mv

Yes.

put

Yes.

rm

Yes.

rmdir

Yes.

rmr

Yes.

setfacl

No.

setfattr

No.

setrep

No.

stat

Yes.

tail

Yes.

test

Yes.

text

Yes. Because Erasure Code is used to store data, the OceanStor 9000 does not perform Cyclic Redundancy Check (CRC) no matter whether the ignorCrc parameter is added or not.

touchz

Yes.

createSnapshot

No.

deleteSnapshot

No.

renameSnapshot

No.

HDFS Plugin supports Shell and Java APIs and does not support C and HTTP REST APIs.

The following restrictions are placed on data migration and capacity expansion:

  • If there is data in the HBase and Hive tables before the OceanStor 9000 is interconnected with the Hadoop cluster, manually process the data after the interconnection.

    If the data needs to be retained, migrate the data to save it. If the data does not need to be retained, delete it and then restart the Hadoop cluster. For details, see (Optional) Migrating Service Data.

  • During HDFS data migration, metadata in NameNode, including directory permissions and owning groups of files, cannot be migrated.

    For a Kerberos-authenticated Hadoop cluster, the original permission of data becomes invalid after being migrated and is uniformly transformed to 777 (that is, rwxrwxrwx).

  • A newly-added Hadoop node cannot immediately take effect. For details on how to make it take effect, see Adding a Hadoop Node.

Interaction with Other Features

This section describes the interaction of the HDFS feature with other value-added features.

Table 14-10 shows the interaction of the HDFS feature with other features.

Table 14-10  Interaction with other features

Feature

Interaction with HDFS

InfoEqualizer

InfoEqualizer and HDFS can be used at the same time.

InfoTier

InfoTier can be enabled for the Hadoop data analysis directory.

InfoAllocator

InfoAllocator can be enabled for the Hadoop data analysis directory.

InfoStamper

InfoStamper can be enabled for the Hadoop data analysis directory.

InfoReplicator

InfoReplicator can be enabled for the Hadoop data analysis directory.

InfoLocker

InfoLocker can be enabled for the Hadoop data analysis directory. The root directory for which InfoLocker is enabled cannot be changed after files are written to the root directory (appending is allowed).

InfoRevive

InfoRevive can be enabled for the Hadoop data analysis directory. InfoRevive is exclusively used in video surveillance scenarios. You are advised not to enable the error tolerance read/write function for the Hadoop data analysis directory.

InfoTurbo

InfoTurbo can be enabled for the Hadoop data analysis directory. The pre-read/write function of InfoTurbo is applicable to video editing scenarios. You are advised not to enable the pre-read/write function for the Hadoop data analysis directory.

InfoScanner

InfoScanner can be enabled for the Hadoop data analysis directory.

NDMP backup

NDMP backup can be enabled for the Hadoop data analysis directory.

Translation
Download
Updated: 2019-06-27

Document ID: EDOC1000122519

Views: 74791

Downloads: 145

Average rating:
This Document Applies to these Products
Related Documents
Related Version
Share
Previous Next