No relevant resource is found in the selected language.

This site uses cookies. By continuing to browse the site you are agreeing to our use of cookies. Read our privacy policy>Search

Reminder

To have a better experience, please upgrade your IE browser.

upgrade

FusionInsight HD 6.5.0 Administrator Guide 02

Rate and give feedback :
Huawei uses machine translation combined with human proofreading to translate this document to different languages in order to help you better understand the content of this document. Note: Even the most advanced machine translation cannot match the quality of professional translators. Huawei shall not bear any responsibility for translation accuracy and it is recommended that you refer to the English document (a link for which has been provided).
About Logs

About Logs

Log Description

The log file path of the FusionInsight HD cluster is /var/log/Bigdata. The following table lists the log types.

Table 12-1 Log types

Log Type

Description

Installation logs

Installation logs record information about FusionInsight Manager, cluster, and service installation to help users locate installation errors.

Run logs

Run logs record the running track information, debugging information, status changes, potential problems, and error information generated during the running of services.

Audit logs

Audit logs record information about users' activities and operation instructions, which can be used to locate fault causes in security events and determine who are responsible for these faults.

The following table lists the FusionInsight HD log directories.

Table 12-2 Log directories

Directory

Log

/var/log/Bigdata/audit

Component audit log.

/var/log/Bigdata/controller

Log collecting script log.

Controller process log.

Controller monitoring log.

/var/log/Bigdata/dbservice

DBService log.

/var/log/Bigdata/flume

Flume log.

/var/log/Bigdata/ftp-server

FTP-Server log.

/var/log/Bigdata/hbase

HBase log.

/var/log/Bigdata/hdfs

HDFS log.

/var/log/Bigdata/hive

Hive log.

/var/log/Bigdata/httpd

httpd log.

/var/log/Bigdata/hue

Hue log.

/var/log/Bigdata/kerberos

Kerberos log.

/var/log/Bigdata/ldapclient

LDAP client log.

/var/log/Bigdata/ldapserver

LDAP server log.

/var/log/Bigdata/loader

Loader log.

/var/log/Bigdata/logman

logman script log management log.

/var/log/Bigdata/mapreduce

MapReduce log.

/var/log/Bigdata/nodeagent

NodeAgent log.

/var/log/Bigdata/okerberos

OMS Kerberos log.

/var/log/Bigdata/oldapserver

OMS LDAP log.

/var/log/Bigdata/metric_agent

Run log file of MetricAgent.

/var/log/Bigdata/omm

oms: complex event processing log, alarm service log, HA log, authentication and authorization management log, and monitoring service run log of the omm server.

oma: installation log and run log of the omm agent.

core: dump log generated when the omm agent and the HA process are suspended.

/var/log/Bigdata/spark

Spark log.

/var/log/Bigdata/sudo

Log generated when the sudo command is executed by user omm.

/var/log/Bigdata/timestamp

Time synchronization management log.

/var/log/Bigdata/tomcat

Tomcat log.

/var/log/Bigdata/watchdog

Watchdog log.

/var/log/Bigdata/yarn

YARN log.

/var/log/Bigdata/zookeeper

ZooKeeper log.

/var/log/Bigdata/oozie

Oozie log.

/var/log/Bigdata/solr

Solr log.

/var/log/Bigdata/elasticsearch

Elasticsearch log.

/var/log/Bigdata/kafka

Kafka log.

/var/log/Bigdata/redis

Redis log.

/var/log/Bigdata/storm

Storm log.

/var/log/Bigdata/metadata

Metadata log.

/var/log/Bigdata/smallfs

SmallFS log.

/var/log/Bigdata/graphbase

GraphBase log.

/var/log/Bigdata/upgrade

OMS upgrade log file.

/var/log/Bigdata/update-service

Upgrade service logs.

/var/log/Bigdata/patch

Patch log.

NOTE:

After the multi-instance function is enabled, if the system administrator adds multiple HBase, Hive, and Spark service instances, the log description, log level, and log format of the newly added service instances are the same as those of the original service logs. Service instance logs are stored separately in the /var/log/Bigdata/servicenameN directory. The audit logs of the HBase and Hive service instances are stored in the /var/log/Bigdata/audit/servicenameN directory. For example, the logs of HBase1 are stored in the /var/log/Bigdata/hbase1 and /var/log/Bigdata/audit/hbase1 directories.

Installation Logs.

Table 12-3 Installation logs

Installation Log

Description

Configuration log

Records information about the configuration process before the installation.

FusionInsight Manager installation log

Records information about the two-node FusionInsight Manager installation.

Cluster installation log

Records information about the cluster installation.

Run Logs

Table 12-4 describes the running information recorded in run logs.

Table 12-4 Running information

Run Log

Description

Installation preparation log

Records information about preparations for the installation, such as the detection, configuration, and feedback operation information.

Process startup log

Records information about the commands executed during the process startup.

Process startup exception log

Records information about exceptions during process startup, such as dependent service errors and insufficient resources.

Process run log

Records information about the process running track information and debugging information, such as function entries and exits as well as cross-module interface messages.

Process running exception log

Records errors that cause process running errors, for example, the empty input objects or encoding or decoding failure.

Process running environment log

Records information about the process running environment, such as resource status and environment variables.

Script log

Records information about the script execution process.

Resource reclamation log

Records information about the resource reclaiming process.

Uninstallation clearing logs

Records information about operations performed during service uninstallation, such as directory deletion and execution time

Audit Logs

Audit information recorded in audit logs includes FusionInsight Manager audit information and component audit information.

Table 12-5 Audit information of FusionInsight Manager

Audit Log

Operation Type

Operation

Manager audit log

Managing users.

Creating a user.

Modifying a user.

Deleting a user.

Creating a user group.

Modifying a user group.

Deleting a group.

Adding a role.

Changing the user's roles.

Deleting a role.

Changing a password policy.

Changing a password.

Resetting a password.

User login.

User logout.

Unlocking the screen.

Downloading the authentication credential.

Unauthorized operation.

Unlocking a user account.

Locking a user account.

Locking the screen.

Exporting a user.

Exporting a user group.

Exporting a role.

Tenant management

Saving the static configuration.

Adding a tenant.

Deleting a tenant.

Associating a service with a tenant.

Deleting a service from a tenant.

Configuring resources.

Creating a resource.

Deleting resources.

Adding a resource pool.

Modifying a resource pool.

Deleting a resource pool.

Restoring tenant data.

Cluster management

Starting a cluster.

Stopping a cluster.

Saving the configurations.

Synchronizing the cluster configuration

Customizing cluster monitoring indicator.

Configuring monitoring dump.

Creating a cluster.

Uninstalling a cluster.

Saving monitoring threshold.

Downloading a client configuration file.

Configuring the northbound interface.

Configuring the northbound SNMP interface.

Creating a threshold template.

Deleting a threshold template.

Applying a threshold template.

Saving cluster monitoring configuration data.

Exporting configuration.

Importing cluster configuration data.

Exporting the installation template.

Modifying a threshold template.

Canceling the application of a threshold template.

Masking an alarm.

Sending an alarm.

Changing OMS database password.

Changing the component database password.

Restarting OMM and Controller.

Starting the health check of a cluster.

Updating the health check configuration.

Exporting cluster health check results.

Importing a certificate file.

Configuring SSO information.

Deleting historical health check reports.

Exporting historical health check reports.

Changing a cluster name.

Synchronizing maintenance commands.

Asynchronous maintenance commands.

Customizing report monitoring indicators.

Exporting report monitoring data.

SNMP runs an asynchronous command.

Restarting the web service.

Customizing monitoring indicators for static resource pools.

Exporting monitoring data of a static resource pool.

Service management

Adding a service.

Deleting a service.

Starting a service.

Stopping a service.

Synchronizing service configurations.

Refreshing a service queue.

Customizing service monitoring indicator.

Restarting a service.

Exporting service monitoring data.

Importing service configuration data.

Starting the health check of a service.

Exporting service health check results.

Service configuration

Uploading a configuration file.

Download a configuration file.

Instance management

Creating an instance.

Deleting an instance.

Synchronizing instance configurations.

Commissioning an instance.

Decommissioning an instance.

Starting an instance..

Stopping an instance.

Customizing instance monitoring indicators.

Restarting an instance.

Exporting instance monitoring data.

Importing instance configuration data.

Host management.

Adding a host.

Deleting a host.

Setting a node rack.

Starting all roles.

Stopping all roles.

Isolating a host.

Canceling isolation of a host.

Reinstalling a host.

Customizing host monitoring indicators.

Exporting host monitoring data.

Forcibly deleting a host.

Starting the health check of a host.

Exporting the health check result of a host.

Maintenance management

Exporting alarms.

Clearing alarms.

Exporting events.

Clearing alarms in batches.

SNMP clears alarms.

SNMP adds a trap target.

SNMP deletes a trap target.

SNMP checks alarms.

SNMP synchronizes alarms.

Modifying audit dump configurations.

Exporting audit logs.

Collecting log files.

Downloading log files.

Uploading a license file.

Activating a license.

Uploading a file.

Deleting an uploaded file.

Creating a backup task.

Executing a backup task.

Stopping a backup task.

Deleting a backup task.

Modifying a backup task.

Locking a backup task.

Unlocking a backup task.

Creating a backup restoration task.

Executing a backup restoration task.

Stopping a restoration task.

Retrying a restoration task.

Deleting a restoration task.

Table 12-6 Component audit information

Audit Log

Operation Type

Operation

DBService audit log

Maintenance management

Performing backup restoration operations.

Oozie audit log

Task management

Submitting a task.

Starting a task.

Killing a task.

Suspending a task.

Resuming a task.

Running a task again.

Solr audit log

Maintenance management

Authenticating users.

Creating a core.

Creating a collection.

Service operation

Creating an index.

Elasticsearch audit log

Maintenance management

Authenticating users.

Service operation

Creating an index.

HBase audit logs

DDL (data definition) statement

Creating a table.

Deleting a table.

Modifying a table.

Adding a column family.

Modifying a column family.

Deleting a column family.

Enabling a table.

Disabling a table.

Modifying user information.

Changing a password.

User login

DML (data operation) statement

Putting data (to the hbase:meta, _ctmeta_, and hbase:acl tables).

Deleting data (from the hbase:meta, _ctmeta_, and hbase:acl tables).

Checking and importing data (for the hbase:meta, _ctmeta_, and hbase:acl tables).

Checking and deleting data (the hbase:meta, _ctmeta_, and hbase:acl tables).

Permission control

Assigning permissions users.

Canceling user authorization.

Hive audit logs

Metadata operation

Defining metadata, such as creating databases and tables.

Deleting metadata, such as deleting databases and tables.

Modifying metadata, such as adding columns and renaming tables.

Importing and exporting metadata.

Data maintenance

Loading data to a table.

Inserting data into a table.

Rights management

Creating or deleting a role.

Granting/Reclaiming roles.

Granting/Reclaiming permissions.

HDFS audit logs

Rights management

File/Folder access permission.

File/folder owner information.

File operation

Creating a folder.

Creating a file.

Opening a file.

Appending file content.

Changing a file name.

Deleting a file or folder.

Setting time property of a file.

Setting the number of file copies.

Merging files.

Checking the file system.

File link.

MapReduce audit log

Application running

Starting a Container request.

Stopping a Container request.

After Container request is completed, the status of the request is displayed as succeeded.

After Container request is completed, the status of the request is displayed as failed.

After Container request is completed, the status of the request is displayed as suspended.

Submitting a task.

Ending a task.

LdapServer audit log

Maintenance management

Adding an operating system user.

Adding a user group.

Adding a user to user group.

Deleting a user.

Deleting a group.

KrbServer audit log

Maintenance management

Changing the password of a Kerberos account.

Adding a Kerberos account.

Deleting a Kerberos account.

Authenticating users.

Loader audit logs

Security management

User login.

Metadata management

Querying connector information.

Querying a framework.

Querying step information.

Data source connection management

Querying a data source connection.

Adding a data source connection.

Updating a data source connection.

Deleting a data source connection.

Activating a data source connection.

Disabling a data source connection.

Job management

Querying a job.

Creating a job.

Updating a job.

Deleting a job.

Activating a job.

Disabling a job.

Querying all execution records of a job.

Querying the latest execution record of a job.

Submitting a job.

Stopping a job.

Hue audit log

Service startup

Starting Hue.

User operation

User login.

User logout.

Task operations

Creating a task.

Modifying a task.

Deleting a task.

Submitting a task.

Saving a task.

Updating the status of a task.

ZooKeeper audit logs

Rights management

Setting access permission to Znode.

Znode operation

Creating Znodes.

Deleting Znodes.

Configuring ZNODE dataZnode

FTP-Server audit log

Login operation

Login.

File operations

Creating a file.

Deleting a file.

Creating a directory.

Moving a file.

Setting access permissions.

Metadata audit log

Task operations

Saving user-defined metadata tag information.

Saving the configurations.

Updating metadata.

Uploading FTP.

Automatically updating metadata.

Automatically uploading FTP.

Redis audit logs

Maintenance management

Creating Redis clusters.

Deleting a Redis cluster.

Scaling out a Redis cluster.

Scaling in a Redis cluster.

Balancing Redis cluster data.

Storm audit log

Nimbus

Submitting a topology.

Stopping a topology.

Reallocating a topology.

Deactivating a topology.

Activating a topology.

UI

Stopping a topology.

Reallocating a topology.

Deactivating a topology.

Activating a topology.

GraphBase audit log

Service operation

Adding, deleting, and modifying data.

FusionInsight Manager audit logs are stored in the database and can be downloaded on the Audit Management page. For details about how to download the audit logs, view and export audit logs, see.

The following table lists the directories to store component audit logs. Audit log files of some components are stored in /var/log/Bigdata/audit, such as HDFS, HBase, MapReduce, Hive, Hue, YARN, Redis, Storm, and ZooKeeper. The component audit logs are automatically compressed and backed up to /var/log/Bigdata/audit/bk at 03: 00 every day. A maximum of latest 90 compressed backup files are retained, and the backup time cannot be changed. Configure the number of reserved audit log files. For details, see Configuring the Number of Audit Log Backup Files.

Audit log files of other components are stored in the component log directory.

Table 12-7 Directory for storing component audit logs

Component

Audit Log Directory

DBService

/var/log/Bigdata/audit/dbservice/dbservice_audit.log

HDFS

/var/log/Bigdata/audit/hdfs/nn/hdfs-audit-namenode.log

/var/log/Bigdata/audit/hdfs/dn/hdfs-audit-datanode.log

/var/log/Bigdata/audit/hdfs/jn/hdfs-audit-journalnode.log

/var/log/Bigdata/audit/hdfs/zkfc/hdfs-audit-zkfc.log

/var/log/Bigdata/audit/hdfs/httpfs/hdfs-audit-httpfs.log

/var/log/Bigdata/audit/hdfs/router/hdfs-audit-router.log

Oozie

/var/log/Bigdata/audit/oozie/oozie-audit.log

Solr

/var/log/Bigdata/audit/solr/SolrServerAdmin/solr-audit-<hostname>.log

/var/log/Bigdata/audit/solr/SolrServerN/solr-audit-<hostname>.log

Elasticsearch

/var/log/Bigdata/audit/elasticsearch/esmaster/elasticsearch_cluster-audit.log

/var/log/Bigdata/audit/elasticsearch/esnodeN/elasticsearch_cluster-audit.log (The value of N ranges from 1 to 9)

Mapreduce

/var/log/Bigdata/audit/mapreduce/jobhistory/mapred-audit-jobhistory.log

Hive

/var/log/Bigdata/audit/hive/hiveserver/hive-audit.log

/var/log/Bigdata/audit/hive/metastore/metastore-audit.log

/var/log/Bigdata/audit/hive/webhcat/webhcat-audit.log

Loader

/var/log/Bigdata/loader/audit/default.audit

Hue

/var/log/Bigdata/audit/hue/hue-audits.log

ZooKeeper

/var/log/Bigdata/audit/zookeeper/quorumpeer/zk-audit-quorumpeer.log

Spark

/var/log/Bigdata/audit/spark/jdbcserver/jdbcserver-audit.log

/var/log/Bigdata/audit/spark/jobhistory/jobhistory-audit.log

Yarn

/var/log/Bigdata/audit/yarn/rm/yarn-audit-resourcemanager.log

/var/log/Bigdata/audit/yarn/nm/yarn-audit-nodemanager.log

FTP-Server

/var/log/Bigdata/ftp-server/ftp-server-audit.log

Metadata

/var/log/Bigdata/audit/metadata/metadata-audit.log

Redis

/var/log/Bigdata/audit/redis/redisweb_audit.log

/var/log/Bigdata/audit/redis/Redis_N/redis.log

Storm

/var/log/Bigdata/audit/storm/nimbus/audit.log

/var/log/Bigdata/audit/storm/ui/audit.log

GraphBase

/var/log/Bigdata/audit/graphbase/audit.log

Download
Updated: 2019-05-17

Document ID: EDOC1100074522

Views: 5986

Downloads: 12

Average rating:
This Document Applies to these Products
Related Documents
Related Version
Share
Previous Next