No relevant resource is found in the selected language.

This site uses cookies. By continuing to browse the site you are agreeing to our use of cookies. Read our privacy policy>Search

Reminder

To have a better experience, please upgrade your IE browser.

upgrade

FusionInsight HD V100R002C60SPC200 Product Description 06

Rate and give feedback :
Huawei uses machine translation combined with human proofreading to translate this document to different languages in order to help you better understand the content of this document. Note: Even the most advanced machine translation cannot match the quality of professional translators. Huawei shall not bear any responsibility for translation accuracy and it is recommended that you refer to the English document (a link for which has been provided).
YARN

YARN

Task Priority Scheduling

In the native YARN resource scheduling mechanism, if the whole Hadoop cluster resources are occupied by those MapReduce jobs submitted earlier, jobs submitted later will be kept in pending state until all running jobs are executed and resources are released.

Huawei provides a mechanism for scheduling tasks by priority. With this feature, you can define jobs of different priorities. Jobs of high priority can preempt resources of jobs of low priority though they are submitted later. Jobs of low priority will be suspended and cannot start unless those jobs of high priority are completed and resources are released.

This feature enables services to flexibly control their own computing tasks, achieving optimal utilization of cluster resources.

NOTE:

The Container Reuse feature is in conflict with the Task Priority Scheduling feature. If Container Reuse is enabled, resources will not be released. In other words, the Task Priority Scheduling feature does not work.

Figure 4-12 Task Priority Scheduling

The Timeout Parameter can be Set When a User Submits a MapReduce Job

Open-source function: If a MapReduce job is executed for a long time, it will be suspended. Users have to wait, but cannot determine the cause.

Therefore, a timeout parameter -Dapplication.timeout.interval = is added for setting the MapReduce job execution timeout. The unit of the timeout parameter is second. If the job execution time expires, then the job will be stopped.

yarn jar <App_Jar_Name> [Main_Class] -Dapplication.timeout.interval = <timeout>

NOTE:

The value should be an integer. If the value is or not configured (null), then the execution timeout function will not be executed. If the configured value is invalid (other than integer value), then by default the value is set to 5 minutes.

Translation
Download
Updated: 2019-04-10

Document ID: EDOC1000104139

Views: 5887

Downloads: 64

Average rating:
This Document Applies to these Products
Related Documents
Related Version
Share
Previous Next