Yet Another Resource Negotiator (YARN) is a resource management & job scheduling component in Hadoop distribution framework. It decouples the resource management task from data processing, enabling it to support various data processing tools such as Map Reduce & Spark resulting greater scalability & resource utilization.
Map Reduce is a data processing mechanism where parallel computation of large task is split into smaller sub-tasks and each sub-task is computed on separate nodes within a Hadoop cluster. Map reduce is I/O instensive activity
Installing & Configuring Steps
- From Ambari UI select “Add Service“
From Ambari U In “Add Service Wizard” select “YARN + MapReduce2“
Assign services to cluster nodes
Assign Slaves & Clients
Customize service configuration
Review and deploy
Install the services
Review and complete installation
In Ambari UI, review the installed service
KNOWN ISSUE
- mapreduce will not start with the below error message
- resource_management.core.exceptions.Fail: Execution of ‘curl -sS -L -w ‘%{http_code}’ -X PUT –data-binary @/usr/hdp/2.4.3.0-227/hadoop/mapreduce.tar.gz ‘http://snamenode.galaxy.ln:50070/webhdfs/v1/hdp/apps/2.4.3.0-227/mapreduce/mapreduce.tar.gz?op=CREATE&user.name=hdfs&overwrite=True&permission=444‘ 1>/tmp/tmps2GLdy 2>/tmp/tmp721H6s’ returned 52. curl: (52) Empty reply from server 100
- Work around : Run the below command
- sh – hdfs
- hdfs dfs -put /usr/hdp/2.4.3.0-227/hadoop/mapreduce.tar.gz /hdp/apps/2.4.3.0-227/mapreduce/