CODEWITHSUNDEEP
hadoopmapreducehadoop-yarnmrv2
Novice
Novice

Reputation: 155

MRv1 and MRv2 parameters

Entire list of parameter( for Hadoop-2.6) is given on the link

But you can execute a job in either MRv1 or MRv2 style. I think there are some parameters that are only applicable for MRv1 like mapreduce.tasktracker.map.tasks.maximum, is this true ? If yes, then is there any smarter way to figure out all such parameters ? can we pass all parameters via -Dproperty=value or -D property=value or is there any parameter, that i can not pass like this ?

Upvotes: 0

Views: 296

Answers (1)

alekya reddy
alekya reddy

Reputation: 934

In MR2, the number of map tasks will be calculated dynamically by resource manager depending on the memory assigned to the each container.

From hortonworks

With YARN and MapReduce 2, there are no longer pre-configured static slots for Map and Reduce tasks. The entire cluster is available for dynamic resource allocation of Maps and Reduces as needed by the job

This page has got more info. check this http://hortonworks.com/blog/how-to-plan-and-configure-yarn-in-hdp-2-0/

You can pass any parameter using -D property = value from command line. There is no restriction on that. But if these properties are set in code by developer, the command line property will be overidden by job.

Upvotes: 1

Related Questions