CODEWITHSUNDEEP
hadoop
nd07
nd07

Reputation: 127

Limit number of concurrent map tasks in Hadoop 2.6.0

I want to restrict number of simultaneously map tasks running at one slave node.

In my case, when I submit my job, Hadoop generate 8 map tasks, when I looked at Job history UI at port 19888, I always saw that 8 map tasks started at the same time at same slave node.

Even I tried to set this attribute mapreduce.tasktracker.map.tasks.maximum equal to 4 (from how to restrict the concurrent running map tasks?). It still didn't work for me.

Does anyone have a experiment to deal with this issue?

Upvotes: 0

Views: 1327

Answers (1)

Ronak Patel
Ronak Patel

Reputation: 3849

for hadoop 2.6 it should be newer API - see this mapred-default.xml

set mapreduce.tasktracker.map.tasks.maximum to 4

The maximum number of map tasks that will be run simultaneously by a task tracker.

Upvotes: 1

Related Questions