CODEWITHSUNDEEP
hadoopmapreducehadoop2
HHH
HHH

Reputation: 6485

How to change the FileSplit size in new Hadoop api

As in new Hadoop api, the number of mappers can't be explicitly set, I'd like to change the SplitSize to a smaller number than the default so that more mappers get executed. How can I do that in the new Hadoop api?

I have tried the following , but it doesn't work

 conf.setLong(FileInputFormat.Split_MAXSIZE, 16 * 1024 * 1024)

Upvotes: 1

Views: 138

Answers (2)

Ravindra babu
Ravindra babu

Reputation: 38950

You have to change below parameters as per your requirement.

mapreduce.input.fileinputformat.split.minsize

and

mapreduce.input.fileinputformat.split.maxsize

Have a look at this link for more details.

Upvotes: 2

Durga Viswanath Gadiraju
Durga Viswanath Gadiraju

Reputation: 3956

It is mapreduce.input.fileinputformat.split.minsize. If it does not work, please share your code and I will have a look at it.

Upvotes: 0

Related Questions