CODEWITHSUNDEEP
hadoop
Anilkumar Vudugula
Anilkumar Vudugula

Reputation: 1

How to know the number of input splits for a mapper in mapreduce program?

Is there any way that we can know number of input splits before adding them to the output collector in mapreduce program

I tried with below code

opCol.collect(new Text(Integer.toString(total)), new IntWritable(
                Count))

Any help would appreciated. Thanks.

Upvotes: 0

Views: 590

Answers (1)

SMA
SMA

Reputation: 37023

If you check the FileInputFormat class's getSplits method, it sets the number of split as job.getConfiguration().setLong(NUM_INPUT_FILES, files.size());

So from your mapper using int splitCount = context.getConfiguration().get("mapreduce.input.fileinputformat.numinputfiles");

Upvotes: 2

Related Questions