CODEWITHSUNDEEP
hadoopmapreduce
user1716097
user1716097

Reputation: 9

Multiple files in mapreduce

I am very new to mapreduce program.I am following Tom white Hadoop Definitive guide.I am doing the weather data program in which I want to find the maximum temperature per year. I have 4 files for the year 1901,1902,1904,1905, one file per year(I have made a single file per year).

how do I set the four input file in a mapreduce maper program. Hadoop is installed on pseudo distributed mode . Please help me.

Upvotes: 0

Views: 1447

Answers (2)

amit dwivedi
amit dwivedi

Reputation: 1

hadoop picks all the files from the input directory. So if you put all you files into input directory, all will be picked. You can set multiple input paths as well in your driver class like this.

FileInputFormat.setInputPaths(job, commaSeparatedPaths);

Upvotes: 0

luoluo
luoluo

Reputation: 5533

If you are using hadoop streaming, try this:

$HADOOP_HOME/bin/hadoop  jar $HADOOP_HOME/hadoop-streaming.jar \
-input myInputDirs \
-output myOutputDir \
-mapper mapper \
-reducer reducer

Put your files to the input directory would solve your problem.

Full documents

Upvotes: 1

Related Questions