CODEWITHSUNDEEP
javahadoopmapreduce
Hard Worker
Hard Worker

Reputation: 1121

Hadoop Map Reduce let addInputPath work with spesific file name

Hey this is more a java question but it is related to Hadoop .

I have this line on code in my Map Reduce java Job :

 JobConf conf= new JobConf(WordCount.class);
 conf.setJobName("Word Count");
       .............
       .............
       .............
 FileInputFormat.addInputPath(conf, new Path(args[0]));

instead of "giving" a directory with many files how do i set specific file name ?

Upvotes: 0

Views: 970

Answers (2)

yurgis
yurgis

Reputation: 4077

From the book "Hadoop: The Definitive Guide":

An input path is specified by calling the static addInputPath() method on FileInputFormat, and it can be a single file, a directory (in which case the input forms all the files in that directory), or a file pattern. As the name suggests, addInputPath() can be called more than once to use input from multiple paths.

So to answer your question, you should be able to just pass a path to your specific single file, and it will be used as an only input (as long as you do not do more calls of addInputPath() with some other paths).

Upvotes: 2

shutch
shutch

Reputation: 197

If you only want to do map-reduce stuff on one file, a quick and easy work around is to move that file only into a folder by itself and then provide that folder's path to your addInputPath.

If you're trying to read a whole file per map task then might I suggest taking a look at this post: Reading file as single record in hadoop

What exactly are you trying to do?

I would have posted this as a comment, but I don't have sufficient priviledges apparently...

Upvotes: 1

Related Questions