CODEWITHSUNDEEP
hadoopmapreduce
Terminal User
Terminal User

Reputation: 883

TaskTracker use local libraries

When the tasktracker has been assigned a task, it that possible the tasktracker start the task with the library on its local system?

If so, what is the command to enable using local library?

Upvotes: 0

Views: 208

Answers (1)

Matthew Rathbone
Matthew Rathbone

Reputation: 8269

All jars in the local HADOOP_HOME/lib will be added to the class path, so you can just stick stuff in there if you want a quick hack.

The proper way to do it is to explicitly add stuff to the distributed cache before you start your job, then that library will be available to every task of your Map Reduce job.

DistributedCache.addFileToClassPath(new Path("hdfs://path/to/lib.jar"), jobConf)

Upvotes: 1

Related Questions