CODEWITHSUNDEEP
jarhadoopclasspath
Vyacheslav
Vyacheslav

Reputation: 1256

Setting additional classpath for a hadoop tool

How can I dynamically set up an additional classpath so that jars are accessible from a hadoop tool class? I'm running my job via "hadoop jar". I need something like the "-libjars" switch which unfortunately works only for mappers and reducers, and my Tool implementations does not see jars provided by "libjars"

export "HADOOP_CLASSPATH=$HADOOP_CLASSPATH:/path/to/my.jar && hadoop jar ..." seems making "my.jar" accessible to the Tool, but then libraries that are inside the job jar in the "/lib" folder become unaccessible.

Upvotes: 5

Views: 7182

Answers (1)

Vyacheslav
Vyacheslav

Reputation: 1256

Additionally, if you have conflicts with some jars included into a Hadoop distribution, you can export HADOOP_USER_CLASSPATH_FIRST=true. Maybe will be useful to somebody.

Upvotes: 2

Related Questions