Hadoop JAR command - Setting java.library.path

Question

I'm trying to run a java program on Hadoop cluster. Here's the command-

export HADOOP_CLASSPATH=$HADOOP_CLASSPATH:/usr/local/lib/*:/home/rgupta/bdAnalytics/lib/*
hadoop jar $jarpath bigdat.twitter.queue.TweetOMQSub >  $logsFldr/subsHdpOMQ_$1.log 2>&1 &
#java -Djava.library.path=/usr/local/lib -classpath  class/:lib/:lib/jzmq-2.1.3.jar  bigdat.twitter.queue.TweetOMQSub > log/subsFilterOMQ_$1.log 2>&1 &

---

This throws following error -

Exception in thread "main" java.lang.UnsatisfiedLinkError: no jzmq in java.library.path

If I use the Java native command above, it works fine. Also, the hadoop node where I m trying to test it, does have the necessary jzmq jars under /usr/local/lib directory. Is there a way I can set java.library.path to Hadoop JAR command. Please suggest how can I fix this.

Answer 1

Try export HADOOP_OPTS=$HADOOP_OPTS -Djava.library.path=/usr/local/lib

and export other jars the usual way you are doing before running a job - using HADOOP_CLASSPATH

Hope this helps.

Answer 2

sorry misread your question so editing:

you should be able to use the libjars option

In your case:

HADOOP_CLASSPATH=$HADOOP_CLASSPATH:/usr/local/lib/:/home/rgupta/bdAnalytics/lib/

hadoop jar $jarpath bigdat.twitter.queue.TweetOMQSub -libjars /usr/local/lib ...