WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable

Question

alpesh@alpesh-Inspiron-3647:~/hadoop-2.7.2/sbin$ hadoop fs -ls 16/07/05 13:59:17 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable

It is also showing me the the output as below

hadoop check native -a 16/07/05 14:00:42 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable Native library checking: hadoop: false zlib: false snappy: false lz4: false bzip2: false openssl: false 16/07/05 14:00:42 INFO util.ExitUtil: Exiting with status 1

Please help me to solve this

Answer 1

Add these properties in bash profile of Hadoop user, the issue will be solved

export HADOOP_COMMON_LIB_NATIVE_DIR=$HADOOP_HOME/lib/native

export HADOOP_OPTS="-Djava.library.path=$HADOOP_COMMON_LIB_NATIVE_DIR"

Answer 2

It's just a warning, because it can not find the correct .jar. Either by compiling it or because it does not exist.

If I were you, I would simply omit it

To do that add in the corresponding configuration file

log4j.logger.org.apache.hadoop.util.NativeCodeLoader=ERROR

Answer 3

To get rid of this error:

Suppose Jar file is at /home/cloudera/test.jar and class file is at /home/cloudera/workspace/MapReduce/bin/mapreduce/WordCount, where mapreduce is the package name.

Input file mytext.txt is at /user/process/mytext.txt and output file location is /user/out.

We should run this mapreduce program in following way:

$hadoop jar /home/cloudera/bigdata/text.jar mapreduce.WordCount /user/process /user/out

Answer 4

Library you are using is compiled for 32 bit and you are using 64 bit version. so open your .bashrc file where configuration for hadoop exists. Go to this line

export HADOOP_OPTS="-Djava.library.path=$HADOOP_INSTALL/lib"

and replace it with

export HADOOP_OPTS="-Djava.library.path=$HADOOP_INSTALL/lib/native"