CODEWITHSUNDEEP
apache-sparkhadoophbase
sangeeth sasidharan
sangeeth sasidharan

Reputation: 167

When Running Spark job in hadoop cluster i am getting java.lang.NoClassDefFoundError: org/apache/hadoop/hbase/HBaseConfiguration

When i tried to run my scala code which connects hbase database it works perfectly in my local IDE . But when i run the same in hadoop cluster i am getting "Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/hadoop/hbase/HBaseConfiguration" error .

Please help me in this

Upvotes: 0

Views: 1699

Answers (1)

Ravi Mishra
Ravi Mishra

Reputation: 30

Add all the HBase library jars to HADOOP_CLASSPATH -

export HBASE_HOME="YOUR_HBASE_HOME_PATH"
export HADOOP_CLASSPATH="$HADOOP_CLASSPATH:$HBASE_HOME/lib/*"

You can append any external jar needed to HADOOP_CLASSPATH, so that you don't need to explicitly set it in spark-submit command. All dependent jars will be loaded and provided to your Spark application.

Upvotes: 1

Related Questions