CODEWITHSUNDEEP
hadoophbasehive
LMHadoop
LMHadoop

Reputation: 57

Hadoop Hive integration INSERT query

i'am hadoop newbie,and i'am trying this tutorial: https://cwiki.apache.org/confluence/display/Hive/HBaseIntegration

1.Starting hive is done successfully with the parameter: hive --auxpath /cygdrive/c/Hadoop/hive-0.9.0/lib/hive-hbase-handler-0.9.0.jar,/cygdrive/c/javaHBase/hbase-0.94.6/hbase-0.94.6.jar,/cygdrive/c/Hadoop/hive-0.9.0/lib/zookeeper-3.4.3.jar,/cygdrive/c/Hadoop/hive-0.9.0/lib/guava-r09.jar -hiveconf hbase.master=localhost:60010

2.starting hbase is successuful.

3."CREATE TABLE hbase_table_1" is done successfully

4.I verified with commands list and show tables, all is ok

here is my problem "INSERT OVERWRITE TABLE hbase_table_1 SELECT * FROM pokes WHERE foo=98;"

i get this error message searching in "htp://localhost:50060/tasklog?attemptid..."

java.lang.ClassNotFoundException: org/apache/hadoop/hive/hbase/HBaseSerDe
Continuing ...
java.lang.ClassNotFoundException:
org/apache/hadoop/hive/hbase/HiveHBaseTableInputFormat
Continuing ...
java.lang.ClassNotFoundException:
org/apache/hadoop/hive/hbase/HiveHBaseTableOutputFormat
Continuing ...
java.lang.NullPointerException
Continuing ...
java.lang.NullPointerException
 at
org.apache.hadoop.hive.ql.exec.FileSinkOperator.initializeOp(FileSinkOperator.java:280)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:357)
    at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:433)
    at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:389)
    at org.apache.hadoop.hive.ql.exec.SelectOperator.initializeOp(SelectOperator.java:62)
    at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:357)
    at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:433)
    at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:389)
    at org.apache.hadoop.hive.ql.exec.FilterOperator.initializeOp(FilterOperator.java:78)
    at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:357)
    at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:433)
    at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:389)
...

i tried to copy hive jars to hbase install and vice vesa... note: i added necessary JARS with hive command: ADD JAR C:...\hive-hbase-handler-0.9.0.jar etc.

hbase version: 0.94.6 hive version: 0.9.0

any additional export? or configuration? I need help please! Thanks a lot!

Upvotes: 1

Views: 2723

Answers (2)

user3446207
user3446207

Reputation:

Brother do following things. your problem will be solved. ( change according to your version )

copy these files in hadoop lib directory $HIVE_HOME/lib/hive-hbase-handler-0.8.1.jar,$HIVE_HOME/lib/hbase-0.90.0.jar,$HIVE_HOME/lib/zookeeper-3.3.1.jar,$HIVE_HOME/lib/guava-r06.jar, and hive-serde jar

Upvotes: 1

Jaydeep Parikh
Jaydeep Parikh

Reputation: 91

Try adding these jars in ditributed cache by running following commands in hive CLI or include these lines in $HOME/.hiverc file. That should resolve the ClassNotFoundException.

ADD JAR ...../hive-0.9.0/lib/hive-hbase-handler-0.9.0.jar;
ADD JAR ...../hbase-0.94.1/hbase-0.94.6.jar;
ADD JAR ...../hbase-0.94.1/lib/zookeeper-3.4.3.jar;
ADD JAR ...../hbase-0.94.1/lib/guava-11.0.2.jar;
ADD JAR ...../hbase-0.94.1/lib/protobuf-java-2.4.0a.jar;

This is required when running hive queries in mapred mode (not local).

Upvotes: 2

Related Questions