CODEWITHSUNDEEP
hadoophivehbaseintegration
Koushik Chandra
Koushik Chandra

Reputation: 1491

Unable to create hive table for hbase

I am running with hive version 1.1.0 , Hbase version 1.0.1 and hadoop version 2.7.0. Now for the below command I am getting error

hive> CREATE TABLE hbase_table_1(eid int, ename string, esal double) 
    > STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
    > WITH SERDEPROPERTIES 
    > ("hbase.columns.mapping" = ":key,cfstr:enm,cfsal:esl")
    > TBLPROPERTIES ("hbase.table.name" = "emp1");
FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. org.apache.hadoop.hbase.HTableDescriptor.addFamily(Lorg/apache/hadoop/hbase/HColumnDescriptor;)V

If I create a table in HBase and refer that from Hive using external table, that is working fine.

In the hive home I have created an auxlib directory and copied the below jar files.

hduser@ubuntu:/usr/lib/hive/auxlib$ ls
commons-beanutils-1.7.0.jar       commons-httpclient-3.0.1.jar       hbase-checkstyle-1.0.1.jar      hbase-protocol-1.0.1.jar
commons-beanutils-core-1.8.0.jar  commons-io-2.4.jar                 hbase-client-1.0.1.jar          hbase-rest-1.0.1.jar
commons-cli-1.2.jar               commons-lang-2.6.jar               hbase-common-1.0.1.jar          hbase-server-1.0.1.jar
commons-codec-1.4.jar             commons-logging-1.1.3.jar          hbase-common-1.0.1-tests.jar    hbase-server-1.0.1-tests.jar
commons-collections-3.2.1.jar     commons-math-2.1.jar               hbase-examples-1.0.1.jar        hbase-shell-1.0.1.jar
commons-compiler-2.7.6.jar        commons-pool-1.5.4.jar             hbase-hadoop2-compat-1.0.1.jar  hbase-testing-util-1.0.1.jar
commons-compress-1.4.1.jar        commons-vfs2-2.0.jar               hbase-hadoop-compat-1.0.1.jar   hbase-thrift-1.0.1.jar
commons-configuration-1.6.jar     guava-14.0.1.jar                   hbase-it-1.0.1.jar              hive-hbase-handler-1.1.0.jar
commons-dbcp-1.4.jar              hbase-annotations-1.0.1.jar        hbase-it-1.0.1-tests.jar        zookeeper-3.4.6.jar
commons-digester-1.8.jar          hbase-annotations-1.0.1-tests.jar  hbase-prefix-tree-1.0.1.jar

The below details are set in .bashrc file

export JAVA_HOME=/usr/lib/jvm/jdk1.8.0/
export JAVA_LIBRARY_PATH=$HADOOP_HOME/lib/native:$JAVA_LIBRARY_PATH
#alias java="`which java` -Dwhatever"

export HADOOP_HOME=/usr/local/hadoop
export HADOOP_MAPRED_HOME=$HADOOP_HOME
export HADOOP_COMMON_HOME=$HADOOP_HOME
export HADOOP_HDFS_HOME=$HADOOP_HOME
export YARN_HOME=$HADOOP_HOME
export HADOOP_COMMON_LIB=$HADOOP_HOME/share/hadoop/common/lib
export HADOOP_TOOLS_LIB=$HADOOP_HOME/share/hadoop/tools/lib
export HADOOP_COMMON_LIB_NATIVE_DIR=$HADOOP_HOME/lib/native
export HADOOP_OPTS="-Djava.library.path=$HADOOP_HOME/lib"
export HIVE_HOME="/usr/lib/hive"
export HBASE_HOME="/usr/lib/hbase"
export PATH=$JAVA_HOME/bin:$HADOOP_HOME/bin:$HADOOP_HOME/sbin:$HIVE_HOME/bin:$HBASE_HOME/bin:$HADOOP_COMMON_LIB:$HADOOP_TOOLS_LIB:$PATH

Can you please suggest what is wrong here actually.

Upvotes: 2

Views: 2283

Answers (4)

Ha The Trinh
Ha The Trinh

Reputation: 11

To day I got same issue, and build hive-src again with new pom.xml config Please refer and change this line in pom.xml enter code here1.1.2

Note: I use Hadoop 2.7.1, Hive 1.2.1 and Hbase 1.1.2

Upvotes: 0

Koushik Chandra
Koushik Chandra

Reputation: 1491

This issue got fixed after using hbase version 0.98.14 with hive version 1.1.0; plz visit http://stackoverflow.com/questions/32701905/unable-to-load-data-in-hbase-table-‌​‌​from-hive for details

Upvotes: 0

Afshin Moazami
Afshin Moazami

Reputation: 2098

At the current time, according to HBaseIntegration

Hive 2.x will be compatible with HBase 1.x and higher. (See HIVE-10990 for details.) Consumers wanting to work with HBase 1.x using Hive 1.x will need to compile Hive 1.x stream code themselves.

In short, until Hive 2.x release, you need to compile the Hive 1.x stream code yourself!

Upvotes: 0

Rubén Moraleda
Rubén Moraleda

Reputation: 3067

It seems you're missing the hbase.jar

First check your hive-site.xml file for the property hive.aux.jars.path and make sure hbase.jar, zookeeper.jar & hive-hbase-handlerXXXX.jar are specified.

After that, enter into the hive shell and run list jars to make sure they've been loaded. You can also load them manually (for your current session) with the command add jar [your-jar].jar

Additionally, if you don't want to mess with configuration files, you can create your own .hiverc file in your home path with your preferred initialization commands, i.e:

SET hive.cli.print.header=true;
SET hbase.scan.cacheblock=0;
SET hbase.scan.cache=10000;
SET hbase.client.scanner.cache=10000;
add JAR /usr/lib/hive/lib/zookeeper.jar;
add JAR /usr/lib/hive/lib/hbase.jar;
add JAR /usr/lib/hive/lib/hive-hbase-handler-0.10.0-cdh4.7.1.jar;
...

Upvotes: 0

Related Questions