CODEWITHSUNDEEP
hadoophiveprotocol-bufferselephantbird
Avinash
Avinash

Reputation: 1383

Hive Protocol Bufferer - NullPointerException while creating table in Hive

Thanks in advance.

Currently we are trying to create Hive table by using Protocol Buffers byte data. We have followed all possible steps of creating Hive table by using Protocol buffer, however getting NullPointerException while creating Hive table. Below are the all required details.

Versions -
1. protoc 3.0.0
2. elephant-bird - 4.14
3. Hortornworks Sandbox Hive version - 1.2.1
4. Protobuf-java 3.0.0

The proto file used is 

package tutorial; 
option java_package = "com.mycom.hive.protobuf.serialized";
option java_outer_classname = "BankProtoTest";
message BankClass{
    required string bankAmount= 1;
    required string bankLocation= 2;
    optional string bankName= 3;
}
message BankInfo {
   repeated BankClass bankClass = 1;
}

We are creating the Java class by using below command

protoc.exe -I=input-proto --java_out=java-output input-proto\BankProto.proto

The above command generates the Java class for input protocol buffer file.

After this we copied this protocol buffer java file to a maven java project and then created JAR file. We copied the JAR file into hive lib path i.e. '/usr/hdp/current/hive-client/lib'.

Below is the create table command 

create external table bankproto 
   row format serde "com.twitter.elephantbird.hive.serde.ProtobufDeserializer"     
with serdeproperties 
   ("serialization.class"="com.mycom.hive.protobuf.serialized.BankProtoTest$BankInfo")
stored as  
   inputformat "org.apache.hadoop.mapred.SequenceFileInputFormat" 
   outputformat "org.apache.hadoop.mapred.SequenceFileOutputFormat" 
location '/user/root/protobuf-input/';

The input file present in the location is stored as sequence file in HDFS.
After executing this command getting below exception.

FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. java.lang.RuntimeException: MetaException(message:org.apache.hadoop.hive.serde2.SerDeException java.lang.NullPointerException)

Any help related to this is appreciated.

Thanks again.

Avinash Deshmukh

Upvotes: 0

Views: 609

Answers (1)

Avinash
Avinash

Reputation: 1383

We are able to resolve this issue. The issue was about the correct compatible version of protobuf version. We found that current version of elephant-bird (4.14) is depend on protobuf version 2.6.0

Upvotes: 0

Related Questions