CODEWITHSUNDEEP
javahadoophivehdfs
Rama
Rama

Reputation: 875

Hive throws an error while creating table "Cannot validate serde: com.cloudera.hive.serde.JSONSerDe"

Working on apache-hive-0.13.1. while creating table hive throw an error as below 

FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. Cannot validate serde: com.cloudera.hive.serde.JSONSerDe

table structure is 

create external table tweets(id BigInt, created_at String, scource String, favorited Boolean, retweet_count int, 
retweeted_status Struct < 
    text:String,user:Struct< 
        screen_name:String, name:String>>,
    entities Struct<
        urls:Array<Struct<          
             expanded_url:String>>,
        user_mentions:Array<Struct<
            screen_name:String,
            name:String>>,
        hashtags:Array<Struct<text:String>>>,

text String,
user Struct<
    screen_name:String,
    name:String,
    friends_count:int,
    followers_count:int,
     statuses_count:int,
    verified:boolean, 
    utc_offset:int,
    time_zone:String> , 
in_reply_to_screen_name String)
partitioned by (datehour int)
ROW FORMAT SERDE 'com.cloudera.hive.serde.JSONSerDe'
location '/home/edureka/sachinG'

Added a json-serde-1.3.6-SNAPSHOT-jar-with-dependencies.jar in class to resolved the issue but no success

Upvotes: 3

Views: 8235

Answers (2)

KP Fingh
KP Fingh

Reputation: 79

I faced a similar issue while working with hive 1.2.1 and hbase 0.98. I followed below steps and the issue was resolved.

1) Copied all the hbase-* files from hbase/lib location to hive/lib directory

2) Verified that the hive-hbase-handler-1.2.1.jar was present in hive/lib

3) Verified that hive-serde-1.2.1.jar was present in hive/lib

4) Verified that zookeeper-3.4.6.jar was present in hive/lib(If not copy from hbase/lib and paste to hive/lib)

5) In the hive-site.xml(If not present use hive-default.xml.template) located at hive/conf under 'both'

a) hive.aux.jars.path field and

b) hive.added.jars.path field

give path '/usr/local/hive/lib/'.

6) Open hive terminal and create the table using below command:-

CREATE TABLE emp_hive (
RowKey_HIVE String,
Employee_No int,
Employee_Name String,
Job String,
Mgr int,
Hiredate String,
Salary int,
Commision int,
Department_No int
)
STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
WITH SERDEPROPERTIES ("hbase.columns.mapping" =":key,details:Employee_No_hbase,details:Employee_Name_hbase,details:job_hbase,details:Mgr_hbase,details:hiredate_hbase,details:salary_hbase,details:commision_hbase,details:department_no_hbase")
TBLPROPERTIES("hbase.table.name"="emp_hbase");

Upvotes: 1

Rama
Rama

Reputation: 875

Finally , got a solution for this. The issue is with json-serde-1.3.6-SNAPSHOT-jar-with-dependencies.jar Different distribution (Cloudera, Azure, etc ) needed different JSON-Serde jar file. Means, serde jar should be compatible to there distribution.

I changed jar and it worked for me.

Upvotes: 2

Related Questions