CODEWITHSUNDEEP
hadoophivehiveqlhive-serde
Andy Reddy
Andy Reddy

Reputation: 93

Error while trying to load json file into external table in hive

Data looks like:

{"custId":1185972,"movieId":null,"genreId":null,"time":"2012-07-01:00:00:07","recommended":null,"activity":8}

The query i am running is:

add jar /home/student/hive-0.11.0-bin/lib/json-serde-1.3.7-jar-with-dependencies.jar;

CREATE EXTERNAL TABLE movie_json 
( custId INT, movieId INT, genreId INT, 
time STRING, recommended STRING, activity INT, rating INT, price FLOAT ) 
ROW FORMAT SERDE 'org.openx.data.jsonserde.JsonSerDe' 
LOCATION '/user/oracle/movie/';

The error encountered is:

java.lang.NoSuchFieldError: byteTypeInfo at org.openx.data.jsonserde.objectinspector.primitive.TypeEntryShim.(TypeEntryShim.java:27) at org.openx.data.jsonserde.objectinspector.primitive.JavaStringJsonObjectInspector.(JavaStringJsonObjectInspector.java:14) at org.openx.data.jsonserde.objectinspector.JsonObjectInspectorFactory.(JsonObjectInspectorFactory.java:196) at org.openx.data.jsonserde.JsonSerDe.initialize(JsonSerDe.java:125) at org.apache.hadoop.hive.metastore.MetaStoreUtils.getDeserializer(MetaStoreUtils.java:215) at org.apache.hadoop.hive.ql.metadata.Table.getDeserializerFromMetaStore(Table.java:268) at org.apache.hadoop.hive.ql.metadata.Table.getDeserializer(Table.java:261) at org.apache.hadoop.hive.ql.metadata.Table.getCols(Table.java:587) at org.apache.hadoop.hive.ql.metadata.Hive.createTable(Hive.java:576) at org.apache.hadoop.hive.ql.exec.DDLTask.createTable(DDLTask.java:3776) at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:256) at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:144) at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:57) at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1355) at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1139) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:945) at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:259) at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:216) at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:413) at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:756) at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:614) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.hadoop.util.RunJar.main(RunJar.java:156) FAILED: Execution Error, return code -101 from org.apache.hadoop.hive.ql.exec.DDLTask

I used various JsonSerder jars but getting the same error. Please help me.

Upvotes: 0

Views: 774

Answers (1)

Indrajit Swain
Indrajit Swain

Reputation: 1483

Not sure about the JsonSerDe you are using . Bu here this JsonSerDeyou can use.

Hive-JSON-Serde

hive> add jar /User/User1/json-serde-1.3.8-SNAPSHOT-jar-with-dependencies.jar;
Added [/User/User1/json-serde-1.3.8-SNAPSHOT-jar-with-dependencies.jar] to class path
Added resources: [/User/User1/json-serde-1.3.8-SNAPSHOT-jar-with-dependencies.jar]
hive>CREATE EXTERNAL TABLE movie_json ( custId INT, movieId INT, genreId INT, time STRING, recommended STRING, activity INT, rating INT, price FLOAT ) ROW FORMAT SERDE'org.openx.data.jsonserde.JsonSerDe'
LOCATION'/user/oracle/movie/';
OK
Time taken: 0.097 seconds

you can build Jar by using 

C:\Users\User1\Downloads\Hive-JSON-Serde-develop\Hive-JSON-Serde-develop>mvn -Phdp23 clean package.
-Phdp23 is hdp2.3 it should be replaced with your hadoop version.

You can also use inbuilt jsonserde get_json_object json_tupleIf you are looking for example how to use inbuilt jsonserde look at this example Example.

I will suggest validate your Json Validate JSON

Upvotes: 0

Related Questions