Ganga Dhar
Ganga Dhar

Reputation: 19

Unable to create external table in elasticsearch using es-hadoop

i am running a simple spark-submit job, e.g.:

enter code here spark-submit --class com.x.y.z.logan 
 /home/test/spark/sample.jar

  table in jar file

   hiveContext.sql("CREATE TABLE IF NOT EXISTS 
   databasename.tablename(es_column_name STRING) STORED BY 
   'org.elasticsearch.hadoop.hive.EsStorageHandler' 
    TBLPROPERTIES('es.resource' = 'index_name/log','es.mapping.names' 
    ='tablecoulmname :es_column_name ', 'es.nodes' = 
     '192.168.x.1y:9200','es.input.json' = 'false', 
      'es.index.read.missing.as.empty' = 'yes' ,'es.index.auto.create' = 
     'yes') ")

      hiveContext.sql("INSERT INTO TABLE test.incompleterun SELECT 
      s.streamname FROM incomplete s");



       **ERROR**
      client token: N/A
       diagnostics: User class threw exception: java.lang.RuntimeException: 
      java.lang.RuntimeException: class 
      org.elasticsearch.hadoop.mr.EsOutputFormat$EsOutputCommitter not 
      org.apache.hadoop.mapred.OutputCommitter
      ApplicationMaster host: 192.168.x.y
      ApplicationMaster RPC port: 0
      queue: root.users.test
      start time: 1485286033939
      final status: FAILED
      tracking URL: some URL
      user: test
       Exception in thread "main" org.apache.spark.SparkException: 
      Application application_1485258812942_0008 finished with failed status
      at org.apache.spark.deploy.yarn.Client.run(Client.scala:1035)
      at org.apache.spark.deploy.yarn.Client$.main(Client.scala:1082)
      at org.apache.spark.deploy.yarn.Client.main(Client.scala)
      at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

ES-hadoop is working fine when we are creating external table in Hive interface and loading data into external table from hive table. it is not working when we include the same query in jar. the same jar file works fine when we are creating normal hive table. The problem here is showing the below error when we include external table in jar file CAN ANYONE HELP ME HOW TO FIX THIS ?

Upvotes: 1

Views: 419

Answers (0)

Related Questions