CODEWITHSUNDEEP
apache-sparkhadooppysparkhive
vikram bhati
vikram bhati

Reputation: 33

unable load data into hive using pyspark

unable to write the data into hive using pyspark through jupyter notebook .

giving me below error

Py4JJavaError: An error occurred while calling o99.saveAsTable. : org.apache.spark.sql.AnalysisException: java.lang.RuntimeException: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient;

Note these steps already tried:

  1. copied the hdfs-site.xml , core-site.xml to /conf of hive
  2. removed metasotore_db and created again using below cmd $HIVE_HOME/bin/schematool –initschema –dbtype derby

Upvotes: 0

Views: 296

Answers (1)

RacoonOnMoon
RacoonOnMoon

Reputation: 1586

did you use spark-submit for running your script? Also you should add -> ".enableHiveSupport()" like that:

spark = SparkSession.builder \
    .appName("yourapp") \
    .enableHiveSupport() \
    .getOrCreate()

Upvotes: 0

Related Questions