CODEWITHSUNDEEP
scalaapache-spark-sqlapache-spark-2.0
Nisha Gupta
Nisha Gupta

Reputation: 38

"java.lang.IllegalAccessError" while reading multiline json file in Spark2.0

while reading multiline json file in Spark2.0 getting exception

val data = spark.read
    .option("multiline",true)
    .json("C:\\user\\Spark\\DataSets\\employees_multiLine.json")

Exception in thread "main" java.lang.IllegalAccessError: tried to access method com.google.common.base.Stopwatch.()V from class org.apache.hadoop.mapreduce.lib.input.FileInputFormat at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.listStatus(FileInputFormat.java:262) at org.apache.spark.input.StreamFileInputFormat.setMinPartitions(PortableDataStream.scala:51) at org.apache.spark.rdd.BinaryFileRDD.getPartitions(BinaryFileRDD.scala:51) at org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:253) at org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:251)

Upvotes: 1

Views: 999

Answers (1)

Manjula N
Manjula N

Reputation: 46

Updating hadoop to 2.7.2 or higher will resolve this.

This issue is explained here in detail https://stackoverflow.com/a/36443787

Upvotes: 2

Related Questions