CODEWITHSUNDEEP
azure-storageazure-blob-storageapache-spark-sql
Shibu
Shibu

Reputation: 1542

java.lang.ClassNotFoundException: com.microsoft.azure.storage.blob.BlobListingDetails Exception

I am trying to read a table that is on a azure blob storage via pyspark and the below exception is raised even though I have added the below jars in the pyspark --jars.

azure-storage-2.0.0.jar
hadoop-azure-2.7.0.jar

Exception:

py4j.protocol.Py4JJavaError: An error occurred while calling o38.showString.
: java.lang.NoClassDefFoundError: com/microsoft/azure/storage/blob/BlobListingDetails
Caused by: java.lang.ClassNotFoundException: com.microsoft.azure.storage.blob.BlobListingDetails

Any idea as which specific jar needs to be added to resolve the issue and read azure tables in spark?

Upvotes: 1

Views: 4486

Answers (1)

Peter Pan
Peter Pan

Reputation: 24128

My suggestion is that as below.

  1. Please download the jar files of the newest version of Azure Storage Java Client & Hadoop Azure Support instead of their old version.
  2. Check whether the path of these jars were added into the SPARK_CLASSPATH environment variable in the conf/spark-env file, or you can programmatically add the jar path via code SparkContext.addJar("Path to jar created from maven [hint: mvn package]").

Hope it helps.

Upvotes: 2

Related Questions