CODEWITHSUNDEEP
apache-sparksbtguava
Rajiv Singh
Rajiv Singh

Reputation: 1078

Spark 2.2 is using old com.google.guava library version: 14 but I need to use guava version 18 and above in SBT project

code: var queryJob: Job = bigquery.create(JobInfo.newBuilder(queryConfig).setJobId(jobId).build())

error: Caused by: java.lang.NoSuchMethodError: com.google.common.util.concurrent.MoreExecutors.directExecutor()Ljava/util/concurrent/Executor;

Upvotes: 1

Views: 318

Answers (1)

Rajiv Singh
Rajiv Singh

Reputation: 1078

Error: Caused by: java.lang.NoSuchMethodError: com.google.common.util.concurrent.MoreExecutors.directExecutor()Ljava/util/concurrent/Executor

For SBT solution:

FIX: // Shading com.google.** // We need com.google.guava above 18 version but spark uses version 14 and in that we don't have directExecutor() method // as spark give preference to spark used libraries, our code was failing

assemblyShadeRules in assembly := Seq(
    ShadeRule.rename("com.google.**" -> "shadeio.@1").inAll
)

Upvotes: 1

Related Questions