CODEWITHSUNDEEP
javahadoopmapreduce
tom
tom

Reputation: 729

NoSuchMethodError when running on Hadoop but not when run locally

While running program on Hadoop 2.0.0-cdh4.3.1 MapReduce gives me below error :

 java.lang.NoSuchMethodError:com.google.common.util.concurrent.Futures.withFallback

But when I test by executing JAR :

java -cp myclass

It runs flawlessly. I am out of idea here as if so called Futures.withFallback is present in JAR thats why its got executed in local. Its using Guava for connecting Cassandra, full stack trace is below:

attempt_201507081740_21115_m_000050_0: [FATAL] Child - Error running child : java.lang.NoSuchMethodError: com.google.common.util.concurrent.Futures.withFallback(Lcom/google/common/util/concurrent/ListenableFuture;Lcom/google/common/util/concurrent/FutureFallback;Ljava/util/concurrent/Executor;)Lcom/google/common/util/concurrent/ListenableFuture;
attempt_201507081740_21115_m_000050_0:  at com.datastax.driver.core.Connection.initAsync(Connection.java:176)
attempt_201507081740_21115_m_000050_0:  at com.datastax.driver.core.Connection$Factory.open(Connection.java:721)
attempt_201507081740_21115_m_000050_0:  at com.datastax.driver.core.ControlConnection.tryConnect(ControlConnection.java:244)
attempt_201507081740_21115_m_000050_0:  at com.datastax.driver.core.ControlConnection.reconnectInternal(ControlConnection.java:190)
attempt_201507081740_21115_m_000050_0:  at com.datastax.driver.core.ControlConnection.connect(ControlConnection.java:78)
attempt_201507081740_21115_m_000050_0:  at com.datastax.driver.core.Cluster$Manager.init(Cluster.java:1272)
attempt_201507081740_21115_m_000050_0:  at com.datastax.driver.core.Cluster.init(Cluster.java:158)
attempt_201507081740_21115_m_000050_0:  at com.datastax.driver.core.Cluster.connect(Cluster.java:248)
attempt_201507081740_21115_m_000050_0:  at com.datastax.driver.core.Cluster.connect(Cluster.java:281)
attempt_201507081740_21115_m_000050_0:  at com.cassandra.CassandraHandler.getConnection(CassandraHandler.java:40)
attempt_201507081740_21115_m_000050_0:  at com.json.flatten.DynamicJsonFlattener.<init>(DynamicJsonFlattener.java:35)
attempt_201507081740_21115_m_000050_0:  at com.mapreduce.Map.map(Map.java:18)
attempt_201507081740_21115_m_000050_0:  at com.mapreduce.Map.map(Map.java:13)
attempt_201507081740_21115_m_000050_0:  at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:140)
attempt_201507081740_21115_m_000050_0:  at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:672)
attempt_201507081740_21115_m_000050_0:  at org.apache.hadoop.mapred.MapTask.run(MapTask.java:330)
attempt_201507081740_21115_m_000050_0:  at org.apache.hadoop.mapred.Child$4.run(Child.java:268)
attempt_201507081740_21115_m_000050_0:  at java.security.AccessController.doPrivileged(Native Method)
attempt_201507081740_21115_m_000050_0:  at javax.security.auth.Subject.doAs(Subject.java:396)
 attempt_201507081740_21115_m_000050_0:  at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1408)
attempt_201507081740_21115_m_000050_0:  at org.apache.hadoop.mapred.Child.main(Child.java:262)

Is something wrong with Hadoop version or any other version clash. Any ideas please!

EDIT: I have verified that there is no "withFallback" method present in Guava v18 JAR. Now I am clueless, please help me with any ideas !

Upvotes: 4

Views: 1479

Answers (2)

Kurt Freytag
Kurt Freytag

Reputation: 11

I believe I've found the problem (or at least a resolution for me).

If you happen to be using Google's google-api-client library:

com.google.api-client » google-api-client

it has a dependency on guava-jdk5 13.0. This version of guava-jdk5 does not have Futures.withFallback (that was introduced in 14.0) and could likely be conflicting with your dependency on guava.

The solution, if this is your case, is to add a dependency to com.google.guava:guava-jdk5:17.0. It's a JDK5 backport of guava, but it does work to resolve this issue.

Why Google has this dependency on a very old guava version (and a backport nonetheless) is beyond me.

Upvotes: 0

mujeeb rahman
mujeeb rahman

Reputation: 109

Edited: didn't saw your stacktrace. your stacktrace show that there is a version miss match. you can find appropriate compatible version on there documentations.

make sure that required jar files are available in your classpath.

when you are running using -cp option, your jvm searches for required jar file in all directory locations separated by ; given in your classpath variable.

make sure you have also added ;.; in your classes path, which causes the jvm to load/search class/jar files available in your current working directory as well.

Upvotes: 1

Related Questions