forhas
Reputation: 11991
InvalidClassException local class incompatible when running storm topology
My storm application running a single topology is working just fine when running in local mode but when running on a distributed environment I get these exceptions in my worker logs. It's the same exception but it refers to 2 different classes. The first refers to storm.trident.topology.MasterBatchCoordinator:
java.lang.RuntimeException: java.io.InvalidClassException: storm.trident.topology.MasterBatchCoordinator; local class incompatible: stream classdesc serialVersionUID = -5647458900661444882, local class serialVersionUID = 7308128536282303612
at backtype.storm.utils.Utils.deserialize(Utils.java:68) ~[storm-core-0.9.0-wip21.jar:na]
at backtype.storm.utils.Utils.getSetComponentObject(Utils.java:200) ~[storm-core-0.9.0-wip21.jar:na]
at backtype.storm.daemon.task$get_task_object.invoke(task.clj:58) ~[storm-core-0.9.0-wip21.jar:na]
at backtype.storm.daemon.task$mk_task_data$fn__3164.invoke(task.clj:165) ~[storm-core-0.9.0-wip21.jar:na]
at backtype.storm.util$assoc_apply_self.invoke(util.clj:752) ~[storm-core-0.9.0-wip21.jar:na]
at backtype.storm.daemon.task$mk_task_data.invoke(task.clj:158) ~[storm-core-0.9.0-wip21.jar:na]
at backtype.storm.daemon.task$mk_task.invoke(task.clj:169) ~[storm-core-0.9.0-wip21.jar:na]
at backtype.storm.daemon.executor$mk_executor$fn__3352.invoke(executor.clj:303) ~[storm-core-0.9.0-wip21.jar:na]
at clojure.core$map$fn__4087.invoke(core.clj:2432) ~[clojure-1.4.0.jar:na]
at clojure.lang.LazySeq.sval(LazySeq.java:42) ~[clojure-1.4.0.jar:na]
at clojure.lang.LazySeq.seq(LazySeq.java:60) ~[clojure-1.4.0.jar:na]
at clojure.lang.RT.seq(RT.java:473) ~[clojure-1.4.0.jar:na]
at clojure.core$seq.invoke(core.clj:133) ~[clojure-1.4.0.jar:na]
at clojure.core.protocols$seq_reduce.invoke(protocols.clj:30) ~[clojure-1.4.0.jar:na]
at clojure.core.protocols$fn__5875.invoke(protocols.clj:54) ~[clojure-1.4.0.jar:na]
at clojure.core.protocols$fn__5828$G__5823__5841.invoke(protocols.clj:13) ~[clojure-1.4.0.jar:na]
at clojure.core$reduce.invoke(core.clj:6030) ~[clojure-1.4.0.jar:na]
at clojure.core$into.invoke(core.clj:6077) ~[clojure-1.4.0.jar:na]
at backtype.storm.daemon.executor$mk_executor.invoke(executor.clj:303) ~[storm-core-0.9.0-wip21.jar:na]
at backtype.storm.daemon.worker$fn__5862$exec_fn__1224__auto____5863$iter__5868__5872$fn__5873.invoke(worker.clj:360) ~[storm-core-0.9.0-wip21.jar:na]
at clojure.lang.LazySeq.sval(LazySeq.java:42) ~[clojure-1.4.0.jar:na]
at clojure.lang.LazySeq.seq(LazySeq.java:60) ~[clojure-1.4.0.jar:na]
at clojure.lang.Cons.next(Cons.java:39) ~[clojure-1.4.0.jar:na]
at clojure.lang.RT.next(RT.java:587) ~[clojure-1.4.0.jar:na]
at clojure.core$next.invoke(core.clj:64) ~[clojure-1.4.0.jar:na]
at clojure.core$dorun.invoke(core.clj:2726) ~[clojure-1.4.0.jar:na]
at clojure.core$doall.invoke(core.clj:2741) ~[clojure-1.4.0.jar:na]
at backtype.storm.daemon.worker$fn__5862$exec_fn__1224__auto____5863.invoke(worker.clj:360) ~[storm-core-0.9.0-wip21.jar:na]
at clojure.lang.AFn.applyToHelper(AFn.java:185) ~[clojure-1.4.0.jar:na]
at clojure.lang.AFn.applyTo(AFn.java:151) ~[clojure-1.4.0.jar:na]
at clojure.core$apply.invoke(core.clj:601) ~[clojure-1.4.0.jar:na]
at backtype.storm.daemon.worker$fn__5862$mk_worker__5918.doInvoke(worker.clj:329) ~[storm-core-0.9.0-wip21.jar:na]
at clojure.lang.RestFn.invoke(RestFn.java:512) ~[clojure-1.4.0.jar:na]
at backtype.storm.daemon.worker$_main.invoke(worker.clj:439) ~[storm-core-0.9.0-wip21.jar:na]
at clojure.lang.AFn.applyToHelper(AFn.java:172) ~[clojure-1.4.0.jar:na]
at clojure.lang.AFn.applyTo(AFn.java:151) ~[clojure-1.4.0.jar:na]
at backtype.storm.daemon.worker.main(Unknown Source) ~[storm-core-0.9.0-wip21.jar:na]
Caused by: java.io.InvalidClassException: storm.trident.topology.MasterBatchCoordinator; local class incompatible: stream classdesc serialVersionUID = -5647458900661444882, local class serialVersionUID = 7308128536282303612
at java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:617) ~[na:1.7.0_25]
at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1620) ~[na:1.7.0_25]
at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1515) ~[na:1.7.0_25]
at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1769) ~[na:1.7.0_25]
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1348) ~[na:1.7.0_25]
at java.io.ObjectInputStream.readObject(ObjectInputStream.java:370) ~[na:1.7.0_25]
at backtype.storm.utils.Utils.deserialize(Utils.java:64) ~[storm-core-0.9.0-wip21.jar:na]
... 36 common frames omitted
The second refers to backtype.storm.topology.BasicBoltExecutor:
java.lang.RuntimeException: java.io.InvalidClassException: backtype.storm.topology.BasicBoltExecutor; local class incompatible: stream classdesc serialVersionUID = -2269346678248157705, local class serialVersionUID = 1040841720700290343
at backtype.storm.utils.Utils.deserialize(Utils.java:68) ~[storm-core-0.9.0-wip21.jar:na]
at backtype.storm.utils.Utils.getSetComponentObject(Utils.java:200) ~[storm-core-0.9.0-wip21.jar:na]
at backtype.storm.daemon.task$get_task_object.invoke(task.clj:58) ~[storm-core-0.9.0-wip21.jar:na]
at backtype.storm.daemon.task$mk_task_data$fn__3164.invoke(task.clj:165) ~[storm-core-0.9.0-wip21.jar:na]
at backtype.storm.util$assoc_apply_self.invoke(util.clj:752) ~[storm-core-0.9.0-wip21.jar:na]
at backtype.storm.daemon.task$mk_task_data.invoke(task.clj:158) ~[storm-core-0.9.0-wip21.jar:na]
at backtype.storm.daemon.task$mk_task.invoke(task.clj:169) ~[storm-core-0.9.0-wip21.jar:na]
at backtype.storm.daemon.executor$mk_executor$fn__3352.invoke(executor.clj:303) ~[storm-core-0.9.0-wip21.jar:na]
at clojure.core$map$fn__4087.invoke(core.clj:2432) ~[clojure-1.4.0.jar:na]
at clojure.lang.LazySeq.sval(LazySeq.java:42) ~[clojure-1.4.0.jar:na]
at clojure.lang.LazySeq.seq(LazySeq.java:60) ~[clojure-1.4.0.jar:na]
at clojure.lang.RT.seq(RT.java:473) ~[clojure-1.4.0.jar:na]
at clojure.core$seq.invoke(core.clj:133) ~[clojure-1.4.0.jar:na]
at clojure.core.protocols$seq_reduce.invoke(protocols.clj:30) ~[clojure-1.4.0.jar:na]
at clojure.core.protocols$fn__5875.invoke(protocols.clj:54) ~[clojure-1.4.0.jar:na]
at clojure.core.protocols$fn__5828$G__5823__5841.invoke(protocols.clj:13) ~[clojure-1.4.0.jar:na]
at clojure.core$reduce.invoke(core.clj:6030) ~[clojure-1.4.0.jar:na]
at clojure.core$into.invoke(core.clj:6077) ~[clojure-1.4.0.jar:na]
at backtype.storm.daemon.executor$mk_executor.invoke(executor.clj:303) ~[storm-core-0.9.0-wip21.jar:na]
at backtype.storm.daemon.worker$fn__5862$exec_fn__1224__auto____5863$iter__5868__5872$fn__5873.invoke(worker.clj:360) ~[storm-core-0.9.0-wip21.jar:na]
at clojure.lang.LazySeq.sval(LazySeq.java:42) ~[clojure-1.4.0.jar:na]
at clojure.lang.LazySeq.seq(LazySeq.java:60) ~[clojure-1.4.0.jar:na]
at clojure.lang.RT.seq(RT.java:473) ~[clojure-1.4.0.jar:na]
at clojure.core$seq.invoke(core.clj:133) ~[clojure-1.4.0.jar:na]
at clojure.core$dorun.invoke(core.clj:2725) ~[clojure-1.4.0.jar:na]
at clojure.core$doall.invoke(core.clj:2741) ~[clojure-1.4.0.jar:na]
at backtype.storm.daemon.worker$fn__5862$exec_fn__1224__auto____5863.invoke(worker.clj:360) ~[storm-core-0.9.0-wip21.jar:na]
at clojure.lang.AFn.applyToHelper(AFn.java:185) ~[clojure-1.4.0.jar:na]
at clojure.lang.AFn.applyTo(AFn.java:151) ~[clojure-1.4.0.jar:na]
at clojure.core$apply.invoke(core.clj:601) ~[clojure-1.4.0.jar:na]
at backtype.storm.daemon.worker$fn__5862$mk_worker__5918.doInvoke(worker.clj:329) ~[storm-core-0.9.0-wip21.jar:na]
at clojure.lang.RestFn.invoke(RestFn.java:512) ~[clojure-1.4.0.jar:na]
at backtype.storm.daemon.worker$_main.invoke(worker.clj:439) ~[storm-core-0.9.0-wip21.jar:na]
at clojure.lang.AFn.applyToHelper(AFn.java:172) ~[clojure-1.4.0.jar:na]
at clojure.lang.AFn.applyTo(AFn.java:151) ~[clojure-1.4.0.jar:na]
at backtype.storm.daemon.worker.main(Unknown Source) ~[storm-core-0.9.0-wip21.jar:na]
Caused by: java.io.InvalidClassException: backtype.storm.topology.BasicBoltExecutor; local class incompatible: stream classdesc serialVersionUID = -2269346678248157705, local class serialVersionUID = 1040841720700290343
at java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:617) ~[na:1.7.0_25]
at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1620) ~[na:1.7.0_25]
at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1515) ~[na:1.7.0_25]
at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1769) ~[na:1.7.0_25]
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1348) ~[na:1.7.0_25]
at java.io.ObjectInputStream.readObject(ObjectInputStream.java:370) ~[na:1.7.0_25]
at backtype.storm.utils.Utils.deserialize(Utils.java:64) ~[storm-core-0.9.0-wip21.jar:na]
... 35 common frames omitted
2013-09-19 07:42:19 b.s.util [INFO] Halting process: ("Error on initialization")
It keeps on showing up all over the log and my topology obviously doesn't function. I know that this exception is thrown when the serial version of the class is found to be different from that of the class descriptor, but don't really know what causes it and how to fix it.
Here are my pom dependencies:
<properties>
<org.springframework-version>3.2.3.RELEASE</org.springframework-version>
<storm-version>0.9.0-wip17</storm-version>
<storm-kafka-version>0.9.0-wip16a-scala292</storm-kafka-version>
<kafka-version>0.7.1-SNAPSHOT</kafka-version>
<cassandra.unit.version>1.0.3.1</cassandra.unit.version>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
</properties>
<repositories>
<repository>
<id>github-releases</id>
<url>http://oss.sonatype.org/content/repositories/github-releases/</url>
</repository>
<repository>
<id>clojars.org</id>
<url>http://clojars.org/repo</url>
</repository>
<repository>
<id>twitter4j</id>
<url>http://twitter4j.org/maven2</url>
</repository>
</repositories>
<dependencies>
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-core</artifactId>
<version>${org.springframework-version}</version>
<exclusions>
<!-- Exclude Commons Logging in favor of SLF4j -->
<exclusion>
<groupId>commons-logging</groupId>
<artifactId>commons-logging</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>storm</groupId>
<artifactId>storm</artifactId>
<version>${storm-version}</version>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>storm</groupId>
<artifactId>storm-kafka</artifactId>
<version>${storm-kafka-version}</version>
</dependency>
<dependency>
<groupId>org.clojars.smallrivers</groupId>
<artifactId>kafka</artifactId>
<version>${kafka-version}</version>
</dependency>
<dependency>
<groupId>org.twitter4j</groupId>
<artifactId>twitter4j-core</artifactId>
<version>2.2.6-SNAPSHOT</version>
</dependency>
<dependency>
<groupId>org.twitter4j</groupId>
<artifactId>twitter4j-stream</artifactId>
<version>2.2.6-SNAPSHOT</version>
</dependency>
<dependency>
<groupId>com.googlecode.json-simple</groupId>
<artifactId>json-simple</artifactId>
<version>1.1</version>
</dependency>
<dependency>
<groupId>com.google.guava</groupId>
<artifactId>guava</artifactId>
<version>13.0.1</version>
</dependency>
<dependency>
<groupId>commons-collections</groupId>
<artifactId>commons-collections</artifactId>
<version>3.2.1</version>
</dependency>
<dependency>
<groupId>me.prettyprint</groupId>
<artifactId>hector-core</artifactId>
<version>1.0-4</version>
</dependency>
<!-- Tests -->
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.11</version>
</dependency>
<dependency>
<groupId>org.yaml</groupId>
<artifactId>snakeyaml</artifactId>
<version>1.12</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.cassandraunit</groupId>
<artifactId>cassandra-unit</artifactId>
<version>${cassandra.unit.version}</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.mockito</groupId>
<artifactId>mockito-all</artifactId>
<version>1.9.5</version>
<scope>test</scope>
</dependency>
</dependencies>
Any ideas? I'm lost here :\
Upvotes: 2
Views: 1706
Answers (1)
schiavuzzi
Reputation: 1072
Is it possible you're mixing different Storm versions? The error log refers to Storm version 0.9.0-wip21, but Storm version 0.9.0-wip17 is listed among the project dependencies in your pom.xml file. All components on the cluster should use the same Storm version.
Upvotes: 1
Related Questions
- Storm ClassNotFoundException
- Submitting a Storm topology fails /getting started)
- Topology conf is not json-serializable - Storm
- Getting NotSerializableException error in storm topology
- custom deserializer does not get called apache storm
- Exception in thread "main" java.lang.IllegalAccessError: in storm topology
- Custom serializer in Apache Storm 1.1.0
- Serialize error when storm supervisor starts executor
- java.lang.ClassNotFoundException: TopologyMain
- Exception thrown when registering custom serializers in Storm's storm.yaml