CODEWITHSUNDEEP
javaapache-zookeeperapache-storm
ndp
ndp

Reputation: 893

ClassNotFoundException for Zookeeper while building Storm

I'm new to java and Storm so please forgive any obvious mistakes. I'm trying to run storm with a flume connector but It crashes with the following error:

java.lang.reflect.InvocationTargetException
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at org.codehaus.mojo.exec.ExecJavaMojo$1.run(ExecJavaMojo.java:297)
    at java.lang.Thread.run(Thread.java:744)
Caused by: java.lang.ExceptionInInitializerError
    at java.lang.Class.forName0(Native Method)
    at java.lang.Class.forName(Class.java:270)
    at clojure.lang.RT.loadClassForName(RT.java:2056)
    at clojure.lang.RT.load(RT.java:419)
    at clojure.lang.RT.load(RT.java:400)
    at clojure.core$load$fn__4890.invoke(core.clj:5415)
    at clojure.core$load.doInvoke(core.clj:5414)
    at clojure.lang.RestFn.invoke(RestFn.java:408)
    at clojure.core$load_one.invoke(core.clj:5227)
    at clojure.core$load_lib.doInvoke(core.clj:5264)
    at clojure.lang.RestFn.applyTo(RestFn.java:142)
    at clojure.core$apply.invoke(core.clj:603)
    at clojure.core$load_libs.doInvoke(core.clj:5302)
    at clojure.lang.RestFn.applyTo(RestFn.java:137)
    at clojure.core$apply.invoke(core.clj:603)
    at clojure.core$require.doInvoke(core.clj:5381)
    at clojure.lang.RestFn.invoke(RestFn.java:408)
    at backtype.storm.cluster$loading__4784__auto__.invoke(cluster.clj:1)
    at backtype.storm.cluster__init.load(Unknown Source)
    at backtype.storm.cluster__init.<clinit>(Unknown Source)
    at java.lang.Class.forName0(Native Method)
    at java.lang.Class.forName(Class.java:270)
    at clojure.lang.RT.loadClassForName(RT.java:2056)
    at clojure.lang.RT.load(RT.java:419)
    at clojure.lang.RT.load(RT.java:400)
    at clojure.core$load$fn__4890.invoke(core.clj:5415)
    at clojure.core$load.doInvoke(core.clj:5414)
    at clojure.lang.RestFn.invoke(RestFn.java:408)
    at clojure.core$load_one.invoke(core.clj:5227)
    at clojure.core$load_lib.doInvoke(core.clj:5264)
    at clojure.lang.RestFn.applyTo(RestFn.java:142)
    at clojure.core$apply.invoke(core.clj:603)
    at clojure.core$load_libs.doInvoke(core.clj:5302)
    at clojure.lang.RestFn.applyTo(RestFn.java:137)
    at clojure.core$apply.invoke(core.clj:603)
    at clojure.core$require.doInvoke(core.clj:5381)
    at clojure.lang.RestFn.invoke(RestFn.java:408)
    at backtype.storm.daemon.nimbus__init.load(Unknown Source)
    at backtype.storm.daemon.nimbus__init.<clinit>(Unknown Source)
    at java.lang.Class.forName0(Native Method)
    at java.lang.Class.forName(Class.java:270)
    at clojure.lang.RT.loadClassForName(RT.java:2056)
    at clojure.lang.RT.load(RT.java:419)
    at clojure.lang.RT.load(RT.java:400)
    at clojure.core$load$fn__4890.invoke(core.clj:5415)
    at clojure.core$load.doInvoke(core.clj:5414)
    at clojure.lang.RestFn.invoke(RestFn.java:408)
    at clojure.core$load_one.invoke(core.clj:5227)
    at clojure.core$load_lib.doInvoke(core.clj:5264)
    at clojure.lang.RestFn.applyTo(RestFn.java:142)
    at clojure.core$apply.invoke(core.clj:603)
    at clojure.core$load_libs.doInvoke(core.clj:5302)
    at clojure.lang.RestFn.applyTo(RestFn.java:137)
    at clojure.core$apply.invoke(core.clj:603)
    at clojure.core$require.doInvoke(core.clj:5381)
    at clojure.lang.RestFn.invoke(RestFn.java:408)
    at backtype.storm.testing$loading__4784__auto__.invoke(testing.clj:1)
    at backtype.storm.testing__init.load(Unknown Source)
    at backtype.storm.testing__init.<clinit>(Unknown Source)
    at java.lang.Class.forName0(Native Method)
    at java.lang.Class.forName(Class.java:270)
    at clojure.lang.RT.loadClassForName(RT.java:2056)
    at clojure.lang.RT.load(RT.java:419)
    at clojure.lang.RT.load(RT.java:400)
    at clojure.core$load$fn__4890.invoke(core.clj:5415)
    at clojure.core$load.doInvoke(core.clj:5414)
    at clojure.lang.RestFn.invoke(RestFn.java:408)
    at clojure.core$load_one.invoke(core.clj:5227)
    at clojure.core$load_lib.doInvoke(core.clj:5264)
    at clojure.lang.RestFn.applyTo(RestFn.java:142)
    at clojure.core$apply.invoke(core.clj:603)
    at clojure.core$load_libs.doInvoke(core.clj:5302)
    at clojure.lang.RestFn.applyTo(RestFn.java:137)
    at clojure.core$apply.invoke(core.clj:605)
    at clojure.core$use.doInvoke(core.clj:5392)
    at clojure.lang.RestFn.invoke(RestFn.java:408)
    at backtype.storm.LocalCluster$loading__4784__auto__.invoke(LocalCluster.clj:1)
    at backtype.storm.LocalCluster__init.load(Unknown Source)
    at backtype.storm.LocalCluster__init.<clinit>(Unknown Source)
    at java.lang.Class.forName0(Native Method)
    at java.lang.Class.forName(Class.java:270)
    at clojure.lang.RT.loadClassForName(RT.java:2056)
    at clojure.lang.RT.load(RT.java:419)
    at clojure.lang.RT.load(RT.java:400)
    at clojure.core$load$fn__4890.invoke(core.clj:5415)
    at clojure.core$load.doInvoke(core.clj:5414)
    at clojure.lang.RestFn.invoke(RestFn.java:408)
    at clojure.lang.Var.invoke(Var.java:415)
    at backtype.storm.LocalCluster.<clinit>(Unknown Source)
    at com.storm.WordCountTopology.main(WordCountTopology.java:97)
    ... 6 more
Caused by: java.lang.ClassNotFoundException: org.apache.zookeeper.server.NIOServerCnxn$Factory
    at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
    at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
    at java.lang.Class.forName0(Native Method)
    at java.lang.Class.forName(Class.java:190)
    at backtype.storm.zookeeper$loading__4784__auto__.invoke(zookeeper.clj:1)
    at backtype.storm.zookeeper__init.load(Unknown Source)
    at backtype.storm.zookeeper__init.<clinit>(Unknown Source)
    ... 96 more

My pom file

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <groupId>com.storm</groupId>
    <artifactId>my-storm</artifactId>
    <version>1.0-SNAPSHOT</version>
    <packaging>jar</packaging>
    <name>my-storm</name>
    <url>http://maven.apache.org</url>
    <properties>
        <flume.version>1.3.0-cdh4.2.1</flume.version>
        <hadoop.version>2.0.0-cdh4.2.1</hadoop.version>
        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
    </properties>
    <repositories>
        <repository>
            <id>clojars.org</id>
            <url>http://clojars.org/repo</url>
        </repository>
        <repository>
            <id>cdh-maven-repo</id>
            <name>CDH Maven Repository</name>
            <url>https://repository.cloudera.com/content/repositories/releases/</url>
        </repository>
    </repositories> 
    <dependencies>
        <dependency>
            <groupId>storm</groupId>
            <artifactId>storm</artifactId>
            <version>0.8.2</version>
            <scope>compile</scope>
        </dependency>
        <dependency>
            <groupId>junit</groupId>
            <artifactId>junit</artifactId>
            <version>3.8.1</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>org.apache.flume</groupId>
            <artifactId>flume-ng-embedded-agent</artifactId>
            <version>${flume.version}</version>
        </dependency>
        <dependency>
            <groupId>org.apache.flume</groupId>
            <artifactId>flume-ng-core</artifactId>
            <version>${flume.version}</version>
            <exclusions>
                <exclusion>
                    <groupId>org.slf4j</groupId>
                        <artifactId>slf4j-api</artifactId>
                </exclusion>
            </exclusions>
        </dependency>
        <dependency>
            <groupId>org.apache.flume</groupId>
            <artifactId>flume-ng-configuration</artifactId>
            <version>${flume.version}</version>
        </dependency>
            <dependency>
                <groupId>org.apache.flume</groupId>
                <artifactId>flume-ng-node</artifactId>
                <version>${flume.version}</version>
            </dependency>
            <dependency>
                <groupId>org.apache.hadoop</groupId>
                <artifactId>hadoop-common</artifactId>
                <version>${hadoop.version}</version>
            </dependency>
            <dependency>
                <groupId>org.apache.zookeeper</groupId>
                <artifactId>zookeeper</artifactId>
                <version>3.4.5</version>
                <scope>test</scope>
            </dependency>
            <dependency>
                <groupId>org.slf4j</groupId>
                <artifactId>slf4j-log4j12</artifactId>
                <version>1.5.8</version>
            </dependency>
        </dependencies>
       <build>
        <plugins>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-compiler-plugin</artifactId>
                <version>2.3.2</version>
                <configuration>
                    <source>1.7</source>
                    <target>1.7</target>
                </configuration>
            </plugin>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-release-plugin</artifactId>
                <version>2.0-beta-9</version>
            </plugin>
        </plugins>
    </build>
</project>



 $uname -a :Linux 2.6.32-358.23.2.el6.x86_64 #1 SMP Thu Oct 17 10:27:22 CEST 2013 x86_64 x86_64 x86_64 GNU/Linux
    $locate zookeeper
       .m2/repository/org/apache/zookeeper/zookeeper/3.4.5-cdh4.2.1/zookeeper-3.4.5-cdh4.2.1.pom

Can you please explain what am I missing?

Upvotes: 2

Views: 10067

Answers (5)

OjasviHarsola
OjasviHarsola

Reputation: 41

The below POM works for me - 

<dependencies>
    <dependency>
        <groupId>org.apache.zookeeper</groupId>
        <artifactId>zookeeper</artifactId>
        <version>3.4.5</version>
        <exclusions>
            <exclusion>
                <groupId>com.sun.jmx</groupId>
                <artifactId>jmxri</artifactId>
            </exclusion>
            <exclusion>
                <groupId>com.sun.jdmk</groupId>
                <artifactId>jmxtools</artifactId>
            </exclusion>
            <exclusion>
                <groupId>javax.jms</groupId>
                <artifactId>jms</artifactId>
            </exclusion>
        </exclusions>
    </dependency>
    <dependency>
        <groupId>org.apache.storm</groupId>
        <artifactId>storm-kafka</artifactId>
        <version>0.9.3</version>
        <exclusions>
            <exclusion>
                <groupId>org.slf4j</groupId>
                <artifactId>slf4j-log4j12</artifactId>
            </exclusion>
            <exclusion>
                <groupId>log4j</groupId>
                <artifactId>log4j</artifactId>
            </exclusion>
        </exclusions>
    </dependency>
    <dependency>
        <groupId>org.apache.kafka</groupId>
        <artifactId>kafka_2.10</artifactId>
        <version>0.8.2.0</version>
        <exclusions>
            <exclusion>
                <groupId>org.slf4j</groupId>
                <artifactId>slf4j-log4j12</artifactId>
            </exclusion>
            <exclusion>
                <groupId>log4j</groupId>
                <artifactId>log4j</artifactId>
            </exclusion>
        </exclusions>
    </dependency>
    <dependency>
        <groupId>org.apache.storm</groupId>
        <artifactId>storm-core</artifactId>
        <version>0.9.3</version>
        <exclusions>
            <exclusion>
                <groupId>org.slf4j</groupId>
                <artifactId>slf4j-log4j12</artifactId>
            </exclusion>
            <exclusion>
                <groupId>log4j</groupId>
                <artifactId>log4j</artifactId>
            </exclusion>
        </exclusions>
    </dependency>
    <dependency>
        <groupId>com.espertech</groupId>
        <artifactId>esper</artifactId>
        <version>4.11.0</version>
        <exclusions>
            <exclusion>
                <groupId>org.slf4j</groupId>
                <artifactId>slf4j-log4j12</artifactId>
            </exclusion>
            <exclusion>
                <groupId>log4j</groupId>
                <artifactId>log4j</artifactId>
            </exclusion>
        </exclusions>
    </dependency>
    <dependency>
        <groupId>net.sf.opencsv</groupId>
        <artifactId>opencsv</artifactId>
        <version>2.3</version>
    </dependency>
</dependencies>

Upvotes: 1

Harish Shan
Harish Shan

Reputation: 122

I have also faced the same issue, the problem i noticed in my case is: I am using storm-9.0.1 with Zookeeper 3.4.5.

By change my zookeeper to 3.3.6 or by changing the storm to 9.0.2,My problem have solved.

So Storm-9.0.1 is compatibility with Zookeeper 3.3.6 and Storm 9.0.2 is compatibility with Zookeeper 3.4.5

Upvotes: 0

Filippo Vitale
Filippo Vitale

Reputation: 8113

It seems that the key is here:

Caused by: java.lang.ClassNotFoundException: org.apache.zookeeper.server.NIOServerCnxn$Factory

This seems caused by using the version 3.4.5 of zookeeper. Have you tried a downgrade to, maybe version 3.3.6?

The branch-3.3 contains that Inner Class NIOServerCnxn$Factory but the branch-3.4 has the NIOServerCnxnFactory instead

Upvotes: 6

ndp
ndp

Reputation: 893

After some days of banging my head I decided to change the aproach, aparently Eclipse could run the project as it was without any issue.

Creating an uberjar from eclipse and puting all the referenced jars in the project's lib folder and then running the uberjar in the server using java -jar solved the issue.

Not a good solution and I never understood why maven couldn't run the project but it works.

Upvotes: -1

user987339
user987339

Reputation: 10727

It seems that final jar is missing zookeeper. Try adding this:

   <dependency>
        <groupId>org.apache.flume</groupId>
        <artifactId>flume-ng-core</artifactId>
        <version>${flume.version}</version>
        <exclusions>
            <exclusion>
                <groupId>org.slf4j</groupId>
                <artifactId>slf4j-api</artifactId>
            </exclusion>
        </exclusions>
    </dependency>


<dependency>
  <groupId>org.apache.zookeeper</groupId>
  <artifactId>zookeeper</artifactId>
  <version>3.4.5</version>
  <scope>test</scope>
</dependency>
<dependency>
  <groupId>org.slf4j</groupId>
  <artifactId>slf4j-log4j12</artifactId>
  <version>1.5.8</version>
</dependency>

to your pom.xml.

If it doesn't help issue :

mvn dependency:tree

and send me output.

Upvotes: 1

Related Questions