CODEWITHSUNDEEP
javadockermavenjarapache-storm
Peter Prescott
Peter Prescott

Reputation: 908

How to include dependencies with Maven in Storm topology jar for cluster deployment?

I'm trying to deploy a Storm topology to a local standalone cluster. I'll include links to my project's Github repo so you can see my code...

I have got a simple Storm topology running in local mode, which I have tweaked to deploy to a Dockerized cluster (simple configuration based on the recommended instructions).

However, when I try to get it running, I get this error:

Exception in thread "main" java.lang.NoClassDefFoundError: backtype/storm/topology/IRichSpout

It seemed from this question that I just need to include storm-core as a dependency in my Maven project when building the topology.jar.

So I adjusted my pom.xml file (I'm not using Eclipse or any IDE, I'm just running Maven from the command line) to include storm-core as a dependency, and to include the maven-assembly-plugin to package the jar-with-dependencies. But now when I try to run the cluster I get an error saying:

...
java.io.IOException: Found multiple defaults.yaml resources. You're probably bundling the Storm jars with your topology jar
...

So what exactly should I be including as a bundled dependency? And what lines of code do I need to add to/remove from my pom.xml to make it work?

Upvotes: 0

Views: 332

Answers (2)

Peter Prescott
Peter Prescott

Reputation: 908

I have got it working.

The problem was that the Storm dependencies on the Docker cluster were filed under org.apache.storm, whereas the local build was using backtype.storm. Changing the references to org.apache.storm seems to have done the trick.

Upvotes: 1

Alireza
Alireza

Reputation: 25

You should add the "provided" scope in the storm dependency on the pom.xml file.

Upvotes: 0

Related Questions