CODEWITHSUNDEEP
mavenapache-storm
Neil
Neil

Reputation: 1853

Running Word Count in Local cluster in Apache Storm

I am trying to follow the book

http://my.safaribooksonline.com/9781449324025?iid=2013-12-blog-storm-book-9781449324025-SBOBlog

to run Storm cluster and create first topology. These are the steps that i followed

1) Installed Maven

2) Installed maven plugin for Eclipse

3) Create new Maven project in Eclipse with artifact name as stormArtifact

4) It generated a folder structure and a pom.xml file

5) i updated that pom.xml as follows.

6) I added spouts and bolts as specified in this link

https://github.com/storm-book/examples-ch02-getting_started/zipball/master

7) Now i am running it using the command

mvn exec:java -Dexec.mainClass="TopologyMain" -Dexec.args="src/main/resources/words.txt"

But it is throwing an error as follows

[ERROR] Failed to execute goal org.codehaus.mojo:exec-maven-plugin:1.3:java (def
ault-cli) on project stormArtifact: The parameters 'mainClass' for goal org.code
haus.mojo:exec-maven-plugin:1.3:java are missing or invalid -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e swit
ch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please rea
d the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/PluginParamete
rException

My pom.xml

<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0
http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>stormGroup</groupId>
<artifactId>stormArtifact</artifactId>
<version>0.0.1-SNAPSHOT</version>
<build>
<plugins>
<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-compiler-plugin</artifactId>
    <version>2.3.2</version>
    <configuration>
        <source>1.7</source>
        <target>1.7</target>
        <compilerVersion>1.7</compilerVersion>
    </configuration>
</plugin>
</plugins>
</build>
<repositories>
<!-- Repository where we can found the storm dependencies -->
<repository>
<id>clojars.org</id>
<url>http://clojars.org/repo</url>
</repository>
</repositories>
<dependencies>
<!-- Storm Dependency -->
<dependency>
<groupId>storm</groupId>
<artifactId>storm</artifactId>
<version>0.9.0</version>
</dependency>
</dependencies>
</project>

ToplogyMain class

import spouts.WordReader;
import backtype.storm.Config;
import backtype.storm.LocalCluster;
import backtype.storm.topology.TopologyBuilder;
import backtype.storm.tuple.Fields;
import bolts.WordCounter;
import bolts.WordNormalizer;


public class TopologyMain {
public static void main(String[] args) throws InterruptedException {

    //Topology definition
    TopologyBuilder builder = new TopologyBuilder();
    builder.setSpout("word-reader",new WordReader());
    builder.setBolt("word-normalizer", new WordNormalizer())
        .shuffleGrouping("word-reader");
    builder.setBolt("word-counter", new WordCounter(),1)
        .fieldsGrouping("word-normalizer", new Fields("word"));

    //Configuration
    Config conf = new Config();
    conf.put("wordsFile", args[0]);
    conf.setDebug(false);
    //Topology run
    conf.put(Config.TOPOLOGY_MAX_SPOUT_PENDING, 1);
    LocalCluster cluster = new LocalCluster();
    cluster.submitTopology("Getting-Started-Toplogie", conf, builder.createTopology());
    Thread.sleep(1000);
    cluster.shutdown();
}
}

Upvotes: 2

Views: 3575

Answers (1)

Ganesh Pandey
Ganesh Pandey

Reputation: 5276

I had the same problem but i first install the storm binary distribution in my local machine and then ran mvn package command in my main pom.xml.

It worked!

If you are using the latest development version of Storm, e.g. by having cloned the Storm git repository, then you must first perform a local build of Storm itself

Upvotes: 3

Related Questions