CODEWITHSUNDEEP
apache-stormapache-storm-flux
user5898161
user5898161

Reputation: 1

How to submit multiple spouts in single topology in apache storm?

I am trying to use parallelism concept in apache-Storm.I want to know how to submit multiple spouts by single topology.

Here is my code for single topology submission 

TwitterTopologyCreator topology = new TwitterTopologyCreator();
topology.createTopology(topologyName, clientName);

Upvotes: 0

Views: 1751

Answers (2)

Salman Dhariwala
Salman Dhariwala

Reputation: 71

There can be various configuration with 2 spouts. I am posting some commonly used topology configuration with 2 spouts.

public class Test {    
    public static void main(String[] args) throws Exception {           
        /*
         * 
         * topology with 2 spout (configuration 1)
         * 
         * spout-1 -->
         *            |-->  bolt-1
         * spout-2 -->
         * 
         */

        TopologyBuilder configuration1=new TopologyBuilder();
        configuration1.setSpout("spout-1", new Spout1());
        configuration1.setSpout("spout-2", new Spout2());
        configuration1.setBolt("bolt-1", new Bolt1()).shuffleGrouping("spout-1").shuffleGrouping("spout-2");

        /*
         * 
         * topology with 2 spout (configuration 2)
         * 
         * spout-1 --> bolt-1
         *      
         * spout-2 --> bolt-2
         * 
         */

        TopologyBuilder configuration2=new TopologyBuilder();
        configuration2.setSpout("spout-1", new Spout1());
        configuration2.setSpout("spout-2", new Spout2());
        configuration2.setBolt("bolt-1", new Bolt1()).shuffleGrouping("spout-1");
        configuration2.setBolt("bolt-2", new Bolt2()).shuffleGrouping("spout-2");

        /*
         * 
         * topology with 2 spout (configuration 3)
         * 
         * spout-1 --> bolt-1 -->
         *                        |--> bolt-3
         * spout-2 --> bolt-2 -->
         * 
         */
        TopologyBuilder configuration3=new TopologyBuilder();
        configuration2.setSpout("spout-1", new Spout1());
        configuration2.setSpout("spout-2", new Spout2());
        configuration2.setBolt("bolt-1", new Bolt1()).shuffleGrouping("spout-1");
        configuration2.setBolt("bolt-2", new Bolt2()).shuffleGrouping("spout-2");
        configuration2.setBolt("bolt-3", new Bolt3()).shuffleGrouping("bolt-1").shuffleGrouping("bolt-2");                          
    }    
}

Upvotes: 5

Nishu Tayal
Nishu Tayal

Reputation: 20880

you can use multiple spouts in a topology in following way.

TwitterTopologyCreator topology = new TwitterTopologyCreator();    
topology.setSpout("kafka-spout1", new KafkaSpout(spoutConf1), 1); 
topology.setSpout("kafka-spout2", new KafkaSpout(spoutConf2), 1);
topology.createTopology(topologyName, clientName);

Upvotes: 0

Related Questions