CODEWITHSUNDEEP
javahadoopapache-storm
user3354890
user3354890

Reputation: 377

How does apache storm create its bolt implementations

After implementing a very simple apache storm topology in java I ran into a NPE which caused some confusion.

This is basically my main method:

TologyBuilder builder = new TopologyBuilder();
builder.setSpout("spout", new CustomSpout(true), 4);
final CustomBolt bolt = new CustomBolt();
builder.setBolt("bolt", bolt, 1).shuffleGrouping("spout");

LocalCluster cluster = new LocalCluster();
Map conf = new HashMap();
conf.put(Config.TOPOLOGY_WORKERS, 10);
cluster.submitTopology("test", conf, createTopology());

The CustomBolt extends AbstractBasic class and implements IRichBolt. They have these fields:

abstract class AbstractBasic {
    A a;
    B b;
} 

CustomBolt extends AbstractBasic implements IRichBolt {
    A anotherA;
    C c;
    CustomBolt() {
        this.anotherA = new A(true);
        this.a = new A(false);
        this.b = new B();
        this.c = new C();
    }
}

On the execute method of CustomBolt there will be some insertions into all classes. However, the fields that are inherited from AbstractBasic are not instantiated, but the other ones, the ones that live directly in CustomBolt, are...

Does anyone know how a bolt is being contructed and how it's possible that some fields are initialized and some are not.

Upvotes: 2

Views: 319

Answers (2)

Carl Veazey
Carl Veazey

Reputation: 18363

When bolts are submitted to the topology, they are serialized and sent to each worker where they are deserialized. Since AbstractBasic does not implement Serializable, its fields will not be serialized when CustomBolt is serialized.

With the example you give, you should be able to fix your problem by declaring that AbstractBasic implements Serializable:

abstract class AbstractBasic implements Serializable ...

Upvotes: 2

java_enthu
java_enthu

Reputation: 2327

I am not sure how instantiatation works with Bolts but I can find a good reason . According to my understanding we run the topology in the distributed environment and to avoid concurrent modifications to the shared variables ( i.e. class level variables inherited from parent classes ) the parent variables could have been set to null in the child classes.

Upvotes: 0

Related Questions