CODEWITHSUNDEEP
javaserializationapache-stormkryo
dmitrievanthony
dmitrievanthony

Reputation: 1561

Custom serialization in Apache Storm

I try to add custom serializer for my Objects which is used in Apache Storm Spouts/Bolts. Right now my code looks like that:

conf.registerSerialization(MyService.class, MyKryoSerializer.class);

public class MyKryoSerializer extends Serializer<MyService> {

    public MyKryoSerializer() {
        System.out.println("New MyKryoSerializaer!");
    }

    @Override
    public void write(Kryo kryo, Output output, MyService service) {
        System.out.println(72);
    }

    @Override
    public MyService read(Kryo kryo, Input input, Class<MyService> aClass) {
        System.out.println(73);
        return null;
    }

    public MyService copy(Kryo kryo, MyService myService) {
        System.out.println("MyService!");
        return myService;
    }
}

public class MyRandomSpout extends BaseRichSpout {

    private MyService service;

    private SpoutOutputCollector collector;

...

So, when I try to start Storm topology on my local cluster I will see "New MyKryoSerializaer!" in stdout, so constructor is called, but write/read methods are not called. Could anybody tell me, what do I wrong? Does Storm support Skyo serializer for spots/bolts serialization?

Upvotes: 1

Views: 1070

Answers (2)

Samuel
Samuel

Reputation: 17171

It doesn't look like Storm supports Kryo for serialization of bolts or spouts. See this line where it explicitly tries to serialize with Java serialization.

Upvotes: 1

Chris Gerken
Chris Gerken

Reputation: 16392

I had to add the following config to my topologies in order to get Storm to use my own serialization logic:

config.put(Config.TOPOLOGY_FALL_BACK_ON_JAVA_SERIALIZATION, true);

where config is the configuration I eventually pass in to the submit method.

Upvotes: 1

Related Questions