How to serialize Multimaps with Kryo?

Question

Problem

I am trying to serialize a LinkedHashMultimap using Kryo Serialization library, but am getting a NullPointerException upon deserialization. The minimal working example is below:

import com.esotericsoftware.kryo.Kryo;
import com.esotericsoftware.kryo.io.Input;
import com.esotericsoftware.kryo.io.Output;
import com.google.common.collect.LinkedHashMultimap;
import java.io.*;

public class SerializationTest {

    private static final String ioFileName = "someIO.bin";

    public static void main(String[] args0) {

        // Create LinkedHashMultimap to serialize
        LinkedHashMultimap<String, Object> outObj = LinkedHashMultimap.create();
        outObj.put("x", 1);
        outObj.put("y", "abc");

        // Try to serialize and deserialize
        Kryo kryo = new Kryo();
        writeObj(kryo, outObj);
        LinkedHashMultimap<String, Object> inObj = (LinkedHashMultimap<String, Object>) readObj(kryo);

        System.out.println(inObj);
    }

    public static Object readObj(Kryo kryo) {
        Object obj = null;
        try {
            Input input = new Input(new FileInputStream(ioFileName));
            obj = kryo.readClassAndObject(input);
        } catch (FileNotFoundException e) {
            e.printStackTrace();
        }
        return obj;
    }

    public static void writeObj(Kryo kryo, Object obj) {
        try {
            Output output = new Output(new FileOutputStream(ioFileName));
            kryo.writeClassAndObject(output, obj);
            output.flush();
        } catch (FileNotFoundException e) {
            e.printStackTrace();
        }
    }
}

You can see the problem as soon as kryo.readClassAndObject() (line 30) is called (screenshot from debugging in IntelliJ 14):

Either the LinkedHashMultimap is being corrupted upon serialization or is simply being deserialized incorrectly, resulting in a NullPointerException.

The full stacktrace as produced when System.out.println(inObj) is called:

Exception in thread "main" java.lang.NullPointerException
Disconnected from the target VM, address: '127.0.0.1:60310', transport: 'socket'
    at com.google.common.collect.AbstractMapBasedMultimap$AsMap.toString(AbstractMapBasedMultimap.java:1293)
    at com.google.common.collect.AbstractMultimap.toString(AbstractMultimap.java:239)
    at com.google.common.collect.LinkedHashMultimap.toString(LinkedHashMultimap.java:81)
    at java.lang.String.valueOf(String.java:2854)
    at java.io.PrintStream.println(PrintStream.java:821)
    at SerializationTest.main(SerializationTest.java:23)

Does anyone know how to resolve this?

Motivation:

To aid in remote debugging (Java), it's useful to be able to request remote servers to send over arbitrary objects to my local machine for inspection. However, this means that the remote server must be able to serialize an arbitrary java object that is not known in advance at runtime.

So I asked around and stumbled on the Kryo serialization library. From Kryo's documentation, a major feature is that it's very robust at serializing arbitrary java objects. Objects

• don't have to implement Serializable,
• don't need no-arg constructors to be deserializable and
• I don't even need to know anything about the structure of the object prior to serialization.

Answer 1

I encountered a similar issue today, and because it took me some time to identify the root cause behind the observed symptoms, I believe it's valuable to post an answer, even though this question has been around for a while. It's worth noting that this problem was also discussed on GitHub at https://github.com/EsotericSoftware/kryo/issues/573.

It's important to understand that the multimapHeaderEntry attribute within the LinkedHashMultimap class is marked as transient. According to https://github.com/EsotericSoftware/kryo#fieldserializer-settings, Kryo doesn't serialize transient fields by default. To still serialize transient fields, you can configure the default field serializer via the FieldSerializerConfig. See the MRE below:

public class Debug {


    static class Dummy {
        int i;
        transient int ti;
    }

    @Test
    public void kryoTest() {
        // kryo setup
        Kryo kryo = new Kryo();
        FieldSerializer.FieldSerializerConfig config = new FieldSerializer.FieldSerializerConfig();
        config.setSerializeTransient(true); // test will fail if this line is commented out
        kryo.addDefaultSerializer(Dummy.class, new FieldSerializer<>(kryo, Dummy.class, config));
        kryo.register(Dummy.class); // registration after configuration

        // create dummy object
        Dummy dummy = new Dummy();
        dummy.i = 1;
        dummy.ti = 2;

        // serialize
        Output output = new Output(128, -1);
        kryo.writeObject(output, dummy);
        byte[] data = output.getBuffer();

        // deserialize
        Dummy clone = kryo.readObject(new Input(data), Dummy.class);

        assertEquals(dummy.i, clone.i);
        assertEquals(dummy.ti, clone.ti); // fails if you do not use the custom serializer above
    }
}

Answer 2

Try the next before doing serialization/deserialization (in my case it works for kryo.WriteObject and kryo.readObject)

    JavaSerializer serializer = new JavaSerializer();
    kryo.register(LinkedHashMultimap.class, serializer);

The working example:

public class SerializationTest {

private static final String ioFileName = "someIO.bin";

public static void main(String[] args0) {

    // Create LinkedHashMultimap to serialize
    LinkedHashMultimap<String, Object> outObj = LinkedHashMultimap.create();
    outObj.put("x", 1);
    outObj.put("y", "abc");

    // Try to serialize and deserialize
    Kryo kryo = new Kryo();
    kryo.register(LinkedHashMultimap.class, new JavaSerializer());
    writeObj(kryo, outObj);
    LinkedHashMultimap<String, Object> inObj = (LinkedHashMultimap<String, Object>) readObj(kryo);

    System.out.println(inObj);
}

public static Object readObj(Kryo kryo) {
    Object obj = null;
    try {
        Input input = new Input(new FileInputStream(ioFileName));
        obj = kryo.readClassAndObject(input);
    } catch (FileNotFoundException e) {
        e.printStackTrace();
    }
    return obj;
}

public static void writeObj(Kryo kryo, Object obj) {
    try {
        Output output = new Output(new FileOutputStream(ioFileName));
        kryo.writeClassAndObject(output, obj);
        output.flush();
    } catch (FileNotFoundException e) {
        e.printStackTrace();
    }
}

}

The output:

{x=[1], y=[abc]}