Using readClassDescriptor() and maybe resolveClass() to permit Serialization versioning

Question

I am investigating different options in the Java Serialization mechanism to allow flexibility in our class structures for version-tolerant storage (and advocating for a different mechanism, you don't need to tell me).

For instance, the default serialization mechanism can handle both adding and removing fields, if only backwards compatibility is required.

Renaming a class or moving it to a different package has proved to be much harder, though. I found in this question that I was able to do a simple class rename and/or move package, by subclassing ObjectInputStream and overriding readClassDescriptor():

    if (resultClassDescriptor.getName().equals("package.OldClass"))
        resultClassDescriptor = ObjectStreamClass.lookup(newpackage.NewClass.class);

That is fine for simple renames. But if you then try to add or delete a field, you get a java.io.StreamCorruptedException. Worse, this happens even if a field has been added or deleted, and then you rename the class, which could cause problems with multiple developers or multiple checkins.

Based on some reading I had done, I experimented a bit with also overriding resolveClass(), with the idea that we were correctly repointing the name to the new class, but not loading the old class itself and bombing on the field changes. But this comes from a very vague understanding of some the details of the Serialization mechanism, and I'm not sure if I'm even barking up the right tree.

So 2 precise questions:

1. Why is repointing the class name using readClassDescriptor() causing deserialization to fail on normal, compatible class changes?
2. Is there a way using resolveClass() or another mechanism to get around this and allow classes to both evolve (add and remove fields) and be renamed/repackaged?

I poked around and could not find an equivalent question on SO. By all means, point me to such a question if it exists, but please read the question carefully enough that you do not close me unless another question actually answers my precise question.

Answer 1

I recently faced the same problem, that is, StreamCorruptedException deserializing objects of classes that were moved from one package to another and then evolved in a compatible way by adding new fields. Although @gaponov answer initially solved it, I find the following solution more appropriate because it does not need to mess with the Class name. The class using the ObjectInputStreamAdapter defines the mapping and the internal class ObjectInputStreamAdapter only redefines the resolveClass method:

    public class Deserializer {

    /*
     * Mapping that stores the specific new classes to use for old serialized
     * class names in order to transform old classes to the new ones for
     * compatibility reasons
     */
    private static final Map<String, Class<?>> classMapping = new HashMap<>();

    static {
        classMapping.put("com.example.old.SomeClass",
                SomeClass.class);
        classMapping.put("com.example.old.SomeClass2",
                SomeClass2.class);
    }

    public void deserialize(byte[] bytes) {
        try (ObjectInputStream o =
                new ObjectInputStreamAdapter(new ByteArrayInputStream(bytes))) {
            Object object = o.readObject();
            /* ... */
        } catch (Exception e) {
            throw new SerializationException("Cannot deserialize", e);
        }
    }

    /*
     * Adaptor that transform old classes to the new classes for compatibility
     * reasons
     */
    private class ObjectInputStreamAdapter extends ObjectInputStream {

        public ObjectInputStreamAdapter(InputStream in) throws IOException {
            super(in);
        }

        @Override
        protected Class<?> resolveClass(ObjectStreamClass desc)
                throws IOException, ClassNotFoundException {
            Class<?> klazz = classMapping.get(desc.getName());
            if (klazz != null) {
                return klazz;
            } else {
                return super.resolveClass(desc);
            }
        }

    }

}

Answer 2

I had same problems with flexibility like you and I found the way. So here my version of readClassDescriptor()

    static class HackedObjectInputStream extends ObjectInputStream
{

    /**
     * Migration table. Holds old to new classes representation.
     */
    private static final Map<String, Class<?>> MIGRATION_MAP = new HashMap<String, Class<?>>();

    static
    {
        MIGRATION_MAP.put("DBOBHandler", com.foo.valueobjects.BoardHandler.class);
        MIGRATION_MAP.put("DBEndHandler", com.foo.valueobjects.EndHandler.class);
        MIGRATION_MAP.put("DBStartHandler", com.foo.valueobjects.StartHandler.class);
    }

    /**
     * Constructor.
     * @param stream input stream
     * @throws IOException if io error
     */
    public HackedObjectInputStream(final InputStream stream) throws IOException
    {
        super(stream);
    }

    @Override
    protected ObjectStreamClass readClassDescriptor() throws IOException, ClassNotFoundException
    {
        ObjectStreamClass resultClassDescriptor = super.readClassDescriptor();

        for (final String oldName : MIGRATION_MAP.keySet())
        {
            if (resultClassDescriptor.getName().equals(oldName))
            {
                String replacement = MIGRATION_MAP.get(oldName).getName();

                try
                {
                    Field f = resultClassDescriptor.getClass().getDeclaredField("name");
                    f.setAccessible(true);
                    f.set(resultClassDescriptor, replacement);
                }
                catch (Exception e)
                {
                    LOGGER.severe("Error while replacing class name." + e.getMessage());
                }

            }
        }

        return resultClassDescriptor;
    }

Answer 3

This is what writeReplace() and readResolve() are for. You're making it much more complicated than it really is. Note that you can define these methods either in the two objects concerned or in subclasses of thee Object stream classes.

Answer 4

I haven't tinkered with class descriptors much enough, but if your problem is just about renaming and repackaging there is a much easier solution for that. You could just simply edit your serialized data file with a text editor and just replace your old names with new ones. it's there in a human-readable form. for example suppose we have this OldClass placed inside oldpackage and containing an oldField, like this:

package oldpackage;

import java.io.Serializable;

public class OldClass implements Serializable
{
    int oldField;
}

Now when we serialize an instance of this class and get something like this:

¬í sr oldpackage.OldClasstqŽÇ§Üï I oldFieldxp    

Now if we want to change class's name to NewClass and put inside newpackage and change its field's name to newField, I simply rename it the file, like this:

¬í sr newpackage.NewClasstqŽÇ§Üï I newFieldxp    

and define the appropriate serialVersionUID for the new class.

That's all. No extending and overriding required.

Answer 5

The problem is that readClassDescriptor is supposed to tell the ObjectInputStream how to read the data which is currently in the stream you are reading. if you look inside a serialized data stream, you will see that it not only stores the data, but lots of metadata about exactly what fields are present. this is what allows serialization to handle simple field additions/removals. however, when you override that method and discard the info returned from the stream, you are discarding the info about what fields are in the serialized data.

i think the solution to the problem would be to take the value returned by super.readClassDescriptor() and create a new class descriptor which returns the new class name, but otherwise returns the info from the old descriptor. (although, in looking at ObjectStreamField, it may be more complicated than that, but that is the general idea).