Jackson serializing and deserializing Map<String, byte[]>

Question

What is the correct way to serialize and deserialize a Map<String, byte[]> using Jackson?

I had been doing this:

I create a Map<String, byte[]>:

Map<String, String> contextContents = new HashMap<>();
contextContents.put("a-context-key", "a-context-value");

Map<String, byte[]> context = new HashMap<>();
context.put("outer-key", OBJECT_MAPPER.writeValueAsBytes(contextContents));

I need to put this into a Postgres bytea column, so I'm doing this:

byte[] toPostgres = OBJECT_MAPPER.writeValueAsBytes(context);

Then reading it out of postgres:

Map<String, byte[]> fromPostgres = OBJECT_MAPPER.readValue(toPostgres, Map.class)

However, when I pass the fromPostgres field along to another component that tries to serialize it again later:

class MyMessage
{
    public Map<String, byte[]> context;
}

MyMessage myMessage = new MyMessage();
myMessage.context = fromPostgres;

I get the following error:

Section of interest:

com.fasterxml.jackson.databind.JsonMappingException: class java.lang.String cannot be cast to class [B (java.lang.String and [B are in module java.base of loader 'bootstrap') (through reference chain: com.amce.MyMessage["context"]->java.util.LinkedHashMap["outer-key"])

Full error:

"exception":"java.lang.ClassCastException: class java.lang.String cannot be cast to class [B (java.lang.String and [B are in module java.base of loader 'bootstrap')
     at com.fasterxml.jackson.databind.ser.std.ByteArraySerializer.serialize(ByteArraySerializer.java:30)
     at com.fasterxml.jackson.databind.ser.std.MapSerializer.serializeFieldsUsing(MapSerializer.java:817)
     ... 23 common frames omitted
Wrapped by: com.fasterxml.jackson.databind.JsonMappingException: class java.lang.String cannot be cast to class [B (java.lang.String and [B are in module java.base of loader 'bootstrap') (through reference chain: com.amce.MyMessage[\"context\"]->java.util.LinkedHashMap[\"outer-key\"])
     at com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingException.java:397)
     at com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingException.java:356)
     at com.fasterxml.jackson.databind.ser.std.StdSerializer.wrapAndThrow(StdSerializer.java:316)
     at com.fasterxml.jackson.databind.ser.std.MapSerializer.serializeFieldsUsing(MapSerializer.java:822)
     at com.fasterxml.jackson.databind.ser.std.MapSerializer.serialize(MapSerializer.java:641)
     at com.fasterxml.jackson.databind.ser.std.MapSerializer.serialize(MapSerializer.java:33)
     at com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:727)
     at com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:722)
     at com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:166)
     at com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSerializerProvider.java:480)
     at com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(DefaultSerializerProvider.java:319)
     at com.fasterxml.jackson.databind.ObjectMapper._configAndWriteValue(ObjectMapper.java:4110)
     at com.fasterxml.jackson.databind.ObjectMapper.writeValueAsBytes(ObjectMapper.java:3437)
     at 

I found an answer on this post:

https://stackoverflow.com/a/48639642/12177456

Instead of:

Map<String, byte[]> fromPostgres = OBJECT_MAPPER.readValue(toPostgres, Map.class)

use:

TypeReference<Map<String, byte[]>> typeRef
    = new TypeReference<Map<String, byte[]>>()
{
};
Map<String, byte[]> fromPostgres = OBJECT_MAPPER.readValue(toPostgres, typeRef);

I'd just like to be sure that this is the correct way to serialize and deserialize a Map<String, byte[]>, where the byte[] array could be anything, such as another class etc?

Answer 1

Jackson's ObjectMapper converts data to and from JSON. JSON only has five data types (string, number, Boolean, object and array), but Java obviously has many more. Type information is lost when Jackson serializes data and then has to be guessed when it is deserialized.

Jackson serializes the example context contents as:

{"a-context-key":"a-context-value"}

As both the value and key were original strings and a JSON object is a natural equivalent of a Java Map, this is fine. When asked to write the value as bytes, Jackson assumes that a UTF-8 encoding of the JSON is appropriate.

JSON has no representation for a byte array, so Jackson converts it to a JSON string by encoding the data using Base-64. The context therefore looks like this:

{"outer-key":"eyJhLWNvbnRleHQta2V5IjoiYS1jb250ZXh0LXZhbHVlIn0="}

The fact that the value was once a byte[] is lost.

When Jackson is asked to deserialize via:

Map<String, byte[]> fromPostgres = OBJECT_MAPPER.readValue(toPostgres, Map.class)

there is a problem. Jackson only knows the output should be a Map. It is told nothing of the key and value types. The fact that the result will be stored in a Map<String,byte[]> is not visible to Jackson. What it sees is a JSON object with strings for values and the default way to convert that to a Map would be to a Map<String,String>. Knowing no better, that is what it does, and the Java Generics are not sophisticated enough to catch the mismatch.

As you identified, the solution is to give Jackson precise instructions about the key and value types for the Map. Doing it this way:

TypeReference<Map<String, byte[]>> typeRef = new TypeReference<Map<String, byte[]>>() {};
Map<String, byte[]> fromPostgres = OBJECT_MAPPER.readValue(toPostgres, typeRef);

Allows the type of the key and value to be specified at compile time by making an anonymous class whose embedded generic information matches the map's key and value types.

It can also be done at runtime, using Jackson's type factory:

Class<?> keyType = String.class;
Class<?> valueType = byte[].class;

JavaType mapType = OBJECT_MAPPER.getTypeFactory()
    .constructMapType(
        Map.class,
        keyType,
        valueType);
Map<String, byte[]> fromPostgres = OBJECT_MAPPER.readValue(toPostgres, mapType);

In general, Jackson only needs this kind of assistance when Generics are involved and type-erasure has removed the information Jackson needs to deserialize accurately. For example, Jackson can serialize and deserialize the MyMessage class without problem, even though it has a Map<String,byte[]> property as the key and value types are retained in the definition of the MyMessage class.

Jackson may also need classes annotated directly or with mix-ins to enable successful serialization and deserialization. For example:

OBJECT_MAPPER.writeValueAsString(Character.UnicodeBlock.GREEK));

will fail, even though the Jackson could successfully serialize and deserialize the class using just its name. In this case one can tell Jackson what to do by using a "mix-in", like this:

  abstract static class MixInUnicodeBlock {
    @JsonCreator
    public static UnicodeBlock forName(String name) {
      return UnicodeBlock.forName(name);
    }

    @JsonValue
    public abstract String toString();
  }

  public void example() {
    ObjectMapper OBJECT_MAPPER = new ObjectMapper();
    OBJECT_MAPPER.addMixIn(UnicodeBlock.class,MixInUnicodeBlock.class);
    System.out.println(OBJECT_MAPPER.writeValueAsString(UnicodeBlock.GREEK));
    System.out.println(OBJECT_MAPPER.readValue("\"GREEK\"",UnicodeBlock.class));
  }

To sum up, in order to successfully serialize Map<String,SomeClass>:

• If this is the root value, you need to provide the missing generic information.
  • You can use a TypeReference at compile time to store the generic information.
  • You can use the TypeFactory to create a JavaType instance, which allows it to be specified at runtime.
• If the map is a property of a class, the generic information will be retained and may not need to be further specified.
• A class may need to be annotated directly, or via a mix-in, to be processed successfully.
• To precisely control the serialization and deserialization, sometimes a custom serializer or deserializer is required.

Answer 2

Probably, when you save the data from generic type, you would also use the type aware writing.

Instead of:

   byte[] toPostgres = OBJECT_MAPPER.writeValueAsBytes(context);

use:

   byte[] toPostgres = OBJECT_MAPPER.writerFor(typeRef).writeValueAsBytes(context);

where typeRef is the same value like during data restoring.