JsonMappingException occurs while converting object to JSON string - org.apache.avro.AvroRuntimeException: Not an array:

Question

While converting Java object to JSON string, I'm facing JsonMappingException. Below is the complete exception message.

com.fasterxml.jackson.databind.JsonMappingException: Not an array: {"type":"record","name":"ClearingSystemMemberIdentification2","namespace":"com.sam.eps.paymentdomain.avro.pacs002","doc":"Schema for com.sam.eps.iso.pacs002.ClearingSystemMemberIdentification2","fields":[{"name":"clearing_system_identification","type":["null",{"type":"record","name":"ClearingSystemIdentification2Choice","doc":"Schema for com.sam.eps.iso.pacs002.ClearingSystemIdentification2Choice","fields":[{"name":"code","type":["null",{"type":"string","avro.java.string":"String"}],"default":null},{"name":"proprietary","type":["null",{"type":"string","avro.java.string":"String"}],"default":null}]}],"default":null},{"name":"member_identification","type":["null",{"type":"string","avro.java.string":"String"}],"default":null}]} (through reference chain: com.sam.eps.paymentdomain.avro.pacs002.Document["fi_to_fi_payment_status_report"]->com.sam.eps.paymentdomain.avro.pacs002.FIToFIPaymentStatusReportV10["group_header"]->com.sam.eps.paymentdomain.avro.pacs002.GroupHeader91["instructed_agent"]->com.sam.eps.paymentdomain.avro.pacs002.BranchAndFinancialInstitutionIdentification6["financial_institution_identification"]->com.sam.eps.paymentdomain.avro.pacs002.FinancialInstitutionIdentification18["clearing_system_member_identification"]->com.sam.eps.paymentdomain.avro.pacs002.ClearingSystemMemberIdentification2["schema"]->org.apache.avro.Schema$RecordSchema["elementType"])

Below is the java code which I use for converting Java object to JSON String. I tried enabling ACCEPT_SINGLE_VALUE_AS_ARRAY, ACCEPT_EMPTY_ARRAY_AS_NULL_OBJECT but still facing the issue. Not sure what is causing JsonMappingException: Not an array issue.

private String convertObjectToJson(Object request) throws JsonProcessingException {

        ObjectMapper objectMapper = new ObjectMapper();
        objectMapper.registerModule(new JavaTimeModule());
        SimpleModule simpleModule = new SimpleModule();
        simpleModule.addSerializer(OffsetDateTime.class, new JsonSerializer<OffsetDateTime>() {
            @Override
            public void serialize(OffsetDateTime offsetDateTime, JsonGenerator jsonGenerator, SerializerProvider serializerProvider) throws IOException {
                jsonGenerator.writeString(DateTimeFormatter.ISO_OFFSET_DATE_TIME.format(offsetDateTime));
            }
        });
        simpleModule.addSerializer(LocalDate.class, new JsonSerializer<LocalDate>() {
            @Override
            public void serialize(LocalDate localDate, JsonGenerator jsonGenerator, SerializerProvider serializerProvider) throws IOException {
                jsonGenerator.writeString(DateTimeFormatter.ISO_LOCAL_DATE.format(localDate));
            }
        });
        objectMapper.registerModule(simpleModule);
        objectMapper.configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false);
        objectMapper.enable(DeserializationFeature.ACCEPT_SINGLE_VALUE_AS_ARRAY);
        objectMapper.enable(DeserializationFeature.ACCEPT_EMPTY_ARRAY_AS_NULL_OBJECT);
        objectMapper.setSerializationInclusion(JsonInclude.Include.NON_NULL);
        return objectMapper.writeValueAsString(request);
    }

Answer 1

Issue shows out when you convert an Avro POJO to JSON using Jackson. The solution is unbelievably humble: avroPojo.toString().

You can use this utility method to get the difference of the two compared JSONs:

import com.fasterxml.jackson.core.type.TypeReference;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.google.common.collect.MapDifference;
import com.google.common.collect.Maps;
import lombok.SneakyThrows;
import lombok.experimental.UtilityClass;

import java.util.HashMap;
import java.util.Map;

@UtilityClass
public class JsonUtils {

private static final ObjectMapper MAPPER = new ObjectMapper();

/**
 * Compare two String JSON-objects.
 *
 * @param sourceJson Basic object to compare.
 * @param targetJson Comparable object.
 * @return A map of fields that differ.
 */
@SneakyThrows
public static MapDifference<String, Object> compareJsons(String sourceJson, String targetJson) {

    Map<String, Object> sourceJsonMap = MAPPER.readValue(
            sourceJson,
            new TypeReference<HashMap<String, Object>>() {});

    Map<String, Object> targetJsonMap = MAPPER.readValue(
            targetJson,
            new TypeReference<HashMap<String, Object>>() {});

    return Maps.difference(sourceJsonMap, targetJsonMap);
  }
}

Required dependencies:

    <dependency>
        <groupId>org.projectlombok</groupId>
        <artifactId>lombok</artifactId>
        <version>${lombok.version}</version>
        <scope>provided</scope>
    </dependency>
    <dependency>
        <groupId>com.fasterxml.jackson.core</groupId>
        <artifactId>jackson-databind</artifactId>
        <version>${jackson-databind.version}</version>
    </dependency>
    <dependency>
        <groupId>com.google.guava</groupId>
        <artifactId>guava</artifactId>
        <version>32.0.1-jre</version>
        <scope>compile</scope>
    </dependency>

Answer 2

You can use JsonSerializer for Avro Schema checked code:

import com.fasterxml.jackson.core.JsonGenerator;
import com.fasterxml.jackson.databind.JsonSerializer;
import com.fasterxml.jackson.databind.SerializerProvider;
import org.apache.avro.Schema;

import java.io.IOException;

public class AvroSchemaSerializer extends JsonSerializer<Schema> {
    @Override
    public void serialize(Schema value, JsonGenerator jgen, SerializerProvider provider) throws IOException {
        String res = value.toString();
        jgen.writeRaw(res);
    }
}

Answer 3

This issue comes when you auto generate your POJOs from Avro schema using Avro compiler. This auto generated POJO contains below extra field among others (apart from the one specified by the user in the Avro schema)

public static final org.apache.avro.Schema SCHEMA$;

Due to some bug in Avro API the Schema.getElementType() method causes Not an array Exception during serialization. Below is the method definition

 public Schema getElementType() {
    throw new AvroRuntimeException("Not an array: " + this);
  }

To fix this issue, just ignore SCHEMA$ field in the auto generated POJO during serialization as below

// create this class to ignore one or more fields
public abstract class IgnoreSchemaProperty
{
    @JsonIgnore
    abstract void getSchema();
}

public class Test{
    public static void main(String args[]) {
        ObjectMapper objectMapper=new ObjectMapper();
        //pass the above created class in the mixIn
        objectMapper.addMixIn(YourPojo.class, IgnoreSchemaProperty.class);
        String pojoJson = objectMapper.writeValueAsString(YourPojo);
        System.out.println(pojoJson);
    }
}

Answer 4

The Jackson conversion process evaluates each and every field or getter of the object recursively, so it can create the JSON nodes.

You are probably converting a generated AVRO type here. It contains a field called $SCHEMA which itself does not have a method called getElementType() implemented. The default method getElementType() of org.apache.avro.Schema throws this exception explicitly. Since Jackson comes across this method during its conversion process, the exception is thrown.

getElementType() Is just the one of many default implementations in that Schema class which throw an exception by default. I don't think there is any way around it, except for not using Jackson to create JSON, but instead use the same API that created the AVRO type in the first place.

EDIT: maybe you could register a JsonSerializer<Schema> to your mapper, and customize the serialization of Schema yourself.