Jackson JsonNode with empty element key

Question

I am using jackson-dataformat-xml (2.9) to parse an XML into JsonNode and then parse it to JSON (the XML is very dynamic so that is why I am using JsonNode instead of binding to a POJO. e.g 'elementName' and 'id' names may vary).

It happens that during the JSON parsing phase, one of the element keys is empty string ("").

XML:

<elementName>
      <id type="pid">abcdef123</id>
</elementName>

Parsing logic:

public Parser() {
        ObjectMapper jsonMapper = new ObjectMapper();
        XmlMapper xmlMapper = new XmlMapper(new XmlFactory(new WstxInputFactory()));
}

public InputStream parseXmlResponse(InputStream xmlStream) {
        InputStream stream = null;

        try {
            JsonNode node = xmlMapper.readTree(xmlStream);
            stream = new ByteArrayInputStream(jsonMapper.writer().writeValueAsBytes(node));
        } catch (IOException e) {
            e.printStackTrace();
        }

        return stream;
    }

Json:

Result:

{
   "elementName": {
     "id": {
        "type": "pid",
        "": "abcdef123"
     }
   },
}

Expected:

{
   "elementName": {
     "id": {
        "type": "pid",
        "value": "abcdef123"
     }
   },
}

My idea is to find whenever I have the empty key "" and replace it with "value". Either at XML de-serialization or during JSON serialization. I have tried to use default serializer, filter, but haven't got it working in a nice and concise way.

Suggestions are much appreciated.

Thank you for the help.

Possible Solution:

Based on @shoek suggestion I decided to write a custom serializer to avoid creating an intermediate object (ObjectNode) during the process.

edit: refactor based on the same solution proposed by @shoek.

public class CustomNode {
    private JsonNode jsonNode;

    public CustomNode(JsonNode jsonNode) {
        this.jsonNode = jsonNode;
    }

    public JsonNode getJsonNode() {
        return jsonNode;
    }
}

public class CustomObjectsResponseSerializer extends StdSerializer<CustomNode> {

    protected CustomObjectsResponseSerializer() {
        super(CustomNode.class);
    }

    @Override
    public void serialize(CustomNode node, JsonGenerator jgen, SerializerProvider provider) throws IOException {
        convertObjectNode(node.getJsonNode(), jgen, provider);
    }

    private void convertObjectNode(JsonNode node, JsonGenerator jgen, SerializerProvider provider) throws IOException {
        jgen.writeStartObject();
        for (Iterator<String> it = node.fieldNames(); it.hasNext(); ) {
            String childName = it.next();
            JsonNode childNode = node.get(childName);
            // XML parser returns an empty string as value name. Replacing it with "value"
            if (Objects.equals("", childName)) {
                childName = "value";
            }

            if (childNode instanceof ArrayNode) {
                jgen.writeFieldName(childName);
                convertArrayNode(childNode, jgen, provider);
            } else if (childNode instanceof ObjectNode) {
                jgen.writeFieldName(childName);
                convertObjectNode(childNode, jgen, provider);
            } else {
                provider.defaultSerializeField(childName, childNode, jgen);
            }
        }
        jgen.writeEndObject();

    }

    private void convertArrayNode(JsonNode node, JsonGenerator jgen, SerializerProvider provider) throws IOException {
        jgen.writeStartArray();
        for (Iterator<JsonNode> it = node.elements(); it.hasNext(); ) {
            JsonNode childNode = it.next();

            if (childNode instanceof ArrayNode) {
                convertArrayNode(childNode, jgen, provider);
            } else if (childNode instanceof ObjectNode) {
                convertObjectNode(childNode, jgen, provider);
            } else {
                provider.defaultSerializeValue(childNode, jgen);
            }
        }
        jgen.writeEndArray();
    }
}

Answer 1

I figured out that this behaviour can be achieved via configuration. Here is the kotlin code but it's simple to convert to java Just create xmlMapper with appropriate configuration

    fun jacksonCreateXmlMapper(): XmlMapper {
        val module = JacksonXmlModule()
        module.setXMLTextElementName("value")
        return XmlMapper(module)
    }

For input

<products>
    <product count="5">apple</product>
    <product count="10">orange</product>
</products>

you get:

{
  "product" : [ {
    "count" : "5",
    "value" : "apple"
  }, {
    "count" : "10",
    "value" : "orange"
  } ]
}

Answer 2

You also could simply post-process the JSON DOM, traverse to all objects, and rename the keys that are empty strings to "value".

Race condition: such a key may already exist, and must not be overwritten
(e.g. <id type="pid" value="existing">abcdef123</id>).

Usage:
(note: you should not silently suppress the exception and return null, but allow it to propagate so the caller can decide to catch and apply failover logic if required)

public InputStream parseXmlResponse(InputStream xmlStream) throws IOException {
    JsonNode node = xmlMapper.readTree(xmlStream);
    postprocess(node);
    return new ByteArrayInputStream(jsonMapper.writer().writeValueAsBytes(node));
}

Post-processing:

private void postprocess(JsonNode jsonNode) {

    if (jsonNode.isArray()) {
        ArrayNode array = (ArrayNode) jsonNode;
        Iterable<JsonNode> elements = () -> array.elements();

        // recursive post-processing
        for (JsonNode element : elements) {
            postprocess(element);
        }
    }
    if (jsonNode.isObject()) {
        ObjectNode object = (ObjectNode) jsonNode;
        Iterable<String> fieldNames = () -> object.fieldNames();

        // recursive post-processing
        for (String fieldName : fieldNames) {
            postprocess(object.get(fieldName));
        }
        // check if an attribute with empty string key exists, and rename it to 'value',
        // unless there already exists another non-null attribute named 'value' which
        // would be overwritten.
        JsonNode emptyKeyValue = object.get("");
        JsonNode existing = object.get("value");
        if (emptyKeyValue != null) {
            if (existing == null || existing.isNull()) {
                object.set("value", emptyKeyValue);
                object.remove("");
            } else {
                System.err.println("Skipping empty key value as a key named 'value' already exists.");
            }
        }
    }
}

Output: just as expected.

{
   "elementName": {
     "id": {
        "type": "pid",
        "value": "abcdef123"
     }
   },
}

EDIT: considerations on performance:

I did a test with a large XML file (enwikiquote-20200520-pages-articles-multistream.xml, en.wikiquote XML dump, 498.4 MB), 100 rounds, with following measured times (using deltas with System.nanoTime()):

• average read time (File, SSD): 2870.96 ms
  (JsonNode node = xmlMapper.readTree(xmlStream);)
• average postprocessing time: 0.04 ms
  (postprocess(node);)
• average write time (memory): 0.31 ms
  (new ByteArrayInputStream(jsonMapper.writer().writeValueAsBytes(node));)

That's a fraction of a millisecond for an object tree build from a ~500 MB file - so performance is excellent and no concern.

Answer 3

Serializer version.

package com.example;

import java.io.IOException;
import java.util.Iterator;
import java.util.Objects;

import com.fasterxml.jackson.core.JsonGenerator;
import com.fasterxml.jackson.core.JsonProcessingException;
import com.fasterxml.jackson.databind.JsonNode;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.fasterxml.jackson.databind.SerializerProvider;
import com.fasterxml.jackson.databind.module.SimpleModule;
import com.fasterxml.jackson.databind.module.SimpleSerializers;
import com.fasterxml.jackson.databind.node.ArrayNode;
import com.fasterxml.jackson.databind.node.ObjectNode;
import com.fasterxml.jackson.databind.ser.std.StdSerializer;

public class Stackoverflow62009220_B {
    public static void main(String[] args) throws JsonProcessingException {
        // see https://www.baeldung.com/jackson-call-default-serializer-from-custom-serializer

        convert("{\"elementName\":{\"id\":{\"type\":\"pid\",\"\":\"abcdef123\"}}}");

        // j =  {"":"is_empty_field","num":1,"str":"aa","null_val":null,"empty_val":"","array":[3,5],"obj":{"a":"A","b":22}}
        // (simple json object)
        String j = "{\"\":\"is_empty_field\",\"num\":1,\"str\":\"aa\",\"null_val\":null,\"empty_val\":\"\",\"array\":[3,5],\"obj\":{\"a\":\"A\",\"b\":22}}";
        convert(j);

        // g = {"":"is_empty_field","num":1,"str":"aa","null_val":null,"empty_val":"","array":[3,{"":"is_empty_field","num":1,"str":"aa","null_val":null,"empty_val":"","array":[3,5],"obj":{"a":"A","b":22}}],"obj":{"":"is_empty_field","num":1,"str":"aa","null_val":null,"empty_val":"","array":[3,5],"obj":{"a":"A","b":22}}}
        // (includes an array containing object j, and an object j containing array)
        String g = " {\"\":\"is_empty_field\",\"num\":1,\"str\":\"aa\",\"null_val\":null,\"empty_val\":\"\",\"array\":[3,{\"\":\"is_empty_field\",\"num\":1,\"str\":\"aa\",\"null_val\":null,\"empty_val\":\"\",\"array\":[3,5],\"obj\":{\"a\":\"A\",\"b\":22}}],\"obj\":{\"\":\"is_empty_field\",\"num\":1,\"str\":\"aa\",\"null_val\":null,\"empty_val\":\"\",\"array\":[3,5],\"obj\":{\"a\":\"A\",\"b\":22}}}";
        convert(g);
    }

    private static void convert(String str) throws JsonProcessingException {
        JsonNode input = (new ObjectMapper()).readTree(str);
        System.out.println("in:");
        System.out.println(input);

        ObjectMapper mapper = new ObjectMapper();
        SimpleModule module = new SimpleModule();
        SimpleSerializers serializers = new SimpleSerializers();
        serializers.addSerializer(ObjectNode.class, new MyObjectNodeSerializer());
        module.setSerializers(serializers);
        mapper.registerModule(module);

        String output = mapper.writer().writeValueAsString(input);
        System.out.println("out:");
        System.out.println(output);
        System.out.println("----------");
    }
}

class MyObjectNodeSerializer extends StdSerializer<ObjectNode> {

    public MyObjectNodeSerializer() {
        super(ObjectNode.class);
    }

    public static MyObjectNodeSerializer create() {
        return new MyObjectNodeSerializer();
    }

    @Override
    public void serialize(ObjectNode value, JsonGenerator gen, SerializerProvider provider) throws IOException {
        gen.writeStartObject();
        for (Iterator<String> it = value.fieldNames(); it.hasNext();) {
            String childName = it.next();
            JsonNode childNode = value.get(childName);

            if (Objects.equals("", childName)) {
                childName = "value";
            }

            if (childNode instanceof ArrayNode) {
                gen.writeFieldName(childName);
                MyArrayNodeSerializer.create().serialize((ArrayNode) childNode, gen, provider);
            } else if (childNode instanceof ObjectNode) {
                gen.writeFieldName(childName);
                this.serialize((ObjectNode) childNode, gen, provider);
            } else {
                provider.defaultSerializeField(childName, childNode, gen);
            }
        }
        gen.writeEndObject();
    }
}

class MyArrayNodeSerializer extends StdSerializer<ArrayNode> {

    public MyArrayNodeSerializer() {
        super(ArrayNode.class);
    }

    public static MyArrayNodeSerializer create() {
        return new MyArrayNodeSerializer();
    }

    @Override
    public void serialize(ArrayNode value, JsonGenerator gen, SerializerProvider provider) throws IOException {
        gen.writeStartArray();
        for (Iterator<JsonNode> it = value.elements(); it.hasNext();) {
            JsonNode childNode = it.next();
            if (childNode instanceof ArrayNode) {
                this.serialize((ArrayNode) childNode, gen, provider);
            } else if (childNode instanceof ObjectNode) {
                MyObjectNodeSerializer.create().serialize((ObjectNode) childNode, gen, provider);
            } else {
                provider.defaultSerializeValue(childNode, gen);
            }
        }
        gen.writeEndArray();
    }
}

Answer 4

Copying to a new ObjectNode may solve your problem.

package com.example;

import java.util.Iterator;
import java.util.Objects;

import com.fasterxml.jackson.core.JsonProcessingException;
import com.fasterxml.jackson.databind.JsonNode;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.fasterxml.jackson.databind.node.ArrayNode;
import com.fasterxml.jackson.databind.node.ObjectNode;
import com.fasterxml.jackson.databind.node.ValueNode;

public class Stackoverflow62009220 {
    public static void main(String[] args) throws JsonProcessingException {
        convert("{\"elementName\":{\"id\":{\"type\":\"pid\",\"\":\"abcdef123\"}}}");

        convert("{\"array\":[1,99,3]}");

        convert("{\"complex-array\":[null, 1, [3,7,5], {\"type\":\"pid\",\"\":\"abcdef123\"}]}");
    }

    private static void convert(String str) throws JsonProcessingException {
        JsonNode input = (new ObjectMapper()).readTree(str);
        System.out.println("in:");
        System.out.println(input);

        ObjectMapper mapper = new ObjectMapper();

        ObjectNode obj = convertObjectNode(input, mapper);

        String output = mapper.writer().writeValueAsString(obj);
        System.out.println("out:");
        System.out.println(output);
        System.out.println("----------");
    }

    private static ArrayNode convertArrayNode(JsonNode current, ObjectMapper mapper) {
        ArrayNode to = mapper.createArrayNode();
        for (Iterator<JsonNode> it = current.elements(); it.hasNext();) {
            JsonNode childNode = it.next();

            if (childNode instanceof ValueNode) {
                to.add(childNode);
            } else if (childNode instanceof ArrayNode) {
                // recurse
                to.add(convertArrayNode(childNode, mapper));
            } else if (childNode instanceof ObjectNode) {
                to.add(convertObjectNode(childNode, mapper));
            }
        }
        return to;
    }

    private static ObjectNode convertObjectNode(JsonNode current, ObjectMapper mapper) {
        ObjectNode to = mapper.createObjectNode();
        for (Iterator<String> it = current.fieldNames(); it.hasNext();) {
            String childName = it.next();
            JsonNode childNode = current.get(childName);

            if (Objects.equals("", childName)) {
                childName = "value";
            }

            if (childNode instanceof ValueNode) {
                to.set(childName, childNode);
            } else if (childNode instanceof ArrayNode) {
                to.set(childName, convertArrayNode(childNode, mapper));
            } else if (childNode instanceof ObjectNode) {
                // recurse
                to.set(childName, convertObjectNode(childNode, mapper));
            }
        }
        return to;
    }
}

The preceding code results in:

in:
{"elementName":{"id":{"type":"pid","":"abcdef123"}}}
out:
{"elementName":{"id":{"type":"pid","value":"abcdef123"}}}
----------
in:
{"array":[1,99,3]}
out:
{"array":[1,99,3]}
----------
in:
{"complex-array":[null,1,[3,7,5],{"type":"pid","":"abcdef123"}]}
out:
{"complex-array":[null,1,[3,7,5],{"type":"pid","value":"abcdef123"}]}
----------

P.S.

I couldn't find a way to use a custom serializer (like this) for non-typed JsonNode. If someone knows, please post your answer. It may be a better solution with regard to memory usage/processing time.