CODEWITHSUNDEEP
javajsonperformancejackson
namalfernandolk
namalfernandolk

Reputation: 9134

Slowness with creating a String from a ObjectNode with Jackson in Java

I have problem with creating a String from a JSON node.

Currently I'm doing it by node.toString() method. But in sometimes this takes 7-8 seconds to create the JSON string weighted 15MB-18MB.

I tried with mapper.writeValueAsString(node)) method too. But it shows some additional time for the test. This is very difficult to check the issue because it is also harder to reproduce.

I'm currently using only ObjectNode (not TextNode, BooleanNode etc) is it this will be effect to this? Or is there any better way to convert JSONNode to String ?

Sample Code : JsonNodeFactory nodeFactory = JsonNodeFactory.instance;

ObjectNode  node  = nodeFactory.objectNode();
node.put("fnm", "Namal");
node.put("lnm", "Fernando");
node.put("age", 30);

for (int i = 0; i < 10000; i++) {

    ObjectNode  order  = nodeFactory.objectNode();
    order.put("id", (i+1000)+"");
    order.put("nm", "ORD"+(i+1000));
    order.put("ref", "RF-"+i);

    node.put("order"+i, order);

}

long smili  = System.currentTimeMillis();
System.out.println("main().Node : " + node.toString());
System.out.println("main().TIMING 1 : " + (System.currentTimeMillis() - smili) / 1000.0);;

long smili2 = System.currentTimeMillis();
ObjectMapper mapper = new ObjectMapper();

System.out.println("main().Node : " + mapper.writeValueAsString(node));
System.out.println("main().TIMING 2 : " + (System.currentTimeMillis() - smili2) / 1000.0);;

Upvotes: 2

Views: 1973

Answers (1)

StaxMan
StaxMan

Reputation: 116572

First things first: JsonNode.toString() should NOT be used for serialization, ever. It is useful for simple troubleshooting, but since it does not have access to contextual configuration, it will not necessarily produce valid JSON. This is by design. Instead, you should use ObjectMapper or ObjectWriter to serialize it; this will produce valid JSON using exact configuration and settings that mapper has (or writer created by mapper).

Now: your timing comparison is flawed since you only do one call with ObjectMapper. There is overhead in first N calls; partly due to ObjectMapper initialization, partly due to JVM warmup (dynamic JIT compiler running to optimize and such). To get more usable results you would need to call method multiple times, and ideally let it run for couple of seconds.

But beyond this a common problem is to forget to have enough heap space. In Java, Strings require memory at least equivalent to 2x length of String in characters (each char takes 2 bytes). But this is just the size of the result; during serialization a temporary buffer is also needed, using roughly comparable amount. On disk, however, UTF-8 encoding typically uses just 1 byte per character (for ASCII characters).

So assuming you see a 15mB file, it could use 60 mB of memory during processing. If your heap size is set to small (say, 64 megs which is default in many cases) it would lead to very heavy garbage-collection processing. Solution there would be to either increase heap size. Or, unless you actually need the String, to:

  1. Write directly into file: use of intermediate String is an anti-pattern if the result goes into File or network connection
  2. If you do require intermediate form, serialize as byte[]: this will only need same amount of memory as a file would (since it is UTF-8 encoded in memory, instead of being a wrapped char[]

Upvotes: 2

Related Questions