CODEWITHSUNDEEP
javajsonstreamjacksonvert.x
Arie
Arie

Reputation: 12385

How to stream JSON result with Jackson in Vert.x (java)

I'm making a REST api for my Java database-like service using Vert.x. It's not too dificult to write the JSON result as a String to the request's stream, as shown below:

...
routeMatcher.get("/myservice/api/v1/query/:query", req -> {

    // get query
    String querySring = req.params().get("query");           
    Query query = jsonMapper.readValue(querySring, Query.class);

    // my service creates a list of resulting records...
    List<Record> result = myservice.query(query);                
    String jsonResult = jsonMapper.writeValueAsString(result);

    // write entire string to response
    req.response().headers().set("Content-Type", "application/json; charset=UTF-8");
    req.response().end(jsonResult);    
});
...

However I'd like to stream the Java List to the request object by using Jackson's method:

ObjectMapper objectMapper = new ObjectMapper();
objectMapper.writeValue(Outputstream, result);

But I don't know how to connect Jackson's Outputstream argument to Vert.x's re.response(), as they have their own Buffer system that seems incompatible with Jackson's java.io.Outputstream argument.

Can't I use Jackson in combination with Vert.x? Should I write a custom serializer by hand with Vert.x's own JSON library? Other suggestions?

Upvotes: 3

Views: 6214

Answers (2)

igr
igr

Reputation: 10604

This might be a bit better (and updated for Vertx3) answer:

import io.vertx.core.file.AsyncFile;
import io.vertx.core.buffer.Buffer;
import io.vertx.core.http.HttpServerResponse;
import io.vertx.core.streams.WriteStream;

import java.io.IOException;
import java.io.OutputStream;

public class OutputWriterStream extends OutputStream {

    public OutputWriterStream(final WriteStream response) {
        this.response = response;
        this.buffer = new byte[8192];
    }

    @Override
    public synchronized void write(final int b) throws IOException {
        buffer[counter++] = (byte) b;

        if (counter >= buffer.length) {
            flush();
        }
    }

    @Override
    public void flush() throws IOException {
        super.flush();

        if (counter > 0) {
            byte[] remaining = buffer;

            if (counter < buffer.length) {
                remaining = new byte[counter];

                System.arraycopy(buffer, 0, remaining, 0, counter);
            }

            response.write(Buffer.buffer(remaining));
            counter = 0;
        }
    }

    @Override
    public void close() throws IOException {
        flush();

        super.close();

        if (response instanceof HttpServerResponse) {
            try {
                response.end();
            }
            catch (final IllegalStateException ignore) {
            }
        }
        else if (response instanceof AsyncFile) {
            ((AsyncFile) response).close();
        }
    }

    private final WriteStream<Buffer> response;
    private final byte[] buffer;
    private int counter = 0;

}

Upvotes: 2

jozh
jozh

Reputation: 2702

I assume you are generating huge JSON documents as for the small ones string output is good enough: objectMapper.writeValue(<String>, result);

There's a problem with streams. ObjectMapper doesn't know the result size and you will end up with the exception:

java.lang.IllegalStateException: You must set the Content-Length header to be the total size of the message body BEFORE sending any data if you are not using HTTP chunked encoding.
        at org.vertx.java.core.http.impl.DefaultHttpServerResponse.write(DefaultHttpServerResponse.java:474)

So in your example I would use temporary files for JSON output and then flush them into response (I haven't tested the code)

File tmpFile = File.createTempFile("tmp", ".json");
mapper.writeValue(tmpFile, result);
req.response().sendFile(tmpFile.getAbsolutePath(), (result) -> tmpFile.delete());

In case you know content length initially you can use the following code to map OutputStream with WriteStream

import org.vertx.java.core.buffer.Buffer;
import org.vertx.java.core.streams.WriteStream;

import java.io.IOException;
import java.io.OutputStream;

public class OutputWriterStream extends OutputStream {

    public WriteStream writeStream;
    public Runnable closeHandler;

    @Override
    public void write(int b) throws IOException {
        throw new UnsupportedOperationException();
    }

    @Override
    public void write(byte[] b, int off, int len) throws IOException {
        if (off == 0 && len == b.length) {
            writeStream.write(new Buffer(b));
            return;
        }

        byte[] bytes = new byte[len];
        System.arraycopy(b, off, bytes, 0, len);
        writeStream.write(new Buffer(bytes));
    }

    @Override
    public void write(byte[] b) throws IOException {
        writeStream.write(new Buffer(b));
    }

    @Override
    public void close() throws IOException {
        closeHandler.run();
    }
}

Upvotes: 2

Related Questions