CODEWITHSUNDEEP
javacxfjax-rs
Virtually Real
Virtually Real

Reputation: 1685

Disabling Multipart Caching in CXF jax-rs

I posted this question to the CXF list, without any luck. So here we go. I am trying to upload large files to a remote server (think of them virtual machine disks). So I have a restful service that accepts upload requests. The handler for the upload looks like:

@POST
@Consumes(MediaType.MULTIPART_FORM_DATA)
@Path("/doupload")
public Response receiveStream(MultipartBody multipart) {
    List<Attachment> allAttachments = body.getAllAttachments();
    Attachment att = null;
    for (Attachment b : allAttachments) {
        if (UPLOAD_FILE_DESCRIPTOR.equals(b.getContentId())) {
            att = b;
        }
    }
    Assert.notNull(att);
    DataHandler dh = att.getDataHandler();
    if (dh == null) {
        throw new WebApplicationException(HTTP_BAD_REQUEST);
    }
    try {
        InputStream is = dh.getInputStream();
        byte[] buf = new byte[65536];
        int n;
        OutputStream os = getOutputStream();
        while ((n = is.read(buf)) > 0) {
            os.write(buf, 0, n);
        }
        ResponseBuilder rb = Response.status(HTTP_CREATED);
        return rb.build();
    } catch (IOException e) {
        log.error("Got exception=", e);
        throw new WebApplicationException(HTTP_INTERNAL_ERROR);
    } catch (NoSuchAlgorithmException e) {
        log.error("Got exception=", e);
        throw new WebApplicationException(HTTP_INTERNAL_ERROR);
    } finally {}

}

The client for this code is fairly simple:

public void sendLargeFile(String filename) {
    WebClient wc = WebClient.create(targetUrl);
    InputStream is = new FileInputStream(new File(filename));
    Response r = wc.post(new Attachment(Constants.UPLOAD_FILE_DESCRIPTOR,
        MediaType.APPLICATION_OCTET_STREAM, is));
}

The code works fine in terms of functionality. In terms of performance, I noticed that before my handler (receiveStream() method) gets the first byte out of the stream, the whole stream actually gets persisted into a temporary file (using a CachedOutputStream). Unfortunately, this is not acceptable for my purposes.

Given all of this, I am looking for a way to use the HTTP InputStream (or as close to it as possible) to read the attachment directly from there and handle it afterwards. I guess the question translates into one of: - Is there a way to tell CXF not do caching - OR - is there a way to pass CXF an output stream (one I write) to use, rather than using CachedOutputStream

I found a similar question here. The resolution says use CXF 2.2.3 or later, I am using 2.4.4 (and tried with 2.7.0) with no luck.

Thanks.

Upvotes: 3

Views: 2504

Answers (2)

Virtually Real
Virtually Real

Reputation: 1685

I ended up fixing the problem in an unelegant way, but it works, so I wanted to share my experience. Please do let me know if there are some "standard" or better ways.

Since I am writing the server side, I knew I was accessing all the attachments in the order they were sent, and process them as they are streamed in. So, to reflect that behavior of the handler method (receiveStream() method above), I created a new annotation on the server side called "@SequentialAttachmentProcessing" and annotatated my above method with it.

Also, wrote a subclass of Attachment, called SequentialAttachment that acts like a linked list. It has a skip() method that skips over the current attachment, and when an attachment ends, hasMore() method tells you whether there is another one.

Then I wrote a custom multipart/form-data provider which behaves as follows: If the target method is annotated as above, handle the attachment, otherwise call the default provider to do the handling. When it is handled by my provider, it always returns at most one attachment. Hence it could be misleading to a non-suspecting handling method. However, I think it is acceptable since the writer of the server must have annotated the method as "@SequentialAttachmentProcessing" and therefore must know what that entails.

As a result the implementation of the receiveStream() method is now something like:

@POST
@SequentialAttachmentProcessing
@Consumes(MediaType.MULTIPART_FORM_DATA)
@Path("/doupload")
public Response receiveStream(MultipartBody multipart) {
    List<Attachment> allAttachments = body.getAllAttachments();
    Assert.isTrue(allAttachments.size() <= 1);
    if (allAttachment.size() > 0) {
        Attachment head = allAttachments.get(0);
        Assert.isTrue(head instanceof SequentialAttachment);
        SequentialAttachment att = (SequentialAttachment) head;
        while (att != null) {
            DataHandler dh = att.getDataHandler();
            InputStream is = dh.getInputStream();
            byte[] buf = new byte[65536];
            int n;
            OutputStream os = getOutputStream();
            while ((n = is.read(buf)) > 0) {
                os.write(buf, 0, n);
            }
            if (att.hasMore()) {
                att = att.next();
            }
        }
    }
}

While this solved my immediate problem, I still believe there has to be a standard way of doing this. I hope this helps someone.

Upvotes: 1

yegor256
yegor256

Reputation: 105210

I think it's logically not possible (neither in CXF or anywhere else). You're calling getAllAttachements(), which means that the server should collect information about them from the HTTP input stream. It means that the entire stream has to go into memory for MIME parsing.

In your case you should work directly with the stream, and do the MIME parsing yourself:

public Response receiveStream(InputStream input) {

Now you have full control of the input and can consume it into memory byte-by-byte.

Upvotes: 3

Related Questions