CODEWITHSUNDEEP
javajax-rs
lostintranslation
lostintranslation

Reputation: 24583

JaxRS create and return zip file from server

I want to create and return a zip file from my server using JaxRS. I don't think that I want to create an actual file on the server, if possible I would like to create the zip on the fly and pass that back to the client. If I create a huge zip file on the fly will I run out of memory if too many files are in the zip file?

Also I am not sure the most efficient way to do this. Here is what I was thinking but I am very rusty when it comes to input/output in java.

public Response getFiles() {

// These are the files to include in the ZIP file       
String[] filenames = // ... bunch of filenames

byte[] buf = new byte[1024];

try {
   // Create the ZIP file
   ByteArrayOutputStream baos= new ByteArrayOutputStream();
   ZipOutputStream out = new ZipOutputStream(new BufferedOutputStream(baos));

   // Compress the files
   for (String filename : filenames) {
       FileInputStream in = new FileInputStream(filename);

       // Add ZIP entry to output stream.
       out.putNextEntry(new ZipEntry(filename));

       // Transfer bytes from the file to the ZIP file
       int len;
       while ((len = in.read(buf)) > 0) {
         out.write(buf, 0, len);
       }
       // Complete the entry
       out.closeEntry();
       in.close();
   }

   // Complete the ZIP file
   out.close();

   ResponseBuilder response = Response.ok(out);  // Not a 100% sure this will work
   response.type(MediaType.APPLICATION_OCTET_STREAM);
   response.header("Content-Disposition", "attachment; filename=\"files.zip\"");
   return response.build();

} catch (IOException e) {       
}

}

Any help would be greatly appreciated.

Upvotes: 4

Views: 3450

Answers (2)

Piohen
Piohen

Reputation: 1530

There's no need to create the ZIP file from the first to the last byte in the memory before serving it to the client. Also, there's no need to create such a file in temp directory in advance as well (especially because the IO might be really slow).

The key is to start streaming the "ZIP response" and generating the content on the flight.

Let's say we have a aMethodReturningStream(), which returns a Stream, and we want to turn each element into a file stored in the ZIP file. And that we don't want to keep bytes of each element stored all the time in any intermediate representation, like a collection or an array.

Then such a pseudocode might help:

@GET
@Produces("application/zip")
public Response generateZipOnTheFly() {
    StreamingOutput output = strOut -> {
        try (ZipOutputStream zout = new ZipOutputStream(strOut)) {
            aMethodReturningStream().forEach(singleStreamElement -> {
                try {
                    ZipEntry zipEntry = new ZipEntry(createFileName(singleStreamElement));
                    FileTime fileTime = FileTime.from(singleStreamElement.getCreationTime());
                    zipEntry.setCreationTime(fileTime);
                    zipEntry.setLastModifiedTime(fileTime);
                    zout.putNextEntry(zipEntry);
                    zout.write(singleStreamElement.getBytes());
                    zout.flush();
                } catch (IOException e) {
                    throw new RuntimeException(e);
                }
            });
        }
    };
    return Response.ok(output)
                   .header("Content-Disposition", "attachment; filename=\"generated.zip\"")
                   .build();
}

This concept relies on passing a StreamingOutput to the Response builder. The StreamingOutput is not a full response/entity/body generated before sending the response, but a recipe used to generate the flow of bytes on-the-fly (here wrapped into ZipOutputStream). If you're not sure about this, then maybe set a breakpoint next on flush() and observe the a download progress using e.g. wget. The key thing to remember here is that the stream here is not a "wrapper" of pre-computed or pre-fetched items. It must be dynamic, e.g. wrapping a DB cursor or something like that. Also, it can be replaced by anything that's streaming data. That's why it cannot be a foreach loop iterating over Element[] elems array (with each Element having all the bytes "inside"), like

for(Element elem: elems)

if you'd like to avoid reading all items into the heap at once before streaming the ZIP.

(Please note this is a pseudocode and you might want to add better handling and polish other stuff as well.)

Upvotes: 1

angelcervera
angelcervera

Reputation: 4177

There are two options:

1- Create ZIP in a temporal directory and then dump to client.

2- Use OutputStream from the Response to send zip directly to the client, when you are creating them.

But never use memory to create huge ZIP file.

Upvotes: 2

Related Questions