CODEWITHSUNDEEP
node.jsexpress
PixelSlave
PixelSlave

Reputation: 68

How can I stream multiple remote images to a zip file and stream that to browser with ExpressJS?

I've got a small web app built in ExpressJs that allows people in our company to browse product information. A recent feature request requires that users be able to download batches of images (potentially hundreds at a time). These are stored on another server.

Ideally I think I need to to stream the batch of files to a zip file and stream that to the end user's browser as a download. All preferably without having to store the files on the server. The idea being that I want to reduce load on the server as much as possible.

Is it possible to do this or do I need to look at another approach? I've been experimenting with the 'request' module for the initial download.

If anyone can point me in the right direction or recommend any NPM modules that might help it would be very much appreciated.

Thanks.

Upvotes: 5

Views: 2283

Answers (1)

robertklep
robertklep

Reputation: 203519

One useful module for this is archiver, but I'm sure there are others as well.

Here's an example program that shows:

  • how to retrieve a list of URL's (I'm using async to handle the requests, and also to limit the # of concurrent HTTP requests to 3);
  • how to add the responses for those URL's to a ZIP file;
  • to stream the final ZIP file somewhere (in this case to stdout, but in case of Express you can pipe to the response object).

Example:

var async    = require('async');
var request  = require('request');
var archiver = require('archiver');

function zipURLs(urls, outStream) {
  var zipArchive = archiver.create('zip');

  async.eachLimit(urls, 3, function(url, done) {
    var stream = request.get(url);

    stream.on('error', function(err) {
      return done(err);
    }).on('end', function() {
      return done();
    });

    // Use the last part of the URL as a filename within the ZIP archive.
    zipArchive.append(stream, { name : url.replace(/^.*\//, '') });
  }, function(err) {
    if (err) throw err;
    zipArchive.finalize().pipe(outStream);
  });
}

zipURLs([
  'http://example.com/image1.jpg',
  'http://example.com/image2.jpg',
  ...
], process.stdout);

Do note that although this doesn't require the image files to be locally stored, it does build the ZIP file entirely in memory. Perhaps there are other ZIP modules that would allow you to work around that, although (AFAIK) the ZIP file format isn't really great in terms of streaming, as it depends on metadata being appended to the end of the file.

Upvotes: 4

Related Questions