CODEWITHSUNDEEP
javascriptnode.js
Qrow Saki
Qrow Saki

Reputation: 1072

How to make fs writeFile faster for an image?

How can I make this faster? I am using node and fs library. This is the endpoint of a POST request that sends an fs.Readable stream of a jpg image. The code is starting to hang at just around the 800 kB mark. Eventually, I want to be able to save a video. Ideally, I want it to take less than ten seconds.

  var bufs = [];
  readStream.on('data', function(d){ bufs.push(d); });
  readStream.on('end', function(){
  var buf = Buffer.concat(bufs);
  console.log(buf);
  fs.writeFile("test.jpg", buf, ()=>{console.log('hi')});
  });

Upvotes: 1

Views: 926

Answers (1)

jfriend00
jfriend00

Reputation: 708206

A simpler way that also uses less memory and parallelizes the writes (writes chunks as they arrive) and should be a bit faster would be to just pipe the stream directly to the output file.

readStream.pipe(fs.createWriteStream("test.jpg"));
readStream.on('close', () => {
    console.log("all done now");
});

Or, with better error handling that would catch errors on both streams and tell you when everything was done:

const { pipeline } = require('stream');

pipeline(readStream, fs.createWriteStream("test.jpg"), err => {
    if (err) {
        console.log(err);
    } else {
        console.log("all done");
    }
});

Or, using promises and pipeline():

const pipeline = util.promisify(require('stream').pipeline);

pipeline(readStream, fs.createWriteStream("test.jpg")).then(() => {
   console.log("all done");
}).catch(err => {
   console.log(err);
});

Upvotes: 2

Related Questions