Reputation: 1072
How can I make this faster? I am using node and fs library. This is the endpoint of a POST request that sends an fs.Readable stream of a jpg image. The code is starting to hang at just around the 800 kB mark. Eventually, I want to be able to save a video. Ideally, I want it to take less than ten seconds.
var bufs = [];
readStream.on('data', function(d){ bufs.push(d); });
readStream.on('end', function(){
var buf = Buffer.concat(bufs);
console.log(buf);
fs.writeFile("test.jpg", buf, ()=>{console.log('hi')});
});
Upvotes: 1
Views: 926
Reputation: 708206
A simpler way that also uses less memory and parallelizes the writes (writes chunks as they arrive) and should be a bit faster would be to just pipe the stream directly to the output file.
readStream.pipe(fs.createWriteStream("test.jpg"));
readStream.on('close', () => {
console.log("all done now");
});
Or, with better error handling that would catch errors on both streams and tell you when everything was done:
const { pipeline } = require('stream');
pipeline(readStream, fs.createWriteStream("test.jpg"), err => {
if (err) {
console.log(err);
} else {
console.log("all done");
}
});
Or, using promises and pipeline()
:
const pipeline = util.promisify(require('stream').pipeline);
pipeline(readStream, fs.createWriteStream("test.jpg")).then(() => {
console.log("all done");
}).catch(err => {
console.log(err);
});
Upvotes: 2