CODEWITHSUNDEEP
node.jsnode-modulesstreamwriter
Shreyasi Chakraborty
Shreyasi Chakraborty

Reputation: 11

Write huge amounts of data using stream in Node.js

Write a Node.js code to write a string "Hello world" 10^5 times in a file using stream writer in Node.js.

I have tried using loops but there is out of memory error and the system hangs.

Any better method?

var fs = require('fs');
var wstream = fs.createWriteStream('D:\\myOutput.txt');
for(var i=0;i<10^5;i++){
wstream.write('Hello world!\n');
}
wstream.end();

Upvotes: 1

Views: 2181

Answers (1)

Ankur
Ankur

Reputation: 205

Node.js is a single-threaded execution environment, when you write data in a loop the node.js will not get a chance to execute other tasks, as you can see from the documentation(https://nodejs.org/docs/latest-v12.x/api/stream.html#stream_writable_write_chunk_encoding_callback) nodejs does not guarantee that all the data will be written once you call write, it can be buffered, if you want node to stay responsive, you have to free up the event loop (https://nodejs.org/en/docs/guides/event-loop-timers-and-nexttick/#what-is-the-event-loop) for other tasks, an example where you can do both is as follows. hope this clears things up

var fs = require('fs');
var wstream = fs.createWriteStream('myOutput.txt');
wstream.on('error', function (err) {
    console.log(err);
});



async function write()
{
for(var i=0;i<10^5;i++){
await writeStream(wstream, 'Hello world!\n');
}
wstream.end();
}

async function writeStream (stream, data) {
  if (!stream.write(data)) {
   return new Promise((resolv, reject) => {
    stream.once('drain', () => {resolv();});
   });
  } else {
    return Promise.resolve();
  }
}

write();

Upvotes: 1

Related Questions