CODEWITHSUNDEEP
node.js
Adrian E
Adrian E

Reputation: 2093

Writing long strings to file (node js)

I have a string which is 169 million chars long, which I need to write to a file and then read from another process.

I have read about WriteStream and ReadStream, but how do I write the string to a file when it has no method 'pipe'?

Upvotes: 4

Views: 6192

Answers (3)

greuze
greuze

Reputation: 4398

Create a write stream is a good idea. You can use it like this:

var fs = require('fs');
var wstream = fs.createWriteStream('myOutput.txt');
wstream.write('Hello world!\n');
wstream.write('Another line\n');
wstream.end();

You can call to write as many time as you need, with parts of that 16 million chars string. Once you have finished writing the file, you can create a read stream to read chunks of the file.

However, 16 million chars are not that much, I would say you could read and write it at once and keep in memory the whole file.

Update: As requested in comment, I update with an example to pipe the stream to zip on-the-fly:

var zlib = require('zlib');
var gzip = zlib.createGzip();
var fs = require('fs');
var out = fs.createWriteStream('input.txt.gz');

gzip.pipe(out);

gzip.write('Hello world!\n');
gzip.write('Another line\n');

gzip.end();

This will create a gz file, and inside, only one file with same name (without the .gz at the end).

Upvotes: 13

Tirtha
Tirtha

Reputation: 872

This might solve your problem 

var fs = require('fs');
var request = require('request');

var stream = request('http://i.imgur.com/dmetFjf.jpg');
var writeStream = fs.createWriteStream('./testimg.jpg');

stream.pipe(writeStream);

Follow the link for more details http://neethack.com/2013/12/understand-node-stream-what-i-learned-when-fixing-aws-sdk-bug/

Upvotes: 3

Kris Molinari
Kris Molinari

Reputation: 503

If you're looking to write what's called a blocking process, eg something that will prevent you from doing something else, approaching that process asynchronously is the best solution (and why node.js is good at solving these types of problems). With that said, avoid methods that have fs.*Sync as that will be a synchronous method. fs.writeFile is what I believe you're looking for. Read the Docs

Upvotes: 1

Related Questions