CODEWITHSUNDEEP
node.js
Mike Me
Mike Me

Reputation: 2392

Cannot append data to a file using createWriteStream() in Node

I cannot append data from multiple Read Streams using createWriteStreams() in Node. It just creates a new file after each iteration.

chunkNo = 4;

for (i = 0; i < chunkNo; i++) { 
    const response = await this.downloadChunk(downloadUrl, i); // response.data is a ReadStream
    await new Promise<void>((resolve, reject) => {
          const stream = response.data.pipe(fs.createWriteStream('C:\\big_file.mkv'), { flags: 'a'});
          stream.on('finish', () => {
            resolve();
          });
        });
}

What am I missing? Even though the append flag is there. Thank you in advance

Upvotes: 0

Views: 1671

Answers (1)

Delapouite
Delapouite

Reputation: 10177

To better consolidate your intuition about what the a (append) flag purpose is supposed to indicate, first have a look at this answer: Difference between modes a, a+, w, w+, and r+ in built-in open function?

It details the different modes in which a file can be open. As many other languages, JavaScript reuses the same principle that in C.

From this man page, the relevant section in our case is:

 ``a''   Open for writing.  The file is created if it does not exist.  The
         stream is positioned at the end of the file.  Subsequent writes
         to the file will always end up at the then current end of file,
         irrespective of any intervening fseek(3) or similar.

So in your code example, a new file would indeed be created at each iteration of the loop. Therefore, what you can do is move the fs.createWriteStream('C:\\big_file.mkv', { flags: 'a'}) outside of the for loop, assigned a name to this readable stream and use it the pipe(), something along the way of :

chunkNo = 4;
const dest = fs.createWriteStream('C:\\big_file.mkv', { flags: 'a'});

for (i = 0; i < chunkNo; i++) { 
    const response = await this.downloadChunk(downloadUrl, i); // response.data is a ReadStream
    await new Promise<void>((resolve, reject) => {
          const stream = response.data.pipe(dest);
          stream.on('finish', () => {
            resolve();
          });
        });
}

Upvotes: 1

Related Questions