CODEWITHSUNDEEP
javascriptnode.jsstream
Bartvds
Bartvds

Reputation: 3500

What is optimal way to pipe one stream to two streams in node.js

I'm using a ReadableStream from request.js to pipe() a resource from http to a file system WritableStream.

How do I modify this to be able to process the content in memory? (for streamed parsing or whatever)

Ideally a full 'dupe' into two real streams and not just a callback 'tap' (but I'll take any advice).

Upvotes: 0

Views: 618

Answers (2)

Bartvds
Bartvds

Reputation: 3500

Ha, I had some lucky timing: the answer to my problem was announced today:

highlandjs, new stream util library by @caolan (the guy from async).

This is what I needed: http://highlandjs.org/#fork

It has the backpressure feature, lazy evaluation etc.

Upvotes: 1

Shaun Xu
Shaun Xu

Reputation: 4656

I would like to firstly pipe the content from http.req into memory (Buffer) through a module named memory-stream. Then I create a readable from Buffer and pipe into file system. At the same time you can create multiple readable from this Buffer and pipe it to any writable.

In one of my projects I need to process thumbnail files in many sizes once I received an image uploaded from browser and I was using the approach above to implement, save the original image into Buffer and create readable steams for each size, pipe with imagemagick and then pipe file system writable stream to save in parallel.

Upvotes: 0

Related Questions