CODEWITHSUNDEEP
javascriptnode.jsstream
hba
hba

Reputation: 7790

Node.JS - How to stop a piped stream conditionally

I'm a newbee to java-script & nodejs...

I have a file processing-stream which is composed of multiple streams pipped together. Works great...I'd like to enhance this by conditionally stopping the stream from processing and reading when a threshold has been reached.

    inputStream.
    pipe(unzip()).
    pipe(latin1Decoder).
    pipe(
         combine(
                 through(function(obj){
                      //part-1 -
                      this.push(obj);
                      }),
                 through(function(obj){
                      //part-2-
                      this.push(obj);
                      })
         ));

In part-1 if I do this.push(null) then the combiner will ignore incoming input.

However, I can't stop reading the file. That's annoying because the file is pretty huge.

What is the best way for me to access the input stream from inside the pipe-line to close it?

Upvotes: 1

Views: 3502

Answers (1)

hba
hba

Reputation: 7790

OK, here is how I ended up solving it:

    var cnt = 0;
    var processingStream = combine(unzip(), latin1Decoder(), part1(), part2());
    inputStream.pipe(processingStream);
    inputStream.on('data', function(x) {
      var lineCnt = x.toString().split(/\r\n|\r|\n/).length;
      cnt += lineCnt;
      if (cnt > 5) {
        inputStream.close();
        inputStream.unpipe(processingStream);
        processingStream.end();
      }          
    });

Probably not the most efficient way, but it meets my requirements.

Please note that inputstream reads in blocks so multiple lines are read at once.

Upvotes: 4

Related Questions