CODEWITHSUNDEEP
node.jsasynchronouspromisestream
Bluefire
Bluefire

Reputation: 14109

Provide progress on data processing

I've written a Node package that performs some intensive data processing. I'm able to watch progress via console.log, but I'd like to provide the end user with some way of monitoring it in an event-driven way. Basically, it would be like returning a promise, except instead of one then event it would have an event for each "step", and then finally resolve with the data.

Unfortunately, I don't know enough about Node streams (which I'm guessing is the thing I need) to do this. Can I get a few pointers? How can I create a stream which is updated every time there is, say, 1% more progress, and then finally gives the computed data?

EDIT: As an example, consider this loop:

for(let i = 0; i < N; i++) {
    intensiveFunction();
    console.log(`${i} of ${N} completed`);
}

What I want to do instead is

for(let i = 0; i < N; i++) {
    intensiveFunction();
    // send user a signal that i/N of the task has been completed
}

Upvotes: 1

Views: 44

Answers (1)

Marcos Casagrande
Marcos Casagrande

Reputation: 40404

You don't need to use streams, You can use an EventEmitter, and emit the current progress, or any event you may want.

my-package.js

const EventEmitter = require('events');

// Create a class that extends from EventEmitter
// And now you can emit events when something happens, e.g., progress update
class MyPackage extends EventEmitter {

    async intensiveFunction() {
        // Something
    }

    async process() {

        for(let i = 0; i < N; i++) {
            await this.intensiveFunction();
            this.emit('step', i, N); // Or send progress in % or whatever you want
        }
    }
}


module.exports = MyPackage;

index.js

const MyPackage = require('my-package');

const package = new MyPackage();

package.on('step', (step, total) => console.log(`${step}/${total}`));
package.process();

You can either provide a full events API, or mix it with promises. Meaning process can resolve once is done, or you can emit an end event, or do both.

async process() {

   for(let i = 0; i < N; i++) {
      await this.intensiveFunction();
      this.emit('step', i, N); // Or send progress in % or whatever you want
   }

   // Emit the end event
   this.emit('end', result);       

   // And resolve the promise
   return result; // Whatever result is
}

Upvotes: 2

Related Questions