CODEWITHSUNDEEP
javascriptnode.jsunderscore.jslodash
devconcept
devconcept

Reputation: 3685

Does lodash or underscore each method run in parallel in Node.js?

I know that in node everything runs in parallel, except your code. Read here and here.

I’m looking a possible scenario where I have a very large array in memory and I want to perform a small computation to each of its elements. The order in which this computation executes is not important.

In node all I/O is executed very efficiently because of the event loop but when you iterate through a collection there is no I/O involved and if this iteration takes too long you can block all incoming requests in that period of time.

This gist contains a nonBlockingForEach that Neilk wrote in his article Why you should use Node.js for CPU-bound tasks which makes me wonder if I write something like this

var my_very_large_array = [...];
my_very_large_array.forEach(function() { ... })
//or
_.each(my_very_large_array, function() { ... })

I will hit a performance bottleneck on my server (this libraries fallback to the native forEach if present)

From what I learned you there are a lot of libraries like async.js to do that but I always use lodash or underscore for those tasks in the browser.

I also tried bluebird.js but promisifying those methods didn't work as expected.

So my question is this. Is lodash or underscore a performance killer in a node.js environment when you iterate through a large collection using a forEach method?

Upvotes: 1

Views: 4217

Answers (3)

Martin Konecny
Martin Konecny

Reputation: 59651

There is a new standard called "Web Workers" which do allow background work to happen in a separate thread in the same process. This requires later versions of node.js, and a separate package installed from here:

From the wiki page:

The W3C and WHATWG envision web workers as long-running scripts that are not interrupted by user-interface scripts (scripts that respond to clicks or other user interactions). Keeping such workers from being interrupted by user activities should allow Web pages to remain responsive at the same time as they are running long tasks in the background.

The simplest use of workers is for performing a computationally expensive task without interrupting the user interface.

You can enable Web workers in node.js by installing the webworker-threads package

So my question is this. Is lodash or underscore a performance killer in a node.js environment when you iterate through a large collection using a forEach method?

Yes, lodash and underscore will both kill performance in a typical node.js setup when working with a large amount of data, as they will block the only thread available, making other tasks queued up in the event-loop suffer. However if you were to run these in a web-worker thread, then your main thread would be free to continue processing work as normal.

Upvotes: 3

Reactgular
Reactgular

Reputation: 54791

NodeJS is a single-thread single-process application. It does not matter if you process the entire array in a single forEach, or break it apart into multiple loops processed by the event queue. It's still going to take the same amount of work for that CPU core.

If you want to take advantage of multiple cores. You need to use the cluster module, and create multiple processes that work on different parts of the array.

There is no shared memory, or thread locking in NodeJS. So you will have to break apart the array into pieces for each process to work on.

https://nodejs.org/api/cluster.html

Upvotes: 0

Kyeotic
Kyeotic

Reputation: 19847

"Performance Killer" is a relative, and fairly loaded term.

To answer your direct question, does forEach in lodash or underscore run in parallel: no. It uses a standard, synchronous iteration.

Upvotes: 1

Related Questions