CODEWITHSUNDEEP
node.jsmultithreadingexpresschild-process
lifwanian
lifwanian

Reputation: 654

Node/Express: running specific CPU-instensive tasks in the background

I have a site that makes the standard data-bound calls, but then also have a few CPU-intensive tasks which are ran a few times per day, mainly by the admin.

These tasks include grabbing data from the db, running a few time-consuming different algorithms, then reuploading the data. What would be the best method for making these calls and having them run without blocking the event loop?

I definitely want to keep the calculations on the server so web workers wouldn't work here. Would a child process be enough here? Or should I have a separate thread running in the background handling all /api/admin calls?

Upvotes: 0

Views: 310

Answers (1)

Jerome WAGNER
Jerome WAGNER

Reputation: 22422

The basic answer to this scenario in Node.js land is to use the core cluster module - https://nodejs.org/docs/latest/api/cluster.html

It is an acceptable API to :

  • easily launch worker node.js instances on the same machine (each instance will have its own event loop)
  • keep a live communication channel for short messages between instances

this way, any work done in the child instance will not block your master event loop.

Upvotes: 1

Related Questions