CODEWITHSUNDEEP
javascriptnode.js
Tero Tolonen
Tero Tolonen

Reputation: 4254

Overcome node.js memory limit with fork() ?

According to my knowledge most node.js versions have memory limit for each process around 1GB, based on this information:

Currently, by default v8 has a memory limit of 512MB on 32-bit systems, and 1.4GB on 64-bit systems. https://github.com/nodejs/node-v0.x-archive/wiki/FAQ

It is an old archive, things may have improved since. However, what I am interested about is if one is about to use

require('child_process').fork("newProcess.js")

Is it possible to overcome this limitation?

Thus creating a new V8 instance would create a new memory pool for JavaScript Objects.

The reason I am asking is that I could not find a reliable source exactly confirming this case, even though the documentation strongly suggests that this would be the case because it states:

These child Node.js processes are still whole new instances of V8.

Upvotes: 5

Views: 5997

Answers (3)

Sebastien
Sebastien

Reputation: 712

This is quite similar to Boriel's answer, but better (IMHO).

Passing the node option --max-old-space-size through the environment allows you to invoke ".js" files directly.

var env = Object.assign({ 'NODE_OPTIONS' : '--max-old-space-size=4096' }, process.env );
require('child_process').fork("newProcess.js", { env: env });

Please note that if you were already using some node options when running the original program, you would have to concat this option to the existing ones.

Upvotes: 0

Boriel
Boriel

Reputation: 151

If I understood your question, you want to fork() with a different memory limit (--max-old-space-size parameter). By default, the new nodeJS instance will inherit this from the parent.

However, you can bypass this limitation passing parameters to the forked() instance like this:

require('child_process').fork("newProcess.js", { 
        execArgv: ['--max-old-space-size=4096']
    });)

This will fork your process but with a 4Gb memory limit.

Upvotes: 6

eljefedelrodeodeljefe
eljefedelrodeodeljefe

Reputation: 6791

node --max-old-space-size=4096 should work.

AFAIK the limit is depending on V8's buffer size. fork() can of course handle bigger data as long as you don't hold all of it in a buffer.

Upvotes: 1

Related Questions