Is there a way to share memory among workers/threads/something in Node.JS?

Question

I have a Node app which accesses a static, large (>100M), complex, in-memory data structure, accepts queries, and then serves out little slices of that data to the client over HTTP.

Most queries can be answered in tenths of a second. Hurray for Node!

But, for certain queries, searching this data structure takes a few seconds. This sucks because everyone else has to wait.

To serve more clients efficiently, I would like to use some sort of parallelism.

But, because this data structure is so large, I would like to share it among the workers or threads or what have you, so I don't burn hundreds of megabytes. This would be perfectly safe, because the data structure is not going to be written to. A typical 'fork()' in any other language would do it.

However, as far as I can tell, all the standard ways of doing parallelism in Node explicitly make this impossible. For safety, they don't want you to share anything.

But is there a way?

Background:

It is impractical to put this data structure in a database, or use memcached, or anything like that.

WebWorker API libraries and similar only allow short serialized messages to be passed in and out of the workers.

Node's Cluster uses a call named 'fork', but it is not really a fork of the existing process, it is spawning a new one. So once again, no shared memory.

Probably the really correct answer would be to use filesystem-like access to shared memory, aka tmpfs, or mmap. There are some node libraries that make mount() and mmap() available for exactly something like this. Unfortunately then one has to implement complex data structure access on top of synchronous seeks and reads. My application uses arrays of arrays of dicts and so on. It would be nice to not have to reimplement all that.

Answer 1

Most web applications spend the majority of their life waiting for network buffers and database reads. Node.js is designed to excel at this io bound work. If your work is truly bound by the CPU, you might be served better by another platform.

With that out of the way...

1. Use process.nextTick (perhaps even nested blocks) to make sure that expensive CPU work is properly asynchronous and not allowed to block your thread. This will make sure one client making expensive requests doesn't negatively impact all the others.

2. Use node.js cluster to add a worker process for each CPU in the system. Worker processes can all bind to a single HTTP port and use Memcached or Redis to share memory state. Workers also have a messaging API that can be used to keep an in-process memory cache synchronized, however it has some consistency limitations.

Answer 2

I tried write a C/C++ binding of shared memory access from nodejs. https://github.com/supipd/node-shm

Still work in progress (but working for me), maybe usefull, if bug or suggestion, inform me.

Answer 3

Actually Node does support spawning processes. I'm not sure how close Node's fork is to real fork, but you can try it:

http://nodejs.org/api/child_process.html#child_process_child_process_fork_modulepath_args_options

By the way: it is not true that Node is unsuited for that. It is as suited as any other language/web server. You can always fire multiple instances of your server on different ports and put a proxy in front.

If you need more memory - add more memory. :) It is as simple as that. Also you should think about putting all of that data on a dedicated in-memory database like Redis or Memcached ( or even Couchbase if you need complex queries ). You won't have to worry about duplicating that data any more.

Answer 4

building with waf is old style (node 0.6 and below), new build is with gyp.

You should look at node cluster (http://nodejs.org/api/cluster.html). Not clear this is going to help you without having more details, but this runs multiple node processes on the same machine using fork.