CODEWITHSUNDEEP
javascriptnode.jsshared-memory
Kosmas Papadatos
Kosmas Papadatos

Reputation: 1317

NodeJs shared objects between processes

I am creating a caching module for Nodejs that needs to utilize multiple CPUs using sub processes.

possible setup

I would like to store data in an index in the master process or, preferably in a subprocess like so:

var index = { key1: 21, key2: 22, key3: 55 }

and another process should be able to search in that index as efficiently as:

if('key2' in index) // do stuff

I assume using IPC would be significantly slower than achieving shared objects. Is this even possible? Thanks.

Upvotes: 7

Views: 5345

Answers (2)

Allen Luce
Allen Luce

Reputation: 8389

You might want to try mmap-object. It coordinates shared memory based on a memory-mapped file which means the data is persistent as well as shared among processes. I wrote it so to avoid both the performance and management overhead of redis or a similar solution.

For your application the read-write version may be what you want.

Upvotes: 7

Angelos Kyriakopoulos
Angelos Kyriakopoulos

Reputation: 444

I would use redis in your case. There are ways to configure redis on how frequently it will store its memory data on disk based on how many entries are saved or just time (http://redis.io/topics/persistence).

Another option might be to send requests between instances to save that memory data only on master.

When a non master instance wants to save or load data it will request the master.

Here is some pseudocode:

var SharedMemory = {
  storeObject: function(args, done) {
    if (IAmMaster()) {
      storeObjectToMemory(args);
      done();
    } else {
      sendRequestToMasterForSave(args, done);
    }
  },
  getObject: function(args, done) {
    if (IAmMaster()) {
      done(getObjectFromMemory(args));
    } else {
      getResponseFromMasterForLoad(args, done);
    }
  }
}

However, this is probably gonna be a painful process

Upvotes: 2

Related Questions