Scaling Nodejs server to multiple systems?

Question

I want to build the chat servers in nodejs using express.I have used cluster module for scaling the server among the multiple cores but how do I scale up to different system?

Answer 1

Since Node.js does not support shared memory, distributing Node.js processes across multiple machines provides for the same experience as using a cluster to distribute processes across multiple cores—if your application can run as multiple independent processes within a single system, then it can also be distributed to run as multiple independent processes across multiple systems.

Great, so that's one less thing to worry about! Now, there are many infrastructure solutions out there that would abstract running clusters on several systems, but your application is otherwise oblivious to any one you might pick.

What will concern you, though, within the realm of your application and any single process, is discovering external services, communicating to processes across the infrastructure and communicating with processes within a cluster. Again, there are many solutions out there that will curtail to any particular requirement your application needs to address.

So far, the Node.js community has favored simple approaches that are highly specialized for solving a particular problem and then get out of your way. For instance:

• Web socket clients and servers: low latency within a cluster; also works well across the whole network when you can just send some data and get on with your life, but it will bring things down to a crawl if you need to synchronize processes, such as sending some data, waiting and idling until a result eventually comes back
• Redis: clusters are easy to set up, instances handle discovery on their own, enough atomic operations to provide a solid approach to sharing data among different instances and the pub-sub support provides for low-latency IPC
• ZMQ: lauded for it's intelligent, highly-available connections, you can devise any messaging protocol with a few dozen lines of code that the next human being maintaining your application will be able to reason about
• etcd: distributed, consistent key-value store; low infrastructural overhead, allows for implementing straightforward service discovery on top that will integrate nicely with every infrastructure solution out there
• Consul: based on serf, like etcd, but strongly opinionated, provides for service discovery on steroids with many additional niceties; if you like managing things on your own and have the time to invest up front, I would heartily recommend further investigation

While this certainly doesn't cover all the options available, it should be enough to get you going in the right direction. With just these simple building blocks that are ridiculously easy to reason about, you should be able to distribute your application across several systems, running across several machines in several datacenters.

Answer 2

If you're using a process manager like PM2, it will take care of starting up your node app on different or same machines but to handle multiple machines you should look into Puppet, Chef or Ansible to scale. If you're on AWS, EC2 can be set to do it automatically.

Answer 3

Actually there can be multiple answers to this question because the answer depends on how you want to communicate amongst nodes, how you want to assign tasks to nodes and how you manage failures.

You may want to research on how other cluster managers work and then try to design something similar in your application.

Few Approaches:

1) Use a load balancer in the front and distribute load amongst the machine. This I think can be the simplest approach.

2) Use a messaging system like RabbitMQ/ActiveMQ (or any other AMQP) system for inter node communication and let there be a pool of master nodes who assigns tasks to specific nodes and communicates to node via AMQP Protocol.