CODEWITHSUNDEEP
node.jsmongodbnosql
Tug
Tug

Reputation: 189

Scalable push application with node.js

I'm thinking about writing a few web applications having almost the same requirements as a chat. And I would like them to be able to scale easily.

I have worked a bit with node.js and I understand how it can help design push applications but I have some difficulties when thinking about having them run on multiple servers.

Here are some design I can think of for a large scale chat app :

1 - Servers have state, they keep the connections opened and clients can have new messages pushed to them. In this scenario, we are limited by the physical memory of one server so we cannot scale linearly if we have too many users per room.

2 - Servers have no state, they request a distributed database to respond to clients requests. In this scenario, clients poll the servers. We could scale linearly but the throughput is decreased, the messages are not delivered instantly and polling has been shown as a bad practice when scaling.

3 - Mix of 1 and 2. Servers keep the connections of its clients opened and poll the distributed database. The application is more complex to write and we still use polling. Similar client's requests (clients of the same room) are just grouped into a single one done by the server. The code becomes unnecessary complicated and it does not scale in the situation where we have many rooms and a few users per room.

4 - Servers have no state and the database cluster uses event to notify every registered servers about new messages. This is the solution I would like to have but I haven't heard of any database which has this feature. (Some people are talking about this feature for mongodb here: https://jira.mongodb.org/browse/SERVER-124)

So Why is the 4th solution not used so much today?

How do people usually design their applications in this case?

Upvotes: 0

Views: 1291

Answers (2)

Sergio Tulentsev
Sergio Tulentsev

Reputation: 230336

Instead of triggers for case 4, you might want to hook into MongoDB replication.

Let's assume you have a replica set (you wouldn't run single mongod, would you?).

Every change is written to the oplog on the primary and then is replicated to secondaries.

You can efficiently pull new updates from the oplog, using tailable cursors. Note, this is still pull, not push.

Then your node.js will push these events to the clients.

Upvotes: 1

alessioalex
alessioalex

Reputation: 63663

Since you want a push application, you would probably use Socket.IO with RedisStore.

By using this combination, the data for all the connections is kept in Redis (in-memory database), so you can scale outside a process. Another use of Redis here is for pub-sub.

The idea is to trigger an event when something needs to be pushed, then sent a message to the browser using Socket.io. If you want to listen to database changes, perhaps it's better to use CouchDB with it's _changes feature.

Resources:

https://github.com/dshaw/talks/tree/master/2011-10-jsclub/sample-app
http://www.ranu.com.ar/2011/11/redisstore-and-rooms-with-socketio.html
How to reuse redis connection in socket.io?

Upvotes: 4

Related Questions