CODEWITHSUNDEEP
javascriptperformancenode.jsscaling
Peter Corbett
Peter Corbett

Reputation: 21

Scaling a Node.js application to 10s of 1000s of simultaneous connections

We did some work on an app that lets people fire baseballs over the internet.

It lives entirely within Amazon's AWS ecosystem, and we're building off that for a new project. The stack includes:

-Dedicated MongoDB and Redis servers -three different groups of nodejs servers -additionally, we're making use of Amazon's API for server configuration and autoscaling

The issue we're facing is that we haven't been able to simulate more than about 15000 concurrent users (websocket connections) per instance. We should be getting considerably more; we think 10s of thousands. The server CPU usage is only at 40%.

Any thoughts on how to scale a node.js app to enable it to have many more simultaneous connections to a single server?

Upvotes: 2

Views: 2323

Answers (4)

Aleksandr Guidrevitch
Aleksandr Guidrevitch

Reputation: 459

  1. Is Redis running replicated ? Problem can be with Redis - it is single-threaded. Quote from their docs: Redis uses a mostly single threaded design. This means that a single process serves all the client requests, using a technique called multiplexing. This means that Redis can serve a single request in every given moment, so all the requests are served sequentially. So the processes can be in Redis queue waiting for their turn

  2. Are locks used at mongodb side ? I've observed this kind of performance issues with code using mysql locks: processes are waiting for the lock.

Upvotes: 0

knshn
knshn

Reputation: 3461

Every tcp connection has a file descriptor open in file operating system. It is important to set the limit to a number above what you need.

For example, in ubuntu you can see this limit by commands:

$ulimit -a
$ulimit -n

To set this limit permanently in Ubuntu, you need to change the file /etc/security/limits.conf and add these lines with the number you want:

* soft nofile 100000
* hard nofile 100000

And then restart:

$sudo reboot

Upvotes: 1

kkubasik
kkubasik

Reputation: 3734

I would assume your starting to hit some of the kernel's default limits on the tcp stack/file descriptors. Have you tried any system-level optimizations yet? If so, which?

Upvotes: 0

PP.
PP.

Reputation: 10864

A websocket is a TCP connection, no? And how long do your customers keep your connections open for?

A server will have a limit on the number of open TCP connections you can have. Your operating system will also have a limit on the number of open file handles a process may have at any one time.

So:

  • what is the TCP open socket limit on your server, and
  • what is the open file handle limit on your server

?

Upvotes: 0

Related Questions