How to scale node.js / socket.io server?

Question

I am currently running a node.js app and am about to introduce socket.io to allow real time updates (chat, in-app notifications, ...). At the moment, I am running the smallest available setup from DigitalOcean (1 vCPU, 1 GB RAM) for my node.js server. I stress-tested the node.js app connecting to socket.io using Artillery:

config:
  target: "https://my.server.com"
  socketio:
    - transports: ["websocket"] // optional, same results if I remove this
  phases:
    - duration: 600
      arrivalRate: 20
scenarios:
- name: "A user that just connects"
  weight: 90
  engine: "socketio"
  flow:
    - get:
        url: "/"
    - think: 600

It can handle a couple hundred concurrent connections. After that, I start getting the following errors:

Errors:
  ECONNRESET: 1
  Error: xhr poll error: 12

When I resize my DigitalOcean droplet to 8 vCPU's and 32 GB RAM, I can get upwards of 1700 concurrent connections. No matter how much more I resize, it always sticks around that number.

My first question: is this normal behavior? Is there any way to increase this number per droplet, so I can have more concurrent connections on a single node instance? Here is my configuration:

sysctl -p

fs.file-max = 2097152
vm.swappiness = 10
vm.dirty_ratio = 60
vm.dirty_background_ratio = 2
net.ipv4.tcp_synack_retries = 2
net.ipv4.ip_local_port_range = 2000 65535
net.ipv4.tcp_rfc1337 = 1
net.ipv4.tcp_fin_timeout = 15
net.ipv4.tcp_keepalive_time = 300
net.ipv4.tcp_keepalive_probes = 5
net.ipv4.tcp_keepalive_intvl = 15
net.core.rmem_default = 31457280
net.core.rmem_max = 12582912
net.core.wmem_default = 31457280
net.core.wmem_max = 12582912
net.core.somaxconn = 4096
net.core.netdev_max_backlog = 65536
net.core.optmem_max = 25165824
net.ipv4.tcp_mem = 65536 131072 262144
net.ipv4.udp_mem = 65536 131072 262144
net.ipv4.tcp_rmem = 8192 87380 16777216
net.ipv4.udp_rmem_min = 16384
net.ipv4.tcp_wmem = 8192 65536 16777216
net.ipv4.udp_wmem_min = 16384
net.ipv4.tcp_max_tw_buckets = 1440000
net.ipv4.tcp_tw_reuse = 1

ulimit

core file size          (blocks, -c) 0
data seg size           (kbytes, -d) unlimited
scheduling priority             (-e) 0
file size               (blocks, -f) unlimited
pending signals                 (-i) 3838
max locked memory       (kbytes, -l) 16384
max memory size         (kbytes, -m) unlimited
open files                      (-n) 65535
pipe size            (512 bytes, -p) 8
POSIX message queues     (bytes, -q) 819200
real-time priority              (-r) 0
stack size              (kbytes, -s) 8192
cpu time               (seconds, -t) unlimited
max user processes              (-u) 10000000
virtual memory          (kbytes, -v) unlimited
file locks                      (-x) unlimited

nginx.conf

user www-data;
worker_processes auto;
worker_rlimit_nofile 1000000;
pid /run/nginx.pid;
include /etc/nginx/modules-enabled/*.conf;

events {
        multi_accept on;
        use epoll;
        worker_connections 1000000;
}

http {

        ##
        # Basic Settings
        ##

        client_max_body_size 50M;
        sendfile on;
        tcp_nopush on;
        tcp_nodelay on;
        keepalive_timeout 120;
        keepalive_requests 10000;
        types_hash_max_size 2048;
        # server_tokens off;

        # server_names_hash_bucket_size 64;
        # server_name_in_redirect off;

        include /etc/nginx/mime.types;
        default_type application/octet-stream;

        ##
        # SSL Settings
        ##

        ssl_protocols TLSv1 TLSv1.1 TLSv1.2; # Dropping SSLv3, ref: POODLE
        ssl_prefer_server_ciphers on;

        ##
        # Logging Settings
        ##

        access_log /var/log/nginx/access.log;
        error_log /var/log/nginx/error.log;
        ##
        # Gzip Settings
        ##

        gzip on;

        # gzip_vary on;
        # gzip_proxied any;
        # gzip_comp_level 6;
        # gzip_buffers 16 8k;
        # gzip_http_version 1.1;
        # gzip_types text/plain text/css application/json application/javascript text/xml application/xml application/xml+rss text/javascript;

        ##
        # Virtual Host Configs
        ##

        include /etc/nginx/conf.d/*.conf;
        include /etc/nginx/sites-enabled/*;
}

Another question: I am thinking about scaling horizontally and spinning up more droplets. Let's say 4 droplets to proxy all connections to. How would this be set up in practice? I would use Redis to emit through socket.io to all connected clients. Do I use 4 droplets with the same configuration? Do I run the same stuff on all 4 of them? For instance, should I upload the same server.js app on all 4 droplets? Any advice is welcome.

How to scale node.js / socket.io server?

Answers (1)

Related Questions