Reputation: 2222
I am testing the scalability of a websockets application I wrote using socket.io and Node.js. I want to stress test and benchmark the performance. What are some tools I can use?
So far I've looked into wsbench but I'd like to explore any alternatives.
Upvotes: 16
Views: 19817
Reputation: 1
You can try artillery, artillery supports WebSocket.
https://www.artillery.io/docs/reference/engines/websocket
Upvotes: 0
Reputation: 99
https://www.npmjs.org/package/thor
It a commandline utility and has similar output as Apache Bench
Thor: version: 1.0.0 God of Thunder, son of Odin and smasher of WebSockets! Thou shall: - Spawn 1 workers. - Create 2 concurrent/parallel connections. - Smash 100 connections with the mighty Mjölnir. The answers you seek shall be yours, once I claim what is mine. Connecting to ws://server:8080/examples/websocket/chat Opened 100 connections Online 439 milliseconds Time taken 441 milliseconds Connected 100 Disconnected 0 Failed 55 Total transferred 132.62kB Total received 71.85kB Durations (ms): min mean stddev median max Handshaking 1 5 3 4 20 Latency 0 0 1 0 3 Percentile (ms): 50% 66% 75% 80% 90% 95% 98% 98% 100% Handshaking 4 5 5 6 7 11 17 20 20 Latency 0 0 1 1 1 1 1 2 3
Upvotes: 5
Reputation: 3205
Spent the day looking into benchmarking node.js and socket.io, I tried a number of solutions, I found you can't use wsbench to benchmark socket.io because socket.io has its own protocol that needs to be addressed to make the connections.
After trying a few other solutions with no luck I ended up using socket.io-benchmark and I just hit 60,000 connections on a m1.large EC2 instance using only one core with room to spare. In my case I only wanted to measure the overhead of the socket.io connections (which required me to comment out some code in socket.io-benchmark) so I didn't have any messages sent between client and server.
It was interesting to see the clients took more CPU and Memory than the server. So make sure the client server you're benchmarking from is beefy, I used a EC2 c1.xlarge and had plenty of head room. And I broke the benchmark into 6 individual processes so it could scale across the multiple cores and avoid memory limits.
Be careful to set the <rampup in seconds>
option high enough on the clients, if you set it to low everything buffers and grinds to halt.
The only problem with socket.io-benchmark is I had to use node v0.4.12 to get it all working due to a dependency on v8-profiler which won't compile under newer versions of node (as of writing).
Upvotes: 12
Reputation: 16
For just benchmarking of how many concurrent connections can be set up I'd recommend wsbench. However it will not work for Socket.IO sites (because more complicated handshake). I was researching it and found a couple of tools that just set up connections to Socket.IO server. It's not enough for measuring real performance. What you need is to send and receive real messages that simulate real client of your web application. I've described some ideas on how to create your own benchmark here: Benchmarking for Node.JS / Socket.IO sites
Upvotes: 0
Reputation: 19943
There's a very simple Socket.IO client implementation for Node.js. It's not an actual benchmarking tool, but it does know about Socket.IO's protocol.
Using that, it's fairly easy to write a client that interacts with your application.
Upvotes: 4