CODEWITHSUNDEEP
node.jsapihttpposthapi.js
jcollum
jcollum

Reputation: 46599

What should I limit my POST per second rate to?

I'm building out an API using Hapi.js. Some of my code is pushing small amounts of data to the API. The issue seems to be that the pusher code is swamping the API and I'm getting ECONNRESET errors -- which means messages are getting lost. I'm planning on installing a rate-limiter in the pusher code, probably node-rate-limiter (link).

The question is, what should I set that limit to? I want to max out performance for this app, so I could easily be attempting to send in thousands of messages per hour. The data just gets dumped into redis, so I doubt the code in the API will be an issue but I still need to get an idea of what kind of message rate Hapi is comfortable with. Do I need to just start with something reasonable and see how it goes? Maybe 1 message per 10 milliseconds? 

Hapi = require('hapi');
server = new (Hapi.Server);
server.connection(port: config.port, routes: {
  cors: {
    origin: ['*']
  }
});

server.route({method: 'POST', path: '/update/{id}', ...})

Upvotes: 1

Views: 704

Answers (1)

jfriend00
jfriend00

Reputation: 707786

There is no generic answer to how many requests per second you can process. It depends upon many things in your configuration and code such as:

  1. Type and performance of server hardware
  2. The amount of CPU time an average request uses
  3. Whether your requests are CPU or disk bound. If disk bounded, then it depends a lot on your database and disk performance.
  4. Whether you implement clustering to use multiple cores (if CPU bound)
  5. Whether you're on shared infrastructure or not
  6. The max number of incoming connections your server is configured for

So, there is no absolute answer here that works for everyone. If you don't have some sort of design problem that is artificially limiting your concurrency, then the best way to discover what your server can actually handle is to build a test engine and test it. Find where and how it fails and either fix those issues to extend the scalability further or implement protections to avoid hitting that limit.

Note: When a public API makes rate limiting choices, it is typically done on a per-client basis and the limit is set to a value that seems to be a little above what a reasonable client would be doing. This is more to allow fair use of the server by many clients to that one single client does not consume too much of the overall resource. If issuing thousands of small requests from a single client is not considered "good practice" in using your API, then you can just pick a number that is much smaller than that for a per-client limit.

Note: You may also want to make it easier for clients by having your API let them upload multiple messages in one API request rather than lots of API requests.

Upvotes: 1

Related Questions