CODEWITHSUNDEEP
goconcurrency
Matias Barrios
Matias Barrios

Reputation: 5054

Blockers for maximum number of http requests from a pod

I have a Go app which is deployed to two 8 core pods instances on Kubernetes. From it, I receive a list of ids than later on I use to retrieve some data from another service by sending each id to a POST endpoint. I am using a bounded concurrency pattern to have a maximum number of simulataneous goroutines (and therefore, of requests) to this external service.

I set the limit of concurrency as:

sem := make(chan struct{}, MAX_GO_ROUTINES)

With this setup I started playing around with the MAX_GO_ROUTINES number by increasing it. I usually receive around 20000 ids to check. So I have played around by setting MAX_GO_ROUTINES from anywhere between 100 and 20000.

One thing I notice is as I go higher and higher some requests start to fail with the message: connection reset from this external service.

So my questions are:

  1. What is the blocker in this case?
  2. What is the limit of concurrent HTTP POST requests a server with 8 cores and 4GB of ram can send? Is it a memory limit? or file descriptors limit?
  3. Is the error I am getting coming from my server or from the external one?

Upvotes: 2

Views: 486

Answers (1)

dm03514
dm03514

Reputation: 55972

What is the blocker in this case?

As the comment mentioned: HTTP "connection reset" generally means:

the connection was unexpectedly closed by the peer. The server appears to have dropped the connection on the unsuspecting HTTP client before sending back a response. This is most likely due to the high load.

Most webservers (like nginx) have a queue where they stage connections while they wait to be serviced. When the queue exceeds some limit the connections may be shed and "reset". So it's most likely this is your upstream service being saturated (i.e. your app sends more requests than it can service and overloads its queue)

What is the limit of concurrent HTTP POST requests a server with 8 cores and 4GB of ram can send? Is it a memory limit? or file descriptors limit?

All :) At some point your particularl workload will overload a logical limit (like file descriptors) or a "physical" limit like memory. Unfortunately the only way to truly understand which resource is going to be exhausted (and which constraints you hit up against) is to run tests and profile and benchmark your workload :(

Is the error I am getting coming from my server or from the external one?

HTTP Connection reset is most likely the external, it indicates the connection peer (the upstream service) reset the connection.

Upvotes: 1

Related Questions