CODEWITHSUNDEEP
pythontornadobotobotocore
estherika
estherika

Reputation: 7

max_clients limit reached error on tornado-botocore server

I've developed a Tornado server using the tornado-botocore package for interacting with Amazon SQS service. When I'm trying to load test the server i get the following log: [simple_httpclient:137:fetch_impl] max_clients limit reached, request queued. 10 active, 89 queued requests. I assume it's from the ASyncHTTPClient used by the botocore package. I've tried to set the max_clients to an higher number but with no success:

    def _connect(self, operation):
    sqs_connection = Botocore(
        service='sqs', operation=operation,
        region_name=options.aws_sqs_region_name,
        session=session)
    sqs_connection.http_client.configure(None, defaults=dict(max_clients=5000))

what am i doing wrong?

Thanks.

Upvotes: 1

Views: 1264

Answers (1)

Ben Darnell
Ben Darnell

Reputation: 22134

configure is a class method that must be called before an AsyncHTTPClient is created: tornado.httpclient.AsyncHTTPClient.configure(None, max_clients=100).

The log message does not indicate an error (it's logged at debug level). It's up to you whether it's appropriate for this service to respond to load by using more connections or queuing things up. 5000 connections for a single application process seems like too much to me.

Upvotes: 3

Related Questions