CODEWITHSUNDEEP
amazon-web-servicesbotoboto3amazon-sqs
J L
J L

Reputation: 319

How to long poll amazon sqs service using boto?

I have a sqs queue on amazon that have several consumer polling it.

Recently I found out that the numberofEmptyReceives is in the 10 millions, this means that I am being charge on those request.

The following is how I am consuming message

while True:
    for message in queue.receive_messages(AttributeNames=['All'], MaxNumberOfMessages=10):

I know there is WaitTimeSeconds option but the documentation seems to suggest it does not do long poll

The duration (in seconds) for which the call waits for a message to arrive in the queue before returning. If a message is available, the call returns sooner than WaitTimeSeconds . If no messages are available and the wait time expires, the call returns successfully with an empty list of messages.

Specifically

If a message is available, the call returns sooner than WaitTimeSeconds .

From the above sentence it seems to suggest boto3 still calls sqs to check if there is message or not.

What is the proper way to long poll using boto3 to avoid getting charge for request?

Is it as simple as setting thread.sleep ?

I also can't find the source code on github

Upvotes: 5

Views: 8086

Answers (1)

craigcaulfield
craigcaulfield

Reputation: 3538

Just send a WaitTimeSeconds parameter (up to a maximum of 20 seconds) in your receive_message call. From the AWS documentation:

# Long poll for message on provided SQS queue
response = sqs.receive_message(
    QueueUrl=queue_url,
    AttributeNames=[
        'SentTimestamp'
    ],
    MaxNumberOfMessages=1,
    MessageAttributeNames=[
        'All'
    ],
    WaitTimeSeconds=20
)

If there is no message in the queue, then the call will wait up to WaitTimeSeconds for a message to appear. If a message appears before the time expires, the call will return that message right away.

See Enabling Long Polling in Amazon SQS for more details.

You'd still be charged for the long-poll request. The cost saving is in performing one billable SQS action and waiting up to the long-poll timeout, where otherwise you might have had to perform two or three or more billable actions over the same period.

Upvotes: 7

Related Questions