CODEWITHSUNDEEP
amazon-web-servicesaws-lambdaamazon-sqslong-polling
naribo
naribo

Reputation: 710

AWS SQS Long Polling doesn't reduce empty receives

Currently, I have an AWS SQS as a trigger to my AWS Lambda function.

I would like to implement long polling to reduce costs since I've used up 70% of my monthly free tier, mostly from empty receives.

I tried setting up long polling by changing the queue attribute ReceiveMessageWaitTimeSeconds to 20 seconds:

SQS Details showing ReceiveMessageWaitTimeSeconds set to 20s

However, this didn't seem to reduce the number of empty receives, where the settings were changed on 11/19, between 2:00 - 3:00. Empty Receives graph shows the same trend even after setting long polling

According to the AWS Documentation, WaitTimeSeconds has priority over the queue attribute ReceiveMessageWaitTimeSeconds

Short polling occurs when the WaitTimeSeconds parameter of a ReceiveMessage request is set to 0 in one of two ways:

  • The ReceiveMessage call sets WaitTimeSeconds to 0.
  • The ReceiveMessage call doesn’t set WaitTimeSeconds, but the queue attribute ReceiveMessageWaitTimeSeconds is set to 0.

Note

For the WaitTimeSeconds parameter of the ReceiveMessage action, a value set between 1 and 20 has priority over any value set for the queue attribute ReceiveMessageWaitTimeSeconds.

Since AWS Lambda is receiving the SQS requests, I don't think WaitTimeSeconds can be configured.

Why doesn't my long polling configuration work in this situation? Am I misunderstanding something, or did I configure it wrong?

Thank you!

Upvotes: 18

Views: 6899

Answers (2)

Steve Kehlet
Steve Kehlet

Reputation: 6426

As of Jan 2023, you can specify a maximum concurrency on your event source mapping, and this can reduce these long-polling charges. You can set it as low as 2. This can potentially reduce your costs from 15 Empty Receives per minute (5 lambdas, each long-polling for 20sec, so each polling 3 times a minute) to 6 (2 lamdbas, each long-polling 3 times a minute). At $0.40 per 1 million SQS requests, we're not talking huge savings here, but for smaller projects, this could potentially prevent you from going over the 1M free tier limit. Hope this helps!

References:

Upvotes: 0

thomasmichaelwallace
thomasmichaelwallace

Reputation: 8454

Actually Long Polling is working in your situation.

5 lambdas * polling / 20 seconds * 3600 seconds in an hour = 900 receives/hour

What I think you've missed is the "5 minimum concurrent lambdas". This is implied in the Lambda Scaling Behaviour documentation, but is more helpfully and explicitly laid out in the "Additional Information" section of the announcement/deep-dive blog.

When an SQS event source mapping is initially created and enabled, or when messages first appear after a period with no traffic, then the Lambda service will begin polling the SQS queue using five parallel long-polling connections. The Lambda service monitors the number of inflight messages, and when it detects that this number is trending up, it will increase the polling frequency by 20 ReceiveMessage requests per minute and the function concurrency by 60 calls per minute.

Upvotes: 19

Related Questions