CODEWITHSUNDEEP
amazon-web-servicesaws-lambdaamazon-cloudwatch
Chau Giang
Chau Giang

Reputation: 1554

Best solution to retry AWS lambda function when it got timeout

I have a serious question and I need your help. I can not find any solution on the Internet after spending a lot of time.

I made a bot to get data which is really heavy task because I need to setup a scraper and then it extract data from a webpage through many steps (login, logout, click, submit button, ...) and after got this result, it will post to an API to make a report.

I use Cloudwatch event to make my lambda function run in certain time every day.

The problem is although I set my lambda function in its max settings (3GB RAM, and 15 minutes timeout, the metrics is in Jan 2019), but sometime my lambda function failed when executing (maybe the scrape tasks take too lot of steps or maybe the webpage I tried to scrape is not stable) and it rarely failed, about only 5% I think.

But I want to know if there is any approach to deal with this situation, I want my lambda function can be auto retry when it fail without doing manual.

Upvotes: 4

Views: 6151

Answers (3)

Cleriston
Cleriston

Reputation: 770

Although the answer has already been given, in this article I explain the problem and share an example of step functions handling lambdas.

Lambdas with step function

Upvotes: 0

iTech
iTech

Reputation: 18430

This is a perfect example for a StepFunction you can have it scheduled by CloudWatch Event instead of the lambda.

The StepFunction can call your lambda and handle the retry logic on failure with configurable exponential back-off if needed.

Here is an example of a StepFunction

{
  "Comment": "Call lambda with retry",
  "StartAt": "Scraper",
  "States": {
    "Scraper": {
      "Type": "Task",
      "Resource": "<LAMBDA_ARN>",
      "Retry": [
        {
          "ErrorEquals": [
            "States.ALL"
          ],
          "IntervalSeconds": 20,
          "MaxAttempts": 5,
          "BackoffRate": 2
        }
      ],
      "End": true
    }
  }
}

Upvotes: 4

niekname
niekname

Reputation: 2616

Your lambda is already retried because a trigger from a cloudwatch event is async. (see the docs below) I would set up a DLQ for your lambda, and then reprocess from there.

From the docs (aws lambda event sources)

Error handling for a given event source depends on how Lambda is invoked. Amazon CloudWatch Events invokes your Lambda function asynchronously. For more information on how errors are retried, see AWS Lambda Retry Behavior.

And: (aws lambda retry behaviour)

Asynchronous invocation – Asynchronous events are queued before being used to invoke the Lambda function. If AWS Lambda is unable to fully process the event, it will automatically retry the invocation twice, with delays between retries. If you have specified a Dead Letter Queue for your function, then the failed event is sent to the specified Amazon SQS queue or Amazon SNS topic. If you don't specify a Dead Letter Queue (DLQ), which is not required and is the default setting, then the event will be discarded. For more information, see AWS Lambda Function Dead Letter Queues.

Upvotes: 1

Related Questions