Reputation: 1270
Problem:
I have a python lambda that constantly receives data every second and puts it into DynamoDB. I noticed that after the first time DynamoDB takes a little more and the function times out, all the following calls also timeout and it never recovers. The way to bring the lambda back to normal is to redeploy it. When it starts timing out, it does not display any logs. It times out without executing any of the code.
Below is a picture of our console that represents the issue.
In order to reproduce the issue faster with this function I did the following:
Is there a way to resolve this issue without having to redeploy?
I have seen the same description of issue but with nodejs in this post: https://forums.aws.amazon.com/thread.jspa?threadID=234417.
I haven't seen any description related with the python lambda env
More information about the setup:
Lambda environments tested: python3.6 and python3.7
Tool to deploy lambda: serverless 1.57.0
serverless plugins used: serverless-python-requirements, serverless-wsgi
I am not using any VPC for the lambda
Thank you for the help,
Upvotes: 2
Views: 578
Reputation: 1270
Figured out the trigger for the bug. When the lambda function zip uploaded is too large, after the first time it times out, it never recovers!
My solution was to carefully strip out the unnecessary dependencies to make the package smaller.
I created a repository using a docker container for people to reproduce the issue more easily:
https://github.com/pedrohbtp/bug-aws-lambda-infinite-timeout
Thanks for the messages in the comments. I appreciate whoever takes time to try to help here in SO.
Upvotes: 1