CODEWITHSUNDEEP
python-3.xamazon-web-servicesamazon-s3
Steve Porter
Steve Porter

Reputation: 19

AWS Lambda function times out when reading bucket file

Last two lines of code below are the issue. I have line of sight to the csv file in the bucket as can be seen in the printout below, the file in the bucket is an object that is returned with key/value conventions. The problem is the .read(). It ALWAYS times out. Per the pointers when I first posted this question I've changed my settings in AWS to 3 minutes before a function times out and I also try to download it but that returns None. I guess the central questions are why does the .read() function take so long and what is missing in my download_file command? The file is small: 1KB. Any help appreciated thanks

import boto3
import csv

s3 = boto3.resource('s3')
bucket = s3.Bucket('polly-partner')
obj = bucket.Object(key='CyclingLog.csv')

def lambda_handler(event, context):

    response = obj.get()
    print(response)
    key = obj.key
    filepath = '/tmp/' + key
    print(bucket.download_file(key, filepath))
    lines = response['Body'].read()
    print(lines)

Printout is:

Response:
{
  "errorType": "Runtime.ExitError",
  "errorMessage": "RequestId: 541f6cc6-2195-409a-88d3-e98c57fbd539 Error: Runtime exited with error: signal: killed"
}

Request ID:
"541f6cc6-2195-409a-88d3-e98c57fbd539"

Function Logs:
START RequestId: 541f6cc6-2195-409a-88d3-e98c57fbd539 Version: $LATEST
{'ResponseMetadata': {'RequestId': '0860AE16F7A96522', 'HostId': 'D6k1kFcCv9Qz70ANXjEnPQEFsKpAntqJND9FRf5diae3WWmDbVDJENkPCd1oOOOfFt8BJ8b8OOY=', 'HTTPStatusCode': 200, 'HTTPHeaders': {'x-amz-id-2': 'D6k1kFcCv9Qz70ANXjEnPQEFsKpAntqJND9FRf5diae3WWmDbVDJENkPCd1oOOOfFt8BJ8b8OOY=', 'x-amz-request-id': '0860AE16F7A96522', 'date': 'Wed, 01 Apr 2020 17:51:49 GMT', 'last-modified': 'Thu, 19 Mar 2020 17:17:37 GMT', 'etag': '"b56479c4073a90943b3d862d5d4ff38d-6"', 'accept-ranges': 'bytes', 'content-type': 'text/csv', 'content-length': '50000056', 'server': 'AmazonS3'}, 'RetryAttempts': 1}, 'AcceptRanges': 'bytes', 'LastModified': datetime.datetime(2020, 3, 19, 17, 17, 37, tzinfo=tzutc()), 'ContentLength': 50000056, 'ETag': '"b56479c4073a90943b3d862d5d4ff38d-6"', 'ContentType': 'text/csv', 'Metadata': {}, 'Body': <botocore.response.StreamingBody object at 0x7f536df1ddc0>}
None
END RequestId: 541f6cc6-2195-409a-88d3-e98c57fbd539
REPORT RequestId: 541f6cc6-2195-409a-88d3-e98c57fbd539  Duration: 12923.11 ms   Billed Duration: 13000 ms   Memory Size: 128 MB Max Memory Used: 129 MB Init Duration: 362.26 ms    
RequestId: 541f6cc6-2195-409a-88d3-e98c57fbd539 Error: Runtime exited with error: signal: killed
Runtime.ExitError

Upvotes: 1

Views: 4285

Answers (2)

FunWithDunne
FunWithDunne

Reputation: 31

I know this is an old post, (and hopefully solved long ago!), but I ended up here so I'll share my findings.

These generic Runtime error messages:

"Error: Runtime exited with error: signal: killed Runtime.ExitError"

...when accompanied by something like this on the REPORT line:

Memory Size: 128 MB Max Memory Used: 129 MB Init Duration: 362.26 ms

...Looks like a low memory issue. Especially when "Max Memory Used" is >= "Memory Size"

From what I've seen, Lambda can and often will utilize up to 100% memory without issue (Discussed in this post). But when you attempt to load data into memory, or perform memory intensive processing (copying large data sets stored in variables?), the Python runtime can hit a memory error and exit. Unfortunately, it isn't very well documented, or logged, or captured with CloudWatch metrics.

I believe the same error in NodeJS runtime looks like:

"Error: Runtime exited with error: signal: aborted (core dumped)"

Upvotes: 1

John Rotenstein
John Rotenstein

Reputation: 269091

The error message says: Task timed out after 3.00 seconds

You can increase the Timeout on a Lambda function by opening the function in the console, going to the Basic settings section and clicking Edit.

While you say that you increased this timeout setting, the fact that it is timing-out after exactly 3 seconds suggests that the setting has not been changed.

Upvotes: 2

Related Questions