CODEWITHSUNDEEP
pythonamazon-web-servicesamazon-s3aws-lambdaamazon-ecr
JackWills
JackWills

Reputation: 21

Lambda built from container image (Docker ECR) hanging when retrieving data from S3

I have a lambda function that uses a Docker container stored in ECR.

When I test the image locally I have no issues at all, however when I deploy it in lambda it fails due to timeout (no matter the timeout length).

Here's a slimmed down version of the lambda:

import pandas as pd
import json 
import pickle

def read_obj_from_s3(bucket, key):
    file_type = key.split('.')[-1]
    s3 = boto3.resource('s3')
    s3_bucket = s3.Bucket(bucket)
    s3_obj = s3_bucket.Object(key)
    file = s3_obj.get()['Body'].read() # here is where it hangs
    if file_type == 'json':
        return json.loads(file)
    return pickle.loads(file)

bucket = 'project-underdog-s3'

df = read_obj_from_s3('BUCKET_NAME', 'S3_LOCATION/file.json')

def lambda_handler(event, context):
    use_the_file(df)
    return json.dumps({
            'statusCode': 200,
                })

It seems the file always hangs at s3_obj.get()['Body'].read().

I have checked the Execution Role and it has AmazonS3FullAccess so permissions should be fine.

A number of similar questions were resolved as they were using a VPC. To confirm, I'm not using a VPC. This is what is shown in the VPC section of the lambda:

No VPC configuration This function isn't connected to a VPC.

Appreciate any advice on where to investigate next.

Upvotes: 1

Views: 312

Answers (1)

JackWills
JackWills

Reputation: 21

The issue was memory. Once I increased the memory to 500Mb (from 128Mb) it worked. The files were small but with the packages it was enough to push it over the edge.

Upvotes: 1

Related Questions