CODEWITHSUNDEEP
pythonaws-lambdapython-imaging-libraryin-memory
Johan Kuylenstierna
Johan Kuylenstierna

Reputation: 189

Pillow save to buffer slow on aws lambda

I'm working on a service to create overlayed images hosted an aws lambda. After managing to setup all the requirements and actually get it working, I noticed that the code ran extremely slow. I managed to track the problem to the in-memory buffering of the image that I used while saving a PIL image to an object that could be sent to an s3 bucket. Here is the problematic code:

def to_s3(img, s3_bucket, key):
    logger.info("attempting to upload {} from s3".format(key))
    buffer = BytesIO()
    ts = time()
    img.save(buffer, "PNG")
    logger.info("image inmem save time {}".format(time() - ts))
    buffer.seek(0)
    sent_data = s3_bucket.put_object(Key=key, Body=buffer)

The img.save(buffer, "PNG") function took around 10 seconds to write an image of around 1.8mb which doesn't seem reasonable. So my questions are:

Any suggestions are appreciated. It's Friday evening where I live and I'd love to not have to work through the weekend. Thanks in advance.

Upvotes: 1

Views: 825

Answers (1)

Johan Kuylenstierna
Johan Kuylenstierna

Reputation: 189

Ok, did some benchmarking locally and it seems like (at least in that environment) saving to an in memory buffer and saving to file takes about the same time to process. Saving to jpeg as suggested in the comments also improved size and speed. The problem lied in the lambdas memory allocation. Scaling up the lambda from 256 to 3072mb lowered the save time to about 1.5 seconds which albeit still quite slow, is still an improvement. I'm new to lambdas so don't judge too harshly.

Upvotes: 1

Related Questions