CODEWITHSUNDEEP
python-3.xdockeraws-lambdadockerfile
Haoyang
Haoyang

Reputation: 33

Long deploy time of AWS lambda function with huge dependencies

I am building an app using AWS lambda function. My own code logic is very simple: one python file calling some ML functions. On the contrary, it requires a lot of packages (dependencies). I know for smaller cases I can zip these dependencies and upload the zip file for the lambda function. However, it doesn't fit my use case as the zip file can easily exceed the 250 MB size limit. So I have to use the Docker image approach - create a lambda function directly from the Docker image.

I am using sam tool to build / deploy changes. Deploy is very time consuming as it needs to push a very big (6G) image to ECR. The worst thing is I have to endure this every time even making a small changes on my own code and never touching the dependencies.

  1. Can I apply the same way as zip approach, i.e, only include dependencies in the docker image and put customized logic out of it? Is it doable?

  2. If 1 is not possible, what is the best practice / tips for this? I guess there are some magic in Dockerfile but I am pretty noob here. Any demo / sample codes would be great. I linked my dockerfile below:

FROM --platform=linux/amd64 public.ecr.aws/lambda/python:3.9

RUN python3.9 -m pip install --upgrade pip
COPY requirements.txt ./
RUN python3.9 -m pip install -r requirements.txt -t .
COPY *.py ./

# Command can be overwritten by providing a different command in the template directly.
CMD ["app.lambda_handler"]

I found some suggestions to upload the package to S3 and download it from S3 into tmp folder. This doesn't sound very cool and even if it is, there is also a 512 MB limit which still too small for my use case.

Update 1

I want to add more context about the app. The app is a prototype to run stable diffusion model. The image is big because it requires bunch of ML packages.

Here is my requirements.txt

numpy==1.19.5
opencv-python==4.5.5.64
transformers==4.16.2
diffusers==0.2.4
tqdm==4.64.0
openvino==2022.3.0
huggingface_hub==0.9.0
scipy==1.9.0
streamlit==1.12.0
watchdog==2.1.9
ftfy==6.1.1
replicate
uvicorn
requests
fastapi
mangum

I would say the application is pretty similar to this project and the dockerfile is also similar.

One tool I found pretty handy is sam cli. When code changes, everytime I just need to run sam build (finish in seconds) and sam deploy (<5 minutes), which greatly reduces my pain. Also, it has a bootstrap functionality which could set up automatically based on the choice of using zip or image approach.

Upvotes: 3

Views: 1908

Answers (3)

Robb
Robb

Reputation: 65

I like your idea much;

-f https://download.pytorch.org/whl/torch_stable.html
torch==2.0.1+cpu

Unfortunately I am getting, this;

"python3.10/site-packages/torchvision/io/image.py:13: UserWarning: Failed to load image Python extension: 'libc10_hip.so: cannot open shared object..."

Upvotes: 0

epursiai
epursiai

Reputation: 76

Not sure if I am late to this party with my answer, but here's some tips anyways.

When using pip+docker, use --no-cache-dir
You usually won't be installing anything with pip after the image has been built. Therefore, you won't be needing that pip cache.

So instead of:
RUN python3.9 -m pip install -r requirements.txt -t .
do:
RUN python3.9 -m pip install --no-cache-dir -r requirements.txt -t .

Lambda doesn't need GPU dependencies
You have transformers listed in your requirements.txt. It has torch as it's dependency. And torch tends to install all sorts of GPU computation stuff, which won't be needed on Lambda, since you won't have a GPU there.

To prevent this, you need to install torch+cpu separately before those transformers' dependencies get installed. Unfortunately, torch+cpu is not available through PyPI, so you need to define it as follows:

-f https://download.pytorch.org/whl/torch_stable.html
torch==2.0.1+cpu

So your new requirements.txt would be:

numpy==1.19.5
opencv-python==4.5.5.64
-f https://download.pytorch.org/whl/torch_stable.html
torch==2.0.1+cpu
transformers==4.16.2
diffusers==0.2.4
tqdm==4.64.0
openvino==2022.3.0
huggingface_hub==0.9.0
scipy==1.9.0
streamlit==1.12.0
watchdog==2.1.9
ftfy==6.1.1
replicate
uvicorn
requests
fastapi
mangum

Note that you want to make sure that torch+cpu is listed before transformers.

These two small changes will slim down your lambda image several GBs, and in most use cases, without any drawbacks.

Upvotes: 1

lxg
lxg

Reputation: 13127

6GB for a Docker image is really a lot, and it will cost you a lot to run it as a Lambda.

There are many things you could try to slim the image down or reorganise your application.

  • Remove unneeded code. You could use a multi-stage build where you pre-build your application in one step and then use a very slim image for the runtime.

  • Manage static assets in S3. Upload them before you deploy the Lambda.

  • Or, consider using Lambda layers for generic code. This means, you can pre-deploy generic parts of your application and only update the logic you’re currently working on.

  • Another option might be to deploy multiple Lambdas and have them communicate through events or Function URLs.

By the way, have you considered running on ECS rather than Lambda? I don’t know your exact requirements, but this might be more convenient to deploy via ECR and Cloudformation (or, Terraform) as well as potentially more cost efficient in the end. Though the above suggestions (reduce size, extract assets, etc.) would be applicable to ECS as well.

Upvotes: 4

Related Questions