CODEWITHSUNDEEP
pythonamazon-web-servicestensorflowaws-lambda
user3447228
user3447228

Reputation: 498

Unable to install tensorflow inside lambda container

I'm trying to build a TensorFlow container to deploy to Lambda (following the instructions here).

My dockerfile file is:

FROM public.ecr.aws/lambda/python:3.8

# Copy function code
COPY app.py ${LAMBDA_TASK_ROOT}

# Install the function's dependencies using file requirements.txt
# from your project folder.

COPY requirements.txt  .
RUN  pip3 install -r requirements.txt --target "${LAMBDA_TASK_ROOT}"

# Set the CMD to your handler (could also be done as a parameter override outside of the Dockerfile)
CMD [ "app.handler" ]

while my requirement.txt is

tensorflow==2.4.0

and the error I'm getting when I build the docker container is:

#8 0.653 ERROR: Could not find a version that satisfies the requirement tensorflow==2.4.0 (from versions: none)
#8 0.653 ERROR: No matching distribution found for tensorflow==2.4.0
#8 0.847 WARNING: You are using pip version 21.1.1; however, version 21.3.1 is available.
#8 0.847 You should consider upgrading via the '/var/lang/bin/python3.8 -m pip install --upgrade pip' command.
------
executor failed running [/bin/sh -c pip3 install -r requirements.txt --target "${LAMBDA_TASK_ROOT}"]: exit code: 1

Things I've tried

  1. Other python libraries (Scikit learn, numpy) install fine
  2. TF requirements are met. Python >= 3.8 && running 68 bit version
  3. Removing the lambda task root has no effect.
  4. Changing the TensorFlow version

The only thing I can guess is the base image does not support TF, but I can't see why?

Upvotes: 0

Views: 951

Answers (2)

Réda Maizate
Réda Maizate

Reputation: 11

The only solution I found was to change the version of Python, I switched from 3.8 (just like you) to 3.6 and it worked. In my Dockerfile below, I passed to python:3.6.2022.01.11.19 because it was the lightest one.

FROM public.ecr.aws/lambda/python:3.6.2022.01.11.19
WORKDIR ${LAMBDA_TASK_ROOT}

RUN yum update -y

COPY src/ .
COPY requirements.txt requirements.txt

RUN pip3 install --upgrade pip
RUN pip3 install -r requirements.txt

CMD ["lambda_function.lambda_handler"]

Upvotes: 1

Allan Chua
Allan Chua

Reputation: 10195

The base image does not support TF because not everybody that needs the Lambda environment needs Tensorflow inside the runtime. You will have to provision a lambda layer using lambci to emulate the environment of a lambda function's execution environment.

PS: Tensorflow library requires 1GB+ worth of dependencies alone which means that your Lambda with the TensorFlow library and your model won't meet the max size limitation of package uploads. To override that problem, you can consider converting your model into its TFLite counterpart. This will allow you to get rid of the heavy TensorFlow library in the package and meet the limitations.

Here is a guide that you could use to get started.

Upvotes: 1

Related Questions