CODEWITHSUNDEEP
pythondockeraws-lambdapip
maurera
maurera

Reputation: 1679

How do I pip install Linux packages on a Windows machine (for AWS Lambda)?

I'm trying to write a Python AWS Lambda script. The Python code works locally in Windows, but it's using the Windows packages installed through pip. When uploading to AWS Lambda, I need to include the Linux packages.

For example, when I run pip install pandas I get:

Downloading pandas-1.0.1-cp37-cp37m-win_amd64.whl

But what I need (for uploading to AWS Lambda) is:

pandas-1.0.1-cp37-cp37m-manylinux1_x86_64.whl

My Attempt

I have tried to use Docker to simulate a Linux environment in Windows. My idea is to pip install the Linux packages in Docker and then copy them to my local machine. I think this can be done through a Docker Volume. I have tried to do this using the Dockerfile:

FROM python:3.7-slim-buster

WORKDIR /usr/src/app

# Download python packages to /usr/src/app/lib
RUN mkdir -p /usr/src/app/lib
RUN pip3 install pandas -t /usr/src/app/lib

# Copy the python pacakges to local machine
VOLUME host:/myvol
RUN mkdir /myvol
COPY /usr/src/app/lib /myvol

But when I run docker build I get the error:

COPY failed: stat /var/lib/docker/tmp/docker-builder233015161/usr/src/app/lib: no such file or directory

Upvotes: 1

Views: 2173

Answers (1)

maurera
maurera

Reputation: 1679

Thanks to explanation from @C.Nivs, this can be done using Docker interactively:

  1. First run docker interactively with docker run -it python:3.6-slim bash. Then create a folder 'target' and pip install into it (this installs the linux packages). Make note of the container id (my command line shows root@31d9be68deec:/#. The container id is 31d9be68deec)
mkdir /target 
pip install pandas -t /target
  1. Then open a new command prompt and use docker cp to copy files from container to local. The following copies from the target folder in the container to the local folder libs.
docker cp <container_id>:/target libs

That's it. The python packages are now available in the local folder libs.

Note from @C.Nivs: "COPY doesn't do what it's name might suggest. COPY is a build step that copies files from the build context (where the image is being built) to the image itself. It doesn't allow things to go the other way".

Upvotes: 0

Related Questions