CODEWITHSUNDEEP
pythonimportaws-lambda
Dr. Chocolate
Dr. Chocolate

Reputation: 2165

How do I import a Python lambda layer?

I have a file with this as the contents.

def print_hello_world():
    print ('Hello World')

It's zipped up in a folder with a __init__.py file.

I add this as the layer and set the correct runtime of python3.6.

How do import it into my lambda function via lambda code?

Edit: After researching I found that lambda mounts the layers at /opt and if you add /opt to your syspath via sys.path.insert(0, '/opt') then you can import your layers.

You can see your layers at: print(os.listdir("/opt"))

There's gotta be a more elegant way to do this!

Upvotes: 13

Views: 20654

Answers (4)

mjesbar
mjesbar

Reputation: 101

SOLUTION

You must to allocate all the packages you install with pip install ... into a zip file that ends with the folllowing file structure:

[your zip file].zip
|_python/
  |_all.../
  |_your.../
  |_packages, modules.../

Another boring way is:

[your zip file].zip
|_python/
  |_lib/
    |_python3.x/
      |_site-packages/
        |_all.../
        |_your.../
        |_packages, modules.../

with this structure you lambda code get the modules you have.

I hope this make done your issue.

Upvotes: 2

Alex B
Alex B

Reputation: 2375

You'll have to:

  1. Build you lambda layer using Docker: sudo docker run -v "$PWD":/var/task "lambci/lambda:build-${penv}" /bin/sh -c "pip install -r requirements.txt -t python/lib/${penv}/site-packages/; exit"
  2. Upload layer to AWS
  3. Add layer to your lambda
  4. Modity sys.path in your Lambda before import
    import sys; 
    sys.path.append('/opt/python/lib/python3.7/site-packages');
    import apsw
    print(apsw.__file__)

/opt/python/lib/python3.7/site-packages/apsw.cpython-37m-x86_64-linux-gnu.so

Update: It looks like you need to place build under /opt/python so you can use it without sys.path.append('/opt/python/lib/python3.7/site-packages');

New build steps:

mkdir -vp my-layer/python && cd my-layer/python
python3 -m pip install click -t ./ 
cd ..
zip -r9 my-layer python

now add layer to lambda function and you can import:

import click

Upvotes: 2

Adam Easterling
Adam Easterling

Reputation: 2405

Your zip file should have the following structure:

python/lib/python3.7/site-packages

That is, it needs a folder named Python, and within that a folder named lib, and within that a folder named python3.7, and within that a folder named site-packages. Anything inside that folder will be available for import.

(If you're using another version of Python, that version should be in the path instead of 3.7)

Upvotes: 7

Cecaro
Cecaro

Reputation: 131

So I've recently ran into this issue, and I believe I found a cleaner way to import your layers.

First for the structure of the zip file which you upload:

  • You do not need an __init__.py file
  • Put all the scripts which you want to import into a folder name python
  • Zip up that python folder (choose any name you want) and upload it to your layer
  • Once uploaded, and the layer has been configured in your lambda function, you can simply use it with import {filename}

So if your script in the python folder was called something like custom_helper.py, import it in your lambda with import custom_helper.

I am not sure if this is the clean way to do it, but it seems simple enough to start.

Upvotes: 12

Related Questions