CODEWITHSUNDEEP
pythonpython-2.7aws-lambda
FCoding
FCoding

Reputation: 121

Module Import Error when executing a Lambda python function

I am trying to execute a python function on AWS Lambda. In my function I am trying to import the mysql.connector module. But an error is throwing up :

errorMessage": "No module named 'mysql.connector'"

I had written my python code initially in my EC2 Instance. I installed mysql-connector there in my python file directory using pip.

pip install mysql-connector -t /path/to/file/dir

I uploaded the zip file of the only the file and not any folder containing the file.

Upvotes: 2

Views: 4805

Answers (3)

Shuvojit
Shuvojit

Reputation: 1470

There are two ways to pack python dependencies with Lambda.

  1. Pack it with the function itself. To do this you just go to the root folder of your function, and install dependencies in that folder itself, not into any other folder.
$ pip install -t <some-package> .

Then zip that folder from root and upload, your zip should look something like this:

.
├── lambda_function.py
├── pymysql
│   └── ...
└── PyMySQL-0.9.3.dist-info
    └── ...
  1. Second method is a standard that I always follow,I never ship libraries or external packages with my lambda function, I always create Layers.

A layer is a ZIP archive that contains libraries, a custom runtime, or other dependencies. With layers, you can use libraries in your function without needing to include them in your deployment package.

Learn More about Lambda Layers in the docs

Upvotes: 0

toske
toske

Reputation: 1754

Pattern that I have been using to deploy Python libs to lambda is following

Firstly, prior to packaging lambda function install all of the requirements into $SOURCE_ROOT/lib folder 

pip install -r requirements.txt -t ./lib

Secondly, dd automatic import of this folder in your lambda entrypoint (that is lambda handler) 

import os
import sys

# if running in lambda
if 'LAMBDA_TASK_ROOT' in os.environ:
  sys.path.append(f"{os.environ['LAMBDA_TASK_ROOT']}/lib")

# this will render all of your packages placed as subdirs available
sys.path.append(os.path.dirname(os.path.realpath(__file__)))

Extending sys.path with your own packaged path is the crucial for this to work.

Note on native compiled extensions

Note that if you are packaging any natively compiled extensions, their compilation should be done on linux-compatible O/S, ideally on EC2 instance created from Amazon Linux AMIs Lambda is running from (currently amzn-ami-hvm-2017.03.1.20170812-x86_64-gp2, but up to date information can be always obtained from Amazon Official Docs. According to my experience, extensions built from official Docker python container worked on lambda, without need to compile on actual EC2 instance created from AMIs, but there is no guarantee as offical docs state that

If you are using any native binaries in your code, make sure they are compiled in this environment. Note that only 64-bit binaries are supported on AWS Lambda.

MySQL Connector

Quick look at MySQL connector for Python gives impression that by default package will use native Python implementation, so no C extension is loaded

Upvotes: 6

aja
aja

Reputation: 139

Lambda is just like an EC2 instance with only python installed. You need to include all the packages that are required to run the python code with the deployment package itself. None of the packages will be pre-installed when you run the code.

Upvotes: 1

Related Questions