CODEWITHSUNDEEP
pythonmongodbamazon-web-servicesaws-lambdapymongo
user8484970
user8484970

Reputation:

Unable to import Pymongo into AWS Lambda Function

How can I use pymongo in a lambda function?

After following the instructions from AWS - https://docs.aws.amazon.com/lambda/latest/dg/lambda-python-how-to-create-deployment-package.html#python-package-dependencies, I've found that the pymongo library is not recognized by my lambda handler function, but other local packages are.

The python-dateutil is a local package that works as expected , however the pymongo package is not recognized as an import and fails when run.

My file name is correctly listed in the lambda handler function as well (hotel-car-rental.lambda_handler) and other local package imports work fine. So I believe it is something to do with pymongo? How can I get pymongo to work with aws lambda?

Here is the lambda file structure:

Here is the code:

import json
import datetime
import dateutil.parser  # <--- Works with local package
import logging

from pymongo import MongoClient   # <--- Does NOT work with local package

# --- Main handler ---

def lambda_handler(event, context):
    """
    Route the incoming request based on intent.
    The JSON body of the request is provided in the event slot.
    """

    logger.debug('event.bot.name={}'.format(event['bot']['name']))

    return dispatch(event)

Upvotes: 4

Views: 5995

Answers (3)

YakovL
YakovL

Reputation: 8347

I'm building AWS lambdas via the Serverless framework and had a similar problem. However, I've found an official example and noticed that they provide a specific plugin for that called serverless-python-requirements. Once I've added it to serverless.yml

plugins:
  - serverless-python-requirements

I've got a warning (on serverless deploy) that I have to install the plugin, which is done via

serverless plugin install -n serverless-python-requirements

Once I've done that, import from pymongo no longer causes the problem.
(also I had to add node_modules/ to .gitignore and add package files to git)

Note: there's also a known problem that some users including those using Windows + VS Code/Powershell may face with serverless-python-requirements: if one creates requirements.txt via

pip freeze > requirements.txt

that may result in wrong encoding and serverless deploy fails in that case. Changing encoding to UTF-8 fixes the issue. They suggest that using

pip freeze | Out-File -Encoding UTF8 requirements.txt

instead allows to avoid this problem.

Upvotes: 0

user10775237
user10775237

Reputation:

You could also - instead of doing this since it requires copying/tinkering with your own stuff - run this inside a docker container to package up lambda.

docker run -v <code directory>:/function -it --rm ubuntu
apt-get update
apt-get install python-pip
apt-get install zip

cd function
pip install -t . pymongo
cd ..
zip -r linux-lambda.zip working

The only reason I do this now is that when using a Mac - which I do - it can sometimes give an "Invalid ELF Header" error and Lambda won't compile!

Upvotes: 1

user8484970
user8484970

Reputation:

The way I solved this issue was by packaging up the entire 'site-packages' directory in my virtual environment, rather than just the individual package locations. It appears pymongo installs additional dependencies, such as bson library, that are required.

Here is the specific process I used to package installed libraries for use in aws lambda.

mkdir -p ./package/tmp/lib
cp -a ./lambda/. ./package/tmp/
cp -a venv/lib/python3.7/site-packages/. ./package/tmp/
rm -rf ./package/tmp/wheel* && rm -rf ./package/tmp/pip*
cd ./package/tmp && zip -r ../../hotel-car-rental.zip .

Upvotes: 1

Related Questions