CODEWITHSUNDEEP
pythonwindowsamazon-web-servicesaws-lambda
harshini
harshini

Reputation: 97

how to upload python code with libraries to aws lambda from windows local machine using aws console

I need to use AWS Lambda triggered through API gateway. I have python script which loads a machine learning model from S3bucket and gets input from api call and predicts the result. I can successfully trigger the lambda function written inline in python. But I want to use machine learning packages to predict in lambda function. So I came to know that I need to upload the code with the packages installed in virtual environment and I did.But the lambda when triggered gives the error 'Unable to import model lambda_function'. I have lambda_function.py with method 'handler'. Please let me know if Iam doing it right(creating virtual env and installing packages and uploading it) and why is this error. Also, let me know the solutions for Windows and AWS console. I have seen many answers with Linux commands and using aws cli.

zip folder

lambda_function

lamnda function settings

lambda function settings

Update:

This is driving me crazy!. I have tried all the methods found in the answers and none works for me. And it gives the same error : 'Unable to import module : lambda_function' So Iam not able to understand where the error is. Please help me if you have any suggestion. Before you say function names: I have correct names: lambda_function.lambda_handler. I zipped the contents and not directory. Please see my lambda code and lambda settings below lambda json file

Lambda function code:

import boto3
import os
import uuid
import sklearn
import pickle

def lambda_handler(event, context):
    s3_client = boto3.client('s3')
    s_desc=event['params']['querystring']['token']
    X_test1=[]
    X_test1.append(s_desc)
    #load model
    bucket = 'harshini-snow-bucket'
    key = 'model.pkl'
    download_path = '/tmp/{}{}'.format(uuid.uuid4(), key)
    s3_client.download_file(bucket, key, download_path)

    f = open(download_path, 'rb')
    model = pickle.load(f)
    f.close()
    #class_predicted = model.predict(X_test1)

    return X_test1

Please tell me if there are any other ways.. I will try anything for this to work.

Update 2:

error

code

Upvotes: 5

Views: 9533

Answers (6)

Scott
Scott

Reputation: 31

This drove me crazy too. I am using Windows to create the python packages. The key to solving this is to store your packages in a folder named python. When I named the folder anything else it wouldnt work.

Upvotes: 0

Mike
Mike

Reputation: 53

You need to add a Layer for outside python libraries in AWS Lambda.

https://docs.aws.amazon.com/lambda/latest/dg/configuration-layers.html

The AWS documentation is very bad with examples and for our team it was not easy. Had to manually install a number of libraries but this worked.

Upvotes: 0

annish kumar
annish kumar

Reputation: 23

I know that i am a bit late for answering this question but I was facing the same problem with pandas library for python.

this was the link which help me to solve this issue HERE

so the answer in a nutshell is

1.go to here

2.search the library you want to use

3.then go to download option

4.download the file with Linux option

5.Unzip the file in your project folder (I use 7-zip for that)

6.now make a new zip of your project and upload on lambda

Upvotes: 0

kirilclicks
kirilclicks

Reputation: 1677

The answers here are technically correct but if you're working on a Windows machine (or anything other than Amazon Linux), you might have a lot of issues putting custom packages in your Python app. Lambda apps run on Amazon Linux so you need to install the packages using that OS or anything as close as possible.

Here's one of my answers which describes how I put together packages for my Lambda app:

https://stackoverflow.com/a/50767639/3023353

Upvotes: 0

Mausam Sharma
Mausam Sharma

Reputation: 892

You have to make a custom deployment package for lambda either using docker or EC2. It will not work if you make a package in local machine as it will not compile the libaries needed.

here is the complete example, how you will make a custom package , this example packages PILLOW image processing library of python along with the lambda code , you can pack all other libraries you need for your model in the same way in the same package along with PIL

link to example

Remember one thing , in example filename is CreateThumbnail.py, you can give it any name , but always configure your handler in this way filename.handler-function , e.g disco.lambda_handler where disco.py is filename and lambda_handler is code handler module for lambda

Upvotes: 4

Manoj Acharya
Manoj Acharya

Reputation: 1479

First install all the required packages inside a folder in local machine. Also include the main lambda_function.py file inside that folder. Now select all the packages and the python file inside the folder and compress them into a .zip file. Here you have to make sure you compress contents of the folder not the folder itself. Then you can upload the zip file to lambda either directly or through s3. Another point to note is if the python file is named "lambda_function.py" or not. By default lambda assumes the main python file to have "lambda_function.py" name. If you have used any other name, you can change the filename from lambda console under Function code section inside Handler...Hope this helps out.

Upvotes: 4

Related Questions