AWS python lambda function:No module named requests

Question

I am fairly new to AWS and I am having some issues. Here is my code:

from __future__ import print_function
from urllib2 import Request, urlopen, URLError
import requests
import boto3
import json

def lambda_handler(event, context):
    url = "https://globalcurrencies.xignite.com/xGlobalCurrencies.json/GetHistoricalRatesRange?Symbol=BTCUSD&PriceType=Mid&StartDate=01/01/2017&EndDate=10/27/2017&PeriodType=Daily&FixingTime=22:00&_token=some_token_xyz"
    response = requests.get(url).json()
    # print json.dumps(response, indent=4) # gives a syntax error
    return response

Name of the file is lambda_function.py; I have checked similar problems on stackoverflow and some mentioned that I have to change the file naming. But it didn't help. Here is how python method was named:
Here is the error I am getting: START RequestId: cf24e9be-bbef-11e7-97b4-d9b586307f3e Version: $LATEST Unable to import module 'lambda_function': No module named requests And when try to print it gives me a syntax error. Sorry for the formatting. Any suggestions?

Answer 1

For Python 3.12 lambda runtime use the following import for requests module

from pip._vendor import requests

Answer 2

POST: 20Feb24

You can use the http.client like below.

This solves the purpose and you don't have to install anything else for http.client.

For scheduler cronjob use https://docs.aws.amazon.com/lambda/latest/dg/services-cloudwatchevents-expressions.html

and https://stackoverflow.com/a/68146073/11207658

PS: cronjob uses the UTC timezone. so apply it accordingly

Answer 3

N.B requests is already included in lambda functions with runtime Python 3.7:

In fact:

from pkg_resources import working_set

def lambda_handler(event, context):
    for p in list(working_set):
        print(p.project_name, p.version)

urllib3 1.26.6 six 1.16.0 s3transfer 0.6.0 requests 2.26.0 python-dateutil 2.8.2 jmespath 1.0.1 idna 2.10 charset-normalizer 2.0.12 chardet 4.0.0 certifi 2020.11.8 botocore 1.29.90 boto3 1.26.90 setuptools 47.1.0 pip 22.0.4

For all other runtimes, see below.

---

If you want to include libraries which are not part of Python's standard library, such as requests, you can use lambda's layers.

1. Create the zip

This is a zip which contains all the libraries you want the lambda function to use. First, create a folder called python:

$ mkdir python
$ cd python

then, install the Python libraries you need in there. You can do this either with a single library:

$ pip install --target . requests

or with a list of libraries (requirements.txt)

$ pip install --target . -r requirements.txt

finally, zip up everything.

... in Bash:

$ zip -r dependencies.zip ../python

... in Powershell:

$ Compress-Archive -Path . -DestinationPath dependencies.zip

---

2. Create a layer

You can do this either in the AWS console or in the CLI. Follow these instructions.

---

3. Add the layer to the lambda function

You can do this either with the Add a layer option in the lambda page, or in the CLI. Follow these instructions.

Answer 4

Use urllib3 instead

While yes, requests is not a standard library in AWS lambda, there are alternatives within the standard library that will allow you to avoid Lambda layers.

If you check the list of libraries that Nilo_DS linked (https://gist.github.com/gene1wood/4a052f39490fae00e0c3), you'll find that urllib3 is available.

import requests

requests.post(<url>, data=json.dumps(<message>))

can be rewritten as

import urllib3

http = urllib3.PoolManager()
http.request('POST', '<url>',
             headers={'Content-Type': 'application/json'},
             body=json.dumps(<message>))

and similar changes for a get request.

Answer 5

For future readers using Python: If you created your Lambda app directly in Cloud9, you would notice that there is a virtual environment for your app. To install requests (or any other package for that matter), do the following:

1. Right-click on the folder for your app
2. Select "Open terminal here"
3. In the terminal type source venv/bin/activate to activate your virtual environment
4. Type pip3 install requests to install requests

That is it. You can now use requests with import requests as per normal.

Answer 6

requests is not a standard library in AWS lambda.

So two ways to solve this:

1- Import it from the Botocore libraries stack as: (Option to be deprecated after the answer was given)

from botocore.vendored import requests

Here there is a list of all the available libraries to import in lambda

2- Create a deployment package with virtualenv.

Answer 7

You want to use Layers. You download a python module e.g pip install requests -t . then simply put all the module folders into a folder called python and zip it and upload it in the lambda layers section of the aws console. Add it to your lambda function and it will work. The dir structure is very important so for requests when you unzip the folder it should be RequestLayer/python/requests/"requests and all the other folders that are part of that package". This guy gives a good step by step.

Answer 8

'requests' module is not in your 'zip' file that your are trying to install. you have to put all modules into the zip file by 'pip install module_name(such as requests) -t .'

Answer 9

This is because it is missing the requests library when running in the lambda - its likely that its installed globally on your local machine. If you run:
pip install requests -t .
in the same directory as your source code it will install the requests package in that directory then you can upload it to lambda along with your lambda_function.py. You may need to do the same for boto3 and json:
pip install boto3 -t .
pip install json -t .

Answer 10

you have to name your lambda this way in python code:

def lambda_function(event, context):

and in lambda console handler:

main.lambda_function

for your request error, must have the folder of that module on your .zip before you upload to lambda.