Reputation: 375
I am trying to put this scraping code on AWS Lambda, when i try to test the code i get the following error:
"errorMessage": "Unable to import module 'lambda_function': No module named 'lambda_function'",
"errorType": "Runtime.ImportModuleError"
The code:
from datetime import datetime
from functions import MultiplePageScraper, CleanTable, upload_unique, pages
from sqlalchemy import create_engine
now = datetime.now()
date = now.date()
# Credentials:
host = xxxxx
user = "admin"
password = xxxxx
port = 3306
database = "vehicles"
# Create connection
mydb = create_engine("mysql+pymysql://" + user + ':' + password + '@' + host + ':' + str(port) + '/' + database , echo=False)
# Target Url :
URL = "https://www.usedcars.co.ke/nairobi/cars-for-sale?page="
def lambda_handler(event, context):
# Retrive the data and clean it
page_count = pages(URL)
data = MultiplePageScraper(URL, int(page_count))
clean_data = CleanTable(data)
# Check for duplicates scraped Vs Database
upload_data = upload_unique(clean_data, mydb)
# Upload data to RDS database
if upload_data.shape[0] > 1:
upload_data.to_sql(name='CARS', con=mydb, if_exists = 'append', index=False)
print(f'Uploaded {upload_data.shape[0]} unique rows to the database!')
else:
print('No new cars to upload.')
I have no idea what the error means, any one can help?
Upvotes: 3
Views: 7570
Reputation: 238051
By default, a new python lambda function will look for lambda_function.py
file and call lambda_handler
method:
You can change it using the runtime settings for your function.
Upvotes: 3