CODEWITHSUNDEEP
pythonaws-lambdaserverless-frameworkserverlessaws-serverless
kevinlinxc
kevinlinxc

Reputation: 624

Serverless Deployment not working (Python, Lambda)

I have a lot of code abstracted away into execute():

xkcdrandomizer.py

import xkcd
import cv2
import imageio
import matplotlib.pyplot as plt
import numpy as np
import random
from kumikolib import Kumiko
import tweepy
import json
import os
...
...
...
def handler(event, context):
    statusout = execute()
    body = {
        "message": statusout,
        "input": event
    }
    response = {
        "statusCode": 200,
        "body": json.dumps(body)
    }
    return response

and I'm trying to use serverless to deploy this using the serverless-python-requirements plugin. Some of these are local packages (kumiko) so I'm expecting errors with those, but I get one with xkcd, which is a package available through pip.

Here is my serverless.yml:

service: xkcdrandomizer

provider:
  name: aws
  runtime: python3.8


functions:
  xkcdrandomizer:
    handler: xkcdrandomizer.handler
    events:
      - schedule:
          rate: cron(0 20 * * ? *)

plugins:
  - serverless-python-requirements
custom:
  pythonRequirements:
    dockerizePip: true
    useDownloadCache: true
    useStaticCache: false
    slim: true
    strip: false
    layer:
      name: ${self:provider.stage}-cv2
      description: Python requirements lambda layer
      compatibleRuntimes:
        - python3.8
      allowedAccounts:
        - '*'

Running sls deploy and sls invoke -f xkcdrandomizer yield the following error:

{
    "errorMessage": "Unable to import module 'xkcdrandomizer': No module named 'xkcd'",
    "errorType": "Runtime.ImportModuleError",
    "stackTrace": []
}

Before, my yml only had this at the bottom:

custom:
  pythonRequirements:
    dockerizePip: true

But this yielded the error

"errorMessage": "Unable to import module 'xkcdrandomizer': libGL.so.1: cannot open shared object file: No such file or directory",

Possibly due to opencv's file requirements, so I copied this SO post and now I have the error from above.

Here is my requirements.txt:

certifi==2020.12.5
chardet==4.0.0
cycler==0.10.0
idna==2.10
imageio==2.9.0
kiwisolver==1.3.1
matplotlib==3.4.1
numpy==1.20.2
oauthlib==3.1.0
opencv-python==4.5.1.48
Pillow==8.2.0
pyparsing==2.4.7
PySocks==1.7.1
python-dateutil==2.8.1
requests==2.25.1
requests-oauthlib==1.3.0
six==1.15.0
tweepy==3.10.0
urllib3==1.26.4
xkcd==2.4.2

My sls version is this My docker version is 20.10.5, build 55c4c88

I've done a few hours of googling but people's problems are very vast with serverless and it feels like I've followed the serverless-python-requirements tutorials pretty closely.

Edit: It seems that the package that appears in the error only depends on what I import first in xkcdrandomizer.py. So it seems like all imported packages aren't working.

Upvotes: 3

Views: 1004

Answers (1)

kevinlinxc
kevinlinxc

Reputation: 624

I figured it out. When I had the libGL.so error, that was the closest I was to success, and adding the opencv "fix" actually just makes all imports break.

The solution is to put opencv-python-headless in requirements instead of opencv-python, because the headless version is made for servers that don't have graphics dependencies or something like that.

Upvotes: 2

Related Questions