CODEWITHSUNDEEP
pythonamazon-web-servicesaws-lambdacx-oracle
Miboon Kim
Miboon Kim

Reputation: 11

How can I solve cx_Oracle problem in python when running on aws lambda

I had encountered below error message when I test on aws lambda.

Unable to import module 'lambda_handler': No module named 'cx_Oracle'

This lambda consists of python code, but especially works well on my localhost.

Also, I executed pip install cx_Oracle -t ./ command on same level of lambda_handler.py file path, and uploaded zip file's file path is below that.

./lambda_handler.py

./cx_Oracle.cp36-win_amd64.pyd

./cx_Oracle-6.4.1.dist-info/*

./cx_Oracle-doc/*

I can't understand why lambda_handler can't find cx_Oracle library in only lambda function.

What am I missing?

import cx_Oracle
import boto3
from docutils.nodes import row

def lambda_handler(event, context):

    url = 'myOracleUrl'
    user = 'myOralceUserId'
    pwd = 'myOraclePwd'
    con = cx_Oracle.connect(user + '/' + pwd + '@' + url)
    cur = con.cursor()
    cur.execute("select 1 from dual")
    cur.close()
    con.close()

    return {
        "statusCode": 200,
        "body": context.invoked_function_arn
    }

Upvotes: 1

Views: 9629

Answers (3)

alvaro
alvaro

Reputation: 462

Short Answer:
The cx_Oracle library you are using is for Windows. Aws Lambda environment is Amazon Linux, *.pyd are Python libraries for Windows.

Long Answer: You have to deploy the linux version The first thing you are getting wrong is the python library you are providing along your code to AWS Lambda. I guess you are developing locally in a Windows machine so when you do:

pip install cx_Oracle -t ./

you get a python precompiled library for Windows which is the cx_Oracle.cp36-win_amd64.pyd file. AWS Lambdas run in Linux containers. You can find some info about the environment here. The precompiled library for Linux is cx_Oracle.cpython-36m-x86_64-linux-gnu.so. You can download it manually from the Pypi repository.

So when you execute a python script in a Linux environment it will try to find the linux precompiled version of the library in the current path, the cx_Oracle.cpython-36m-x86_64-linux-gnu.so file. That is why you are getting the error message:

Unable to import module 'lambda_handler': No module named 'cx_Oracle'

The second thing important here is that the cx_Oracle python library calls the Oracle Instant Client which is a native library. If you have a working example on your windows machine you must have this installed. For windows this library is in the form of .dll files. For linux this native library is in the form of .so files. You will need to provide the Oracle client for Linux along with your code in your deployment package because the are not already available in the Amazon Linux environment.

And the last thing is to tell Python how to find this dynamic library. In Linux systems shared libraries are looked for in the directories present in this environment variable LD_LIBRARY_PATH. In Amazon Linux this variable contains:

LD_LIBRARY_PATH:/var/lang/lib:/lib64:/usr/lib64:/var/runtime:/var/runtime/lib:/var/task:/var/task/lib:/opt/lib

The /var/task directory is whre your deployment resides. So if you add the Oracle instant client for Linux under /var/task/lib which is ./lib in your development directory you are done.

Your deployment package should be similar to this:

├── lib                         
│   ├── libclntsh.so                            <-- Oracle instant client linux files
│            .
│            .
│            .
├── python_handler.py                           <-- Lambda function code
└── cx_Oracle.cpython-36m-x86_64-linux-gnu.so   <-- cx_Oracle library

Upvotes: 8

trnrao
trnrao

Reputation: 94

We had run into this exact problem and we then had to implement the data access functions into AWS Lambda Java 8 instead of Python 3.6

In Java 8, ojdbc*.jar (* --> version number of Oracle driver) can be simply bundled into the final zip and there are no other steps like trying to get Oracle Client Native Libraries.

We did spend good amount of time to make AWS Lambda Python Function access Oracle DB, but I felt developing such modules in AWS Lambda Java 8 function is much simpler.

Upvotes: 0

bimsapi
bimsapi

Reputation: 5065

cx_Oracle requires not just pip install, but the native Oracle client, which the wheel depends on. Lambda functions run on an EC2 with a limited selection of native libraries. For what it's worth, you'd have the same problem with almost any RDBMS - PostgreSQL, MySQL, etc.

See this article for more information: http://kb.sites.apiit.edu.my/knowledge-base/articles/aws/aws-chalice-deployment-with-oracle-vendor/

Note that it is for Chalice-based deployment, but the underlying requirement is the same.

Upvotes: 0

Related Questions