pymssql package does not work with lambda in aws

Question

How do we create a pymssql package for lambda. I tried creating it using pip install pymssql -t . When I run my lambda function it complaints saying that Unable to import module 'lambda_function': No module named lambda_function

I follow the steps on this link http://docs.aws.amazon.com/lambda/latest/dg/lambda-python-how-to-create-deployment-package.html

I have a windows machine

Answer 1

Just got this working for pymssql version 2.2.8 and AWS lambda with python3.11

Steps:

1. Use Cloud9 to open on EC2 instance running Amazon Linux 2
2. Go to the IDE and check the installed python version. Usually it is 3.7
3. Install python3.11 with the below code

sudo yum update -y
sudo yum erase openssl-devel -y
sudo yum install openssl11 openssl11-devel  libffi-devel bzip2-devel wget -y
wget https://www.python.org/ftp/python/3.11.4/Python-3.11.4.tgz
tar -xf Python-3.11.4.tgz
cd Python-3.11.4/
./configure --prefix=/usr --enable-optimizations
make -j $(nproc)
sudo make altinstall
# Verify successful install
python3.11 -V

4. Create and activate a virtual environment with python3.11

python3.11 -m venv venv
. venv/bin/activate

5. Create a folder for layers and navigate into it

mkdir layers
cd layers

6. Use the long-form lambda layer folder structure, for some reason it doesn't working using the shorter form

mkdir -p python/lib/python3.11/site-packages

7. Ensure pip is up to date

pip install --upgrade pip

8. Install pymssql into the site-packages directory

pip install --target=python/lib/python3.11/site-packages pymssql

9. Zip the folders and publish a lambda layer

zip -r pymssql_layer.zip .
aws lambda publish-layer-version --layer-name pymssql-layer --zip-file fileb://pymssql_layer.zip

10. Import the layer in your lambda function and test it out!

Answer 2

Looks like the latest wheel version of pymssql is not supported by AWS Lambda ( manylinux_2_28_x86_64)

I've created a GitHub Repo which contains the working version of pymssql for AWS Lambda (python 3.10). You can find the zip file over here : https://github.com/yogendra-kokamkar/python-package-lambda-layers/blob/main/pymssql3.zip

Below are the steps to create your own zip file as per the requirements:

Need to manually download the older version ( manylinux_2_17_x86_64.manylinux2014_x86_64 )

After downloading the wheel file execute the below command

pip install "path of downloaded wheel file" -t "target folder to store this package"

Zip the content of the pip command and upload it as a layer in lambda

Below are the steps which I used to create my zip file

mkdir wheel; cd wheel
wget https://files.pythonhosted.org/packages/0d/ec/14f9b1fb638e30ab74599aca27a7af551cd7ac58299c4922009623a153b7/pymssql-2.2.8-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
mkdir python; cd python
pip install ../pymssql-2.2.8-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl -t .
cd ..
zip -r /tmp/pymssql.zip .

Answer 3

Just a quick update for new changes.

There seems to be some issue with python 3.9 AWS lambda, it throws the below error on pymssql==2.2.3.

{
  "errorMessage": "Unable to import module 'index': No module named 
  'pymssql._pymssql'",
  "errorType": "Runtime.ImportModuleError",
  "stackTrace": []
}

but if I change the python version to 3.8, the error vanishes.

Answer 4

Along with pymssql try to import cypthon.

Answer 5

Windows "pymssql" file not supported in AWS lambda since lambda running on Amazon Linux 2.

So we can get Linux-supported wheel files from the following official "pymssql" website download link. You look for "manylinux_2_24_x86_64" version file with the required python version. At the time of writing python 3.9 is the latest version. So download file will be "pymssql-2.2.2-cp39-cp39-manylinux_2_24_x86_64.whl".

Once download the wheel file execute the below command

pip install {path of downloaded wheel file} -t {target folder to store this package}

Example

pip install pymssql-2.2.2-cp39-cp39-manylinux_2_24_x86_64.whl  -t /package

Answer 6

Glad that it worked for you, could you please share the working process for your, me too tried different trial and error steps and ended up with the following one which is working fine in AWS Lambda, I am using PYMSSQL package only.

1) did 'pip install pymssql' on amazon EC2 instance as under the hood Amazon uses Linux AMIs to run their Lambda functions.

2) copied the generated .so files and packaged inside the Lambda deployment package hope this will helps others as well who are searching for the solution.

Hope this will help you further, can you please share what you did to connect to MSSQL server using AWS Lambda.

Below is the folder structure of my lambda deployment package

Answer 7

Finally i could do it. It didnt worked with windows packages so used ubuntu to package freetds.so file and it worked.