CODEWITHSUNDEEP
aws-lambdapsycopg2aws-lambda-layerspsycopg3
Raz T
Raz T

Reputation: 41

Psycopg 3 AWS Lambda layer - missing libpq/static linking

Has anyone succeeded in building a psycopg3 AWS lambda layer?

I'm using psycopg version 3.1.8 in my python code, and I get this error when testing the AWS lambda function:

{
  "errorMessage": "Unable to import module 'lambda_function': no pq wrapper available.\nAttempts made:\n- couldn't import psycopg 'c' implementation: No module named 'psycopg_c'\n- couldn't import psycopg 'binary' implementation: cannot import name 'pq' from 'psycopg_binary' (/var/task/psycopg_binary/__init__.py)\n- couldn't import psycopg 'python' implementation: libpq library not found",
  "errorType": "Runtime.ImportModuleError",
  "requestId": "839c4ea4-a313-42ab-b439-4bbde7830268",
  "stackTrace": []
}

I've tried to look it up, there are many implementations on psycopg2 layers, that have been statically linked with the required libpq library.

I've tried to do as explained here but only with psycopg3: https://github.com/jkehler/awslambda-psycopg2

And even here - trying to create a libpq layer: https://github.com/DrLuke/postgres-libpq-aws-lambda-layer

but to no success (I can't seem to obtain a compiled libpq.so file)

Thanks for any help!

Upvotes: 4

Views: 2402

Answers (3)

ncarter
ncarter

Reputation: 11

I was able to get this to work using the following:

pip install \
    --platform manylinux2014_x86_64 \
    --target=my-lambda-function \
    --implementation cp \
    --python 3.9 \
    --only-binary=:all: --upgrade \
    "psycopg[binary]"

You can then zip the content of directory my-lambda-function and deploy those for use by the lambda function. My understanding is that the key here is choosing the right --platform, for the C libraries to work correctly.

I used this psycopg2 thread as a reference.

Upvotes: 1

RodrigoM
RodrigoM

Reputation: 698

Answer provided by @ncarter is almost there but didn't work for us. We followed the AWS documentation on AWS Lambda Layers and it worked smoothly:

This didn't work for us. In order to work in a AWS Lambda Layer we had to use:

python3 -m pip install --platform manylinux2014_x86_64 \
    --target=package \
    --implementation cp \
    --python-version 3.10 \
    --only-binary=:all: \
    --upgrade \
    "psycopg[binary]" -t $(ARTIFACTS_DIR)/python

Upvotes: 2

Mr. Mundkowsky
Mr. Mundkowsky

Reputation: 197

I tried to get psycopg3 work in a AWS Lambda function even using a docker container and could not get it to work. But pg8000 (https://pypi.org/project/pg8000/) module works fine.

Upvotes: 1

Related Questions