CODEWITHSUNDEEP
pythongcclinkerpipaws-lambda
a p
a p

Reputation: 3208

Specifying C(++) dependencies for python packages for use in AWS Lambda

I have a serverless service that I want to be able to use the sasl pypi package from in AWS Lambda. Unfortunately, the AWS Lambda environment doesn't seem to have the newest versions of libstdc++.so, which means that when our build server zips up the pip install'd sasl package and we invoke the lambda, there are the expected errors: 

Unable to import module 'handler': /usr/lib64/libstdc++.so.6: version `GLIBCXX_3.4.21' not found (required by /var/task/sasl/saslwrapper.so)

My question is: How to get around this? Is there a way to build these on an Amazon Linux instance against updated libstdc++.so.6 so that it can be bundled in with the libs? Would setting LD_LIBRARY_PATH env variable to . and including the newest version of libstdc++.so.6 work? Are there best practices around this anywhere?

Upvotes: 6

Views: 2104

Answers (3)

Superpassteque
Superpassteque

Reputation: 101

You can add simply the gcc-c++ binaries needed with this custom gcc version for lambda : https://github.com/lambci/gcc-lambda-layer

Or as Bluu answers compile your own image.

Edit : new solution : because its sometimes can be annoying to package layers especially with C++ packages ( numpy pands scrapy ect...), I made a lambda to create my layers propely and fast : https://github.com/vincentventalon/serverless_layer_generator

Upvotes: 2

bluu
bluu

Reputation: 552

You can use docker with amazonlinux as your base image to build all your dependencies then copy these to your zipped package to be uploaded to your actual lambda: https://medium.com/@johnnyopao/python-dependencies-and-aws-lambda-18acbdebca20

Or you can do the equivalent on a remote EC2 (build) instance: http://www.perrygeo.com/running-python-with-compiled-code-on-aws-lambda.html

Last option, use AWS Cloud 9 as your IDE, as suggested by: https://markn.ca/2018/02/python-extension-modules-in-aws-lambda/

Upvotes: 5

Lightness Races in Orbit
Lightness Races in Orbit

Reputation: 385385

Here's your problem:

AWS Lambda is an ideal compute platform for many application scenarios, provided that you can write your application code in languages supported by AWS Lambda (that is, Node.js, Java, C# and Python)

You're doing something you shouldn't.

You might be able to hack it (I've had success in the past with redistributing libstdc++ and libgcc_s from my build machine, plonking them somewhere along with my executable and making sure they're are preferred at runtime using -Wl,-rpath,'$ORIGIN') but you'll basically be on your own.

C++ isn't really designed to be deployed to a platform other than the one on which it was built, so unless Amazon provide a "AWS Lambda" environment to build things in, you'll be stuck with such hacks.

Upvotes: -4

Related Questions