CODEWITHSUNDEEP
rubyamazon-web-servicesaws-lambdarubygems
Zags
Zags

Reputation: 41328

Ruby gem with native extensions not working on AWS Lambda

I have a ruby script that I am trying to run on AWS Lambda. How do I get this to use a Ruby gem with native extensions?

I have installed my Ruby gems through bundle install --deployment and included them in my deployment. When I run the function on lambda, I get the error:

Ignoring oj-2.18.5 because its extensions are not built. Try: gem pristine oj --version 2.18.5

Init error when loading handler

{

"errorMessage": "libruby.so.2.5: cannot open shared object file: No such file or directory - /opt/ruby/gems/2.5.0/gems/oj-2.18.5/lib/oj/oj.so",

...

I have tried including dependencies both in the lambda code itself and as a lambda layer. The only thing that changes is the path in the error message.

Lambda is able to find my ruby gems. I get a different error when they are in the wrong location.

/opt/ruby/gems/2.5.0/gems/oj-2.18.5/lib/oj/oj.so does exist.

I have tried this with files generated by bundle install on both Ubuntu and AWS-linux. On both systems, bundle informs me that it is "Installing oj 2.18.5 with native extensions".

If I upload a copy of libruby.so to my lambda, and set the environment variable LD_LIBRARY_PATH to its location and using the set of dependencies installed on the AWS-linux fixes the error listed above, but only to give me an error even more opaque:

/lib64/libc.so.6: version `GLIBC_2.25' not found (required by /opt/ruby/lib/libruby.so.2.5)

Upvotes: 6

Views: 2578

Answers (2)

cesartalves
cesartalves

Reputation: 1585

The problem is precisely as you stated, to summarize it: native extension gems need to be built in the same environment as they will be run. Therefore, before uploading the vendor to aws, we must install the gems in an environment which is similar to that of lambda.

To build your vendor/bundle in a way compliant with lambda, use the following docker container:

docker run --rm -v "$PWD":/var/task lambci/lambda:build-ruby2.7 bundle install --deployment

This will give you a working vendor/bundle dependency.

You can run another container to check if the function works:

docker run --rm -v $PWD:/var/task:ro,delegated lambci/lambda:ruby2.7 lambda_handler.lambda_handler

The full list of images is here: https://github.com/lambci/docker-lambda

The question's already 1 year old but I had to search for about 2 hours to solve the exact same problem with the gem nokogiri, so I thought this might be useful to somebody.

Hope this spares you from some googling hours!

Upvotes: 6

Zags
Zags

Reputation: 41328

Running ldd on gems/oj-2.18.5/lib/oj/oj.so clarifies the first error. the problem is not that oj.so does not exist, but that libruby.so.2.5 does not exist. The second problem is that the current Ruby lambda has glibc version 2.17, where AWS-linux comes with glibc version 2.25.

The fundamental problem here is that you need to install your gems on a system identical to the one they will run on if you have native dependencies. The best way I've found to do this is using docker. https://github.com/lambci/docker-lambda has docker images of lambda installs.

For ruby, do the following:

  1. Build the docker ruby image
  2. Copy your gemfile into a pristine directory
  3. From that directory, run: docker run -v "$PWD":/var/task --entrypoint bundle lambci/lambda-base:ruby2.5 install --path=/var/task

This will give you a folder called ruby with a version dependencies compatible with lambda.

If you are planning to use this output with lambda layers, keep in mind that the file structure produced by bunlde is ruby/2.5.0/... and it needs need to be ruby/gems/2.5.0/... before you upload it.

Upvotes: 10

Related Questions