Running 'git' in AWS lambda

Question

I am trying to run git in AWS lambda to make a checkout of a repository.

This is my setup:

• I am using nodejs 4.3
• I am not using nodegit because I want to use the "--depth=1" parameter, which is not supported by nodegit.
• I have copied the git and ssh executable from the correct AWS AMI and placed then in a "bin" folder in the zip I upload.
• I added them to PATH with this:

->

process.env['PATH'] = process.env['LAMBDA_TASK_ROOT'] + "/bin:" + process.env['PATH'];

The input variables are set like this:

"checkout_url": "git@...",
"branch":"master

Now I do this (for brevity, I mixed some pseudo-code in):

downloadDeploymentKeyFromS3Sync('/tmp/ssh_key');
fs.chmodSync("/tmp/ssh_key",0600);
process.env['GIT_SSH_COMMAND'] = 'ssh -o StrictHostKeyChecking=no -i /tmp/ssh_key';
execSync("git clone --depth=1 " + checkout_url + " --branch " + branch + " /tmp/checkout");

Running this in my local computer using lambda-local everything works fine! But when I test it in lambda, I get:

warning: templates not found /usr/share/git-core/templates
PRIV_END: seteuid: Operation not permitted\r
fatal: Could not read from remote repository.

• The "warning" is of course, because I did not install git but just copied the binary. Is that a reason why this should not work?
• Why is git needing "setuid"? I read that in some shells, that is disabled for security reasons. So it makes sense that it does not work in lambda. Can git somehow be instructed to not "need" this command?

Answer 1

Yep, this is definitely possible, I've created a Lambda Layer that achieves just this. No need to mess with any env variables, should work out of the box:

https://github.com/lambci/git-lambda-layer

As stated in the README, all you need to do is add a layer with the following ARN:

arn:aws:lambda:<region>:553035198032:layer:git:<version>

(replace <region> and <version>, check README for latest version)

Answer 2

You might consider this a non-answer, but I've found the easiest way to run arbitrary binaries from Lambda is... not to. If I cannot do the work from within a platform-independent, non-binary approach, I integrate Docker into the workflow, managing Docker containers from the Lambda function.

On AWS one way to do this is to use the Elastic Container Service (ECS) to spawn a task that runs git.

If you stand up a Docker Swarm instance or integrate another Docker-API compatible service such as Rackspace Carina or Joyent's Triton, then you could use a project I personally put together specifically for integrating AWS Lambda with Docker: "Dockaless".

Good luck!

Answer 3

The issue is that you cannot copy just the git binary. You need a portable version of git and even with that you're going to have a bad time because you cannot guarantee that the os the lambda function runs on is going to be compatible with the binary.

Stepping back, I would just walk away from this approach completely. I would clone and build a package that I would just download pretty much the same way you do downloadDeploymentKeyFromS3Sync.