AWS Cloudformation | How to Configure Lambda to Use Latest Code in S3 Bucket

Question

---

Tests3bucketLambda: 
  Type: "AWS::Lambda::Function"
  Properties: 
    Code: 
      S3Bucket: TestS3Bucket
      S3Key: Tests3.zip
   FunctionName: "test-lambda-function"
   Handler: lambda-function-s3.lambda_handler
   Role: !GetAtt LambdaExecutionRole.Arn
   Runtime: python3.6
  


 

Issue: When I update the new code that is zipped and uploaded to the S3 bucket during code build, but the change is not deployed to the existing lambda functions.

Answer 1

This is a bit old, but this is in need of a concrete answer for those who are starting off.

Oleksii's answer is a correct guideline. However, The way to implement Option 2 would be as follows.

I used Java, but the same logic can apply to python too.
In your case imagine your cloud formation template for lambda that you pasted is named as cloud_formation_lambda.yml

Now in the code build stage where you are preparing this artifact that you mention Tests3 in your case. Prepare it with a unique identifier appended such as the epoch.

Then all you need to do in either the build phase or post-build phase is to use some simple linux command to accommodate those name changes.

1. The first step would be to rename your built artifact to append the unique value such as epoch
2. Use a sed command to replace the occurrence of Tests3 in your cloud formation template

Thus the buildspec.yml that implements this will be as follows

phases:
  install:
    runtime-versions:
      java: corretto17
  build:
    commands:
      - echo $(date +%s) > epoch.txt
      - mvn package
  post_build:
    commands:
      - mv target/Tests3.jar target/Tests3-$(cat epoch.txt).jar
      - sed -i "s/Tests3.jar/Tests3-$(cat epoch.txt).jar/g" cloud_formation_lambda.yml
artifacts:
  files:
    - target/Tests3-$(cat epoch.txt).jar
    - cloud_formation_lambda.yml

Answer 2

Expanding on Oleksii's answer, I'll just add that I use a Makefile and an S3 bucket with versioning to handle this issue. A version-enabled S3 bucket creates a new object and a new version number every time a modified file is uploaded (keeping all the old versions and their version numbers). If you don't want a dependency on make in your build/deploy process this won't be of interest to you.

Make can examine the filesystem and look for updated files to trigger a target action based an updated file (as a dependency). Here's a Makefile for a simple stack with one lambda function. The relevant parts of the Cloudformation (CFN) file will be shown below.

.DEFAULT_GOAL := deploy

# Bucket must exist and be versioning-enabled
lambda_bucket = lambda_uploads

deploy: lambda.zip
    @set -e                                                                       ;\
    lambda_version=$$(aws s3api list-object-versions                               \
        --bucket $(lambda_bucket) --prefix lambda.zip                              \
        --query 'Versions[?IsLatest == `true`].VersionId | [0]'                    \
        --output text)                                                            ;\
    echo "Running aws cloudformation deploy with ZIP version $$lambda_version..." ;\
    aws cloudformation deploy --stack-name zip-lambda                              \
        --template-file test.yml                                                   \
        --parameter-overrides LambdaVersionId=$$lambda_version                     \
        --capabilities CAPABILITY_NAMED_IAM

lambda.zip: lambda/lambda_func.js
    @zip lambda.zip lambda
    @aws s3 cp lambda.zip s3://$(lambda_bucket)

The deploy target has a dependency on the lambda.zip target which itself has a dependency on lambda_func.js. This means that the rule for lambda.zip must be valid before the rule for deploy can be run.

So, if lambda_func.js has a timestamp newer than the lambda.zip file, an updated zip file is created and uploaded. If not, the rule is not executed because the lambda function has not been updated.

Now the deploy rule can be run. It:

1. Uses the AWS CLI to get the version number of the latest (or newest) version of the zip file.
2. Passes that version number as a parameter to Cloudformation as it deploys the stack, again using the AWS CLI.

Some quirks in the Makefile:

• The backslashes and semicolons are required to run the deploy rule as one shell invocation. This is needed to capture the lambda_version variable for use when deploying the stack.
• The --query bit is an AWS CLI capability used to extract information from JSON data that has been returned from the command. jq could also be use here.

The relevant parts of the Cloudformation (YAML) file look like this:

AWSTemplateFormatVersion: 2010-09-09
Description: Test new lambda ZIP upload

Parameters:

  ZipVersionId:
    Type: String

Resources:

  ZipLambdaRole: ...

  ZipLambda:
    Type: AWS::Lambda::Function
    Properties:
      FunctionName: zip-lambda
      Role: !GetAtt ZipLambdaRole.Arn
      Runtime: nodejs16.x
      Handler: index.handler
      Code:
        S3Bucket: lambda_uploads
        S3Key: lambda.zip
        S3ObjectVersion: !Ref ZipVersionId
      MemorySize: 128
      Timeout: 3

The zip file is uniquely identified by S3Bucket, S3Key, and the S3ObjectVersion. Note that, and this is important, if the zip file was not updated (the version number remains the same as previous deploys) Cloudformation will not generate a change set--it requires a new version number to do that. This is the desired behavior--there is no new deploy unless the lambda has been updated.

Finally you'll probably want to put a lifecycle policy on the S3 bucket so that old versions of the zip file are periodically deleted.

These answers to other questions informed this answer.

Answer 3

If you want the lambda to automatically pickup the latest code then it is not possible by using cloud formation.

To do that you could synch the code file to a s3 bucket and then try the approach mentioned here How can AWS Lambda pick the latest versions of of the script from S3 . I was able to achieve it and have mentioned the solution there.

Answer 4

If you deploy new code to the object with the same key, CF will not treat it like change, since template itself hasn't been modified. There are few ways to mitigate this.

1. Use bucket versioning and provide object version along with object key: https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-lambda-function-code.html

   Code: 
     S3Bucket: TestS3Bucket
     S3Key: Tests3.zip
     S3ObjectVersion: blablabla....
   

2. Modify your object key on each deployment, with timestamp for example

   Code: 
     S3Bucket: TestS3Bucket
     S3Key: Tests3_2021-05-06T17:15:55+00:00.zip
   

3. Use automated tools like Terraform or AWS CDK to take care of these things