CODEWITHSUNDEEP
amazon-web-servicesaws-lambdaaws-lambda-layers
David Gard
David Gard

Reputation: 12047

Lambda function package still large despite using a layer for dependencies

I have a Python Lambda and since I started using AWS X-Ray the package size has ballooned from 445KB to 9.5MB.

enter image description here

To address this and speed up deployments of my code, I have packaged my requirements separately and added a layer to my template. The documentation suggests that this approach should work.

Packaging dependencies in a layer reduces the size of the deployment package that you upload when you modify your code.

pip install --target ../package/python -r requirements.txt

Resources:
  ...

  ProxyFunction:
    Type: AWS::Serverless::Function
    Properties:
      Architectures:
        - x86_64
      CodeUri: proxy/
      Handler: app.lambda_handler
      Layers:
        - !Ref ProxyFunctionLibraries
      Role: !GetAtt ProxyFunctionRole.Arn
      Runtime: python3.8
      Tracing: Active

  ProxyFunctionLibraries:
    Type: AWS::Serverless::LayerVersion
    Properties:
      LayerName: proxy-function-lib
      Description: Dependencies for the ProxyFunction.
      ContentUri: package/.
      CompatibleRuntimes:
        - python3.8

However, this doesn't seem to have prevented the Lambda from still packaging everything in the top layer, and every time I deploy the package is still 9.5MB. The new layer for some reason is 11MB in size, but that is only being deployed when a change is made.

How can I reduce the size of the Lambda function package?

enter image description here

Upvotes: 3

Views: 1899

Answers (2)

HFinch
HFinch

Reputation: 626

I am struggling with the same problem and solved it differently. Leaving this answer as a note to future question-seekers.

My current lambda structure is the following:

├── events
│   └── event.json
├── ingress
│   ├── app.py # lambda code
│   └── __init__.py
├── __init__.py
├── lib_layer # contains self written helpers and python dependencies
│   ├── helper.py
│   └── requirements.txt
├── samconfig.toml
├── template.yaml
└── tests
    ├── ...

In my template.yaml I have the following code snippets:

SQSIngestion:
    Type: AWS::Serverless::Function
    Properties:
      CodeUri: ingress/
      Handler: app.lambda_handler
      Runtime: python3.9
      Layers:
        - !Ref PythonLibLayer
      Architectures:
        - x86_64

  PythonLibLayer:
    Type: AWS::Serverless::LayerVersion
    Properties:
      ContentUri: lib_layer
      CompatibleRuntimes:
        - python3.9
    Metadata:
      BuildMethod: python3.9

Since the Lambda Function already has the layer containing all dependencies + my helpers attached, the requirements.txt in /ingress can be omitted but works nevertheless when invoking the function in AWS. By calling sam build will automatically build the dependency layer and ignore the function folder for building the dependencies.

As a tip, if working with VSCode, add in your settings.json the following line to fix pylances import error for self written packages.

{
  "python.analysis.extraPaths": ["lib_layer"]
}

Upvotes: 0

David Gard
David Gard

Reputation: 12047

Actually the solution here was quite simple, although not obvious to non-Lambda experts.

As described in the question, the first step was to build the package library.

pip install --target ../package/python -r requirements.txt

However, when building the Lambda using sam build -u the same 'requirements.txt' file is used and the required dependencies were again being installed, this time as part of the app.

So all I had to do was remove the requirements that I wish packaged in a separate layer and rebuild. It does mean that I have to maintain 2x 'requirements.txt' but that is entirely manageable.

I've opened an issue and hopefully AWS will update their documentation.

Upvotes: 2

Related Questions