CODEWITHSUNDEEP
amazon-web-servicesnpmaws-lambda
Fook
Fook

Reputation: 5550

How to load npm modules in AWS Lambda?

I've created several Lambda functions using the web based editor. So far so good. I'd now like to start extending those with modules (such as Q for promises). I can't figure out how to get the modules out to Lambda so they can be consumed by my functions.

I've read through Using Packages and Native nodejs Modules in AWS Lambda but it seems to involve setting up an EC2 and running Lambda functions from there. There is a mechanism to upload a zip when creating a function but that seems to involve sending up functions developed locally. Since I'm working in the web based editor that seems like a strange workflow.

How can I simply deploy some modules for use in my Lambda functions?

Upvotes: 261

Views: 266516

Answers (10)

The_Real_Epk
The_Real_Epk

Reputation: 181

The AWS CDK has a nice solution for this. There is a lambda construct called NodejsFunction. It uses esbuild to package your code with its dependencies into a single asset which is uploaded as a zip when you 'deploy' the application.

You basically just have to give a path to the file that contains your handler function as follows:

    const { Stack } = require('aws-cdk-lib');
    const lambdaNodeJs = require('aws-cdk-lib/aws-lambda-nodejs');
    const path = require("path");
    
    class CdkStack extends Stack {
        constructor(scope, id, props) {
            super(scope, id, props);
    
            const myLambda = new lambdaNodeJs.NodejsFunction(this, "my-lambda-id", {
                functionName: 'my-function-name',
                entry: path.join(path.join(__dirname, '/../../path/to/handler.js')),
                handler: 'handler-function-name'
                // rest of the config.
            });
        }
    }

Make sure that you have the esbuild as a dependency in the package.json file for the lambda code and you are good to go.

Upvotes: 3

David Liao
David Liao

Reputation: 875

Deploying your lambda function using AWS CDK might be a good choice if you have multiple AWS resources to manage. It's an IaC tool that deploys AWS resources and uses CloudFormation. To set up CDK see Getting started with the AWS CDK.

Once you have CDK set up, you can create a Function construct and use its lambda.Code.fromAsset to easily bundle up your source code, including node_modules, and use it as the code for the Function.

import * as lambda from "aws-cdk-lib/aws-lambda";
import * as path from "path";

export class MyStack extends cdk.Stack {
  constructor(scope: Construct, id: string, props?: cdk.StackProps) {

    const lambdaFunc = new lambda.Function(this, "lambdaFunc", {
      code: lambda.Code.fromAsset(path.join(__dirname, "..", "..", "project-containing-node-modules")),
      ...
    });

  }
}

Then use cdk deploy to deploy it to your AWS account.

Lambda layers are also supported in CDK if you are using the same node_modules across multiple Lambdas and don't want to upload duplicate dependencies.

Upvotes: 4

Sunil Chauraha
Sunil Chauraha

Reputation: 555

npm module has to be bundeled inside your nodejs package and upload to AWS Lambda Layers as zip, then you would need to refer to your module/js as below and use available methods from it.

const mymodule = require('/opt/nodejs/MyLogger');

Upvotes: 3

Toai
Toai

Reputation: 149

Hope this helps, with Serverless framework you can do something like this:

  1. Add these things in your serverless.yml file:
    plugins:
      - serverless-webpack
    custom:
      webpackIncludeModules:
        forceInclude:
          - <your package name> (for example: node-fetch)
  1. Then create your Lambda function, deploy it by serverless deploy, the package that included in serverless.yml will be there for you.

For more information about serverless: Setting Up Serverless Framework With AWS

Upvotes: 12

Eldad Hauzman
Eldad Hauzman

Reputation: 699

You can now use Lambda Layers for this matters. Simply add a layer containing the package you need and it will run perfectly.

Follow this post: NodeJS Runtime Environment with AWS Lambda Layers

Upvotes: 59

cfstras
cfstras

Reputation: 1858

After fiddling around with parcel for a few hours, I found that it seems to make some assumptions about running in a browser (even if I tell it to use engine: node).

Instead:

esbuild

Much easier, and also faster, is esbuild!

Simply run npm add --save-dev esbuild, and add these scripts to your package.json:

{
  ...
  "scripts": {
    "build": "esbuild --bundle --minify --platform=node --target=node12 --outdir=build main.js",
    "export": "cd build && zip main.js.zip main.js"
  },
  ...
  "devDependencies": {
    "esbuild": "^0.11.19",
    ...
  }
}

This allowed me to use the aws-sdk while still getting tree-shaking and minifying, while still being able to install other dependencies such as jest and eslint without having to package the whole node_modules folder.

To build a package within CI, simply: npm ci && npm run build && npm run export

The file build/main.js.zip will then contain everything you need!

Upvotes: 10

ConcernedHobbit
ConcernedHobbit

Reputation: 874

This is an old-ish question, but it helped lead me to a really easy way of adding new Lambda dependencies to an Alexa skill.

Like JohnAllen's answer, you need to create a folder on your local machine, title it whatever you want (it's arbitrary):

mkdir lambdaFunc
cd lambdaFunc

Once in your folder, use npm to install the necessary package. For me, I needed to parse ISO8601 durations (my command was npm install iso8601-duration):

npm install <your-package-here>

Once installed, back out of that directory, and zip it. Open up your Alexa Skill in the Alexa Skill developer console, then select the "Import Code" option. From here, you'll upload your .zip file, and select all the code:

Import Lambda Code in Alexa Skill Developer Console

Select all code in .zip file

That's it! Then you can just import the code, like I did:

const DateConverter = require('iso8601-duration');

Upvotes: 1

ElKePoN
ElKePoN

Reputation: 930

Also in the many IDEs now, ex: VSC, you can install an extension for AWS and simply click upload from there, no effort of typing all those commands + region.

Here's an example:

enter image description here

Upvotes: 7

Yarin
Yarin

Reputation: 183899

A .zip file is required in order to include npm modules in Lambda. And you really shouldn't be using the Lambda web editor for much of anything- as with any production code, you should be developing locally, committing to git, etc.

MY FLOW:

1) My Lambda functions are usually helper utilities for a larger project, so I create a /aws/lambdas directory within that to house them.

2) Each individual lambda directory contains an index.js file containing the function code, a package.json file defining dependencies, and a /node_modules subdirectory. (The package.json file is not used by Lambda, it's just so we can locally run the npm install command.)

package.json:

{
  "name": "my_lambda",
  "dependencies": {
    "svg2png": "^4.1.1"
  }
}

3) I .gitignore all node_modules directories and .zip files so that the files generated from npm installs and zipping won't clutter our repo.

.gitignore:

# Ignore node_modules
**/node_modules

# Ignore any zip files
*.zip

4) I run npm install from within the directory to install modules, and develop/test the function locally.

5) I .zip the lambda directory and upload it via the console.

(IMPORTANT: Do not use Mac's 'compress' utility from Finder to zip the file! You must run zip from the CLI from within the root of the directory- see here)

zip -r ../yourfilename.zip *

NOTE:

You might run into problems if you install the node modules locally on your Mac, as some platform-specific modules may fail when deployed to Lambda's Linux-based environment. (See https://stackoverflow.com/a/29994851/165673)

The solution is to compile the modules on an EC2 instance launched from the AMI that corresponds with the Lambda Node.js runtime you're using (See this list of Lambda runtimes and their respective AMIs).

See also AWS Lambda Deployment Package in Node.js - AWS Lambda

Upvotes: 68

JohnAllen
JohnAllen

Reputation: 7531

You cannot load NPM modules without uploading a .zip file, but you can actually get this process down to two quick command lines.

Here's how:

  1. Put your Lambda function file(s) in a separate directory. This is because you install npm packages locally for Lambda and you want to be able to isolate and test what you will upload to Lambda.

  2. Install your NPM packages locally with npm install packageName while you're in your separate Lambda directory you created in step #1.

  3. Make sure your function works when running locally: node lambdaFunc.js (you can simply comment out the two export.handler lines in your code to adapt your code to run with Node locally).

  4. Go to the Lambda's directory and compress the contents, make sure not to include the directory itself. 

    zip -r lambdaFunc.zip .

  5. If you have the aws-cli installed, which I suggest having if you want to make your life easier, you can now enter this command: 

    aws lambda update-function-code --function-name lambdaFunc \
--zip-file fileb://~/path/to/your/lambdaFunc.zip

    (no quotes around the lambdaFunc part above in case you wonder as I did)

  6. Now you can click test in the Lambda console.

  7. I suggest adding a short alias for both of the above commands. Here's what I have in mine for the much longer Lambda update command:

    alias up="aws lambda update-function-code --function-name lambdaFunc \
--zip-file fileb://~/path/to/your/lambdaFunc.zip"

Upvotes: 317

Related Questions