CODEWITHSUNDEEP
javascriptnode.jsaws-lambdapuppeteer
TheCat
TheCat

Reputation: 111

AWS Lambda Puppeteer

Since 2 weeks in lost time I try to install Puppeteer on AWS Lambda without success.

I have try: https://github.com/sambaiz/puppeteer-lambda-starter-kit and https://github.com/deathemperor/puppeteer-lambda-starter-kit

My final code is: https://github.com/sambaiz/puppeteer-lambda-starter-kit

Replace index.js: https://github.com/sambaiz/puppeteer-lambda-starter-kit/blob/master/src/index.js

By: https://github.com/deathemperor/puppeteer-lambda-starter-kit/blob/master/src/index.js

Also, i'm on windows 7 so to build the package I remove/change a lot of stuff on the package.json for scripts sections. I have create package with and without babel and lint. Also, I have try with different version of puppeteer and chronium.

Someone suggest me to fix the version of puppeteer to 1.1.1 without success. See( TheCat and cirdes ): https://github.com/GoogleChrome/puppeteer/issues/323

I always get this error on aws:

{
    "errorMessage": "Failed to launch chrome! spawn /tmp/headless_shell ENOENT\n\n\nTROUBLESHOOTING: [...]",
    "errorType": "Error",
    "stackTrace": [
        "",
        "",
        "TROUBLESHOOTING:[..]",
        "",
        "onClose (/var/task/node_modules/puppeteer/lib/Launcher.js:299:14)",
        "ChildProcess.helper.addEventListener.error (/var/task/node_modules/puppeteer/lib/Launcher.js:290:64)",
        "emitOne (events.js:116:13)",
        "ChildProcess.emit (events.js:211:7)",
        "Process.ChildProcess._handle.onexit (internal/child_process.js:196:12)",
        "onErrorNT (internal/child_process.js:372:16)",
        "_combinedTickCallback (internal/process/next_tick.js:138:11)",
        "process._tickDomainCallback (internal/process/next_tick.js:218:9)"
    ]
}

Config AWS: I use "Upload a file from Amazon S3" option because it always finish by timeout with the UI and same thing for CLI command.

Runtime: Node.js 8.10

Handler: index.handler

Executable role: lambda_basic_execution. I have also try with a custom role who have full access on lambda and S3 just in case.

TimeOut: 30 sec

Memory: 3008 mb.

If someone can guide me a little bit.

Upvotes: 4

Views: 5118

Answers (3)

Robert K.
Robert K.

Reputation: 1073

I finally managed to make it worked and described the solution here, hope it's helpful: https://konarskis.substack.com/p/puppeteer-aws-lambda

The key was to match the versions of puppeteer and chromium, and download the chromium binary at runtime as opposed to trying to put it inside the Lambda itself. This way, we don't need any layers, external dependencies, or other configuration changes.

Upvotes: 0

PeteW
PeteW

Reputation: 731

I have been down this painful road too and would suggest looking at Google Cloud Functions, because Google Cloud Functions installs the NPM packages from the package.json file rather than you having to install them locally and upload the node_modules directory (which is what blows the AWS 50MB limit).

You can do something like:

gcloud functions deploy screenshot --runtime nodejs8 --trigger-http --memory=2048MB --timeout=60 --project=xyz --region europe-west1

Upvotes: 0

TheCat
TheCat

Reputation: 111

I finally managed to deploy the sambaiz package. Also I updated the chronium to the lasted stable version( HeadlessChrome/68.0.3440.106 ) and last version of puppeteer ( 1.7.0 ).

https://www.dropbox.com/s/p4t7zod2nf97cwn/sambaiz-puppeteer.zip?dl=0

If you want to build your own package and you are on windows you can:

  1. Download: https://github.com/sambaiz/puppeteer-lambda-starter-kit
  2. Change package.json by mine:

    {
      "name": "puppeteer-lambda-starter-kit",
      "version": "1.1.2",
      "description": "Starter Kit for running Headless-Chrome by Puppeteer on AWS Lambda",
      "scripts": {
        "package": "npm run package-prepare",
        "package-prepare": "npm run babel && copy package.json dist && cd dist && npm config set puppeteer_skip_chromium_download true -g && npm install --production",
        "babel": "mkdir dist && \"./node_modules/.bin/babel\" src --out-dir dist",
        "local": "npm run babel && copy node_modules dist && node dist/starter-kit/local.js",
        "package-nochrome": "npm run package-prepare && cd dist && zip -rq ../package.zip ."
      },
      "dependencies": {
        "babel": "^6.23.0",
        "puppeteer": "^1.1.1",
        "tar": "^4.0.1"
      },
      "devDependencies": {
        "aws-sdk": "^2.111.0",
        "babel-cli": "^6.26.0",
        "babel-preset-env": "^1.6.0"
      }
    }
  1. Change the version of node in .babelrc to 8.10
  2. npm install babel ( if it's not already install )
  3. npm run package
  4. Copy chrome/headless_shell-67.0.3361.0.tar.gz to dist
  5. Rename dist/headless_shell-67.0.3361.0.tar.gz to headless_shell.tar.gz
  6. Zip the content of dist and you have your package ready to deploy

Upvotes: 5

Related Questions