CODEWITHSUNDEEP
expressamazon-s3aws-lambdaserverless-framework
sami_analyst
sami_analyst

Reputation: 1839

uploading image from lambda to s3 ( corrupted file )

I have created an node express app which is using multer to upload images to my "assets-in" s3 bucket. after the images have been uploaded to s3, an image processing lambda function takes the image and and is resizing it to another s3 bucket.

upload script:

var assetsBucketIn = new AWS.S3({
  accessKeyId     : "...",
  secretAccessKey : "...",
  region          : "...",
  params : {
    Bucket : "..."
  }
});

var uploadAssets     = multer({inMemory : true});
var imgFieldSettings = [
  {
    name     : 'uploadFiles',
    maxCount : 7
  }
];

router.post('/images', uploadAssets.fields(imgFieldSettings), (req, res, next) => {
  ...
  uploadPromises = [];
  for(let i = 0; i < req.files.uploadFiles.length; i++){
    let params = {
      Key  : path.join(srcPath, srcName),
      Body : req.files.uploadFiles[i].buffer
    };
    let uploadPromise = assetsBucketIn.upload(params).promise();
    uploadPromises.push(uploadPromise);
  }

  Promise.all(uploadPromises)
   .then(...).catch(...);

I can run this express app on the localhost without any problems, but when I'm running the express app by lambda using aws-serverless-express, I can't access the image (Access denied) and the image processing lambda functions gives me the follwoing log error:

TypeError: Cannot read property 'width' of undefined at gm. (/var/task/index.js:103:50) at emitMany (events.js:127:13) at gm.emit (events.js:201:7) at gm. (/var/task/node_modules/gm/lib/getters.js:70:16) at cb (/var/task/node_modules/gm/lib/command.js:322:16) at ChildProcess.onExit (/var/task/node_modules/gm/lib/command.js:305:9) at emitTwo (events.js:106:13) at ChildProcess.emit (events.js:191:7) at maybeClose (internal/child_process.js:886:16) at Socket. (internal/child_process.js:342:11)

gm is the GraphicsMagick and ImageMagick module for node

I guess the problem is in the express app, which I'm running on aws lambda using serverless. Maybe its a problem with the multer settings, which are getting in conflict with the lambda environment ? my assetsBucket has the "AmazonS3FullAccess" role by the way.

UPDATE:

Actually I just found out that the problem is not the acces permission, It's ok, that the public access gets denied (sry for that missleading information). When I download the image, which gets uploaded to my assets bucket, I can't open it and getting the following error code:

Error interpreting JPEG image file (Not a JPEG file: starts with 0xef 0xbf)

But I'm getting this error only if I'm uploading the pictures with aws lambda (serverless), when I am running the upload process locally, everything seems to be fine with the file, so somehow the file gets corrupted when uploading via lambda

Upvotes: 4

Views: 5108

Answers (1)

Juan Zapata
Juan Zapata

Reputation: 2026

In my case, it was API Gateway fault, it turns out you have to enable binary support, otherwise, the lambda function will never receive the binary content:

  1. Go to settings of you ApiGateway endpoint
  2. Go to Binary Media Types
  3. Add new binary type and put: image/jpg

Also if your Lambda function is behind a VPC and a security group, first check if it works without VPC first.

Upvotes: 9

Related Questions