CODEWITHSUNDEEP
amazon-web-servicesamazon-s3aws-lambdaaws-sdkaws-sdk-js
7evam
7evam

Reputation: 177

Initiating Multipart Upload to S3 with AWS Javascript SDK

I am trying to initiate a multipart upload to a S3 bucket with a lambda function and eventually upload a large file to the bucket in chunks with pre-signed URLs. I've already successfully uploaded smaller files to my bucket with presigned URLs but when I try to initialize a multipart upload I get an error "Task timed out after 3.00 seconds". From what I can tell looking at documentation and other SO answers I should have everything I need here. Is there anything I'm missing here or is there a way to get an error message instead of a timeout from Lambda?

Here is my lambda function to initiate an upload

const AWS = require('aws-sdk') // lambda includes aws-sdk

const s3 = new AWS.S3({
    accessKeyId: process.env.accessKey,
    secretAccessKey: process.env.secretAccessKey
});

exports.startUpload = function(event,context,callback){
    console.log('at start upload') // this outputs fine in logs
    const s3Params = {
        Bucket: 'bucket-name',
        ContentType: event.type,
        Key: event.name
    }
    console.log('here are params')
    console.log(s3Params) // this outputs as I would expect in logs
    s3.createMultipartUpload(s3Params, function(error,response){
        if(error) callback(error)
        console.log('here is response') // logs never get here
        console.log(response)
        callback(null, response)
    })

And here is the permission policy on the IAM user I created specifically for uploading files

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "s3:PutObject",
                "s3:PutObjectAcl",
                "s3:GetObject",
                "s3:GetObjectAcl",
                "s3:ListBucketMultipartUploads",
                "s3:AbortMultipartUpload",
                "s3:PutObjectVersionAcl",
                "s3:DeleteObject",
                "s3:ListMultipartUploadParts"
            ],
            "Resource": [
                "arn:aws:s3:::bucket-name/*"
            ]
        }
    ]
}

And here is the CORS configuration set on the S3 bucket itself

<?xml version="1.0" encoding="UTF-8"?>
<CORSConfiguration xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
<CORSRule>
    <AllowedOrigin>*</AllowedOrigin>
    <AllowedMethod>HEAD</AllowedMethod>
    <AllowedMethod>PUT</AllowedMethod>
    <AllowedMethod>POST</AllowedMethod>
    <AllowedMethod>GET</AllowedMethod>
    <MaxAgeSeconds>30000</MaxAgeSeconds>
    <AllowedHeader>*</AllowedHeader>
</CORSRule>
</CORSConfiguration>

Upvotes: 2

Views: 1501

Answers (1)

7evam
7evam

Reputation: 177

Well I thought I had a problem with using the AWS Javascript SDK but it was just a problem with my Lambda function not being connected to the internet since it was set up in a VPC with no public subnet. Fixed this issue by creating a new Lambda function not connected to any of my own VPCs since by default Lambda functions are launched in an AWS managed VPC with internet access. Leaving this here if anyone wants to see a working configuration of initiating a S3 multipart upload in JS. Big thanks to Jarmod for helping

Upvotes: 1

Related Questions