Image uploaded to S3 becomes corrupted

Question

I’m having issues uploading a file from postman to aws lambda + s3. If I understand correctly the image has to be a base64 string and send via JSON to work with lambda and API Gateway so I converted an image to a base64 and I’m using the base64 string in postman

The file uploads to S3, but when I download the s3 object and open it I get

So I don’t think I’m uploading it correctly. I’ve used a base64 to image converter and the image appears so the base64 string is correct before sending it via postman so something in my setup is off. What am I doing wrong? I appreciate the help!

upload.js

const AWS = require('aws-sdk');
const s3 = new AWS.S3();
exports.handler = async (event, context, callback) => {
    let data = JSON.parse(event.body);
    let file = data.base64String;


    const s3Bucket = "upload-test3000";
    const objectName = "helloworld.jpg";
    const objectData = data.base64String;
    const objectType = "image/jpg";
    try {
        const params = {
            Bucket: s3Bucket,
            Key: objectName,
            Body: objectData,
            ContentType: objectType
        };
        const result = await s3.putObject(params).promise();
        return sendRes(200, `File uploaded successfully at https:/` + s3Bucket + `.s3.amazonaws.com/` + objectName);

    } catch (error) {
        return sendRes(404, error);
    }
};
const sendRes = (status, body) => {
    var response = {
        statusCode: status,
        headers: {
            "Content-Type": "application/json",
            "Access-Control-Allow-Headers": "Content-Type,X-Amz-Date,Authorization,X-Api-Key,X-Amz-Security-Token",
            "Access-Control-Allow-Methods": "OPTIONS,POST,PUT",
            "Access-Control-Allow-Credentials": true,
            "Access-Control-Allow-Origin": "*",
            "X-Requested-With": "*"
        },
        body: body
    };
    return response;
};

.png

Answer 1

This is occur due to inappropriate UPLOAD of file.

The best way to solve this problem is to use standard methods. Means use XMLHttpRequest in place of axios , and use BLOB of file which we fetch from the URL.

this is the method which is work for me. I hope this is also helpful to you.

const uploadFile = async (fileObject) => {

function uriToBlob(uri) {
    return new Promise((resolve, reject) => {
        const xhr = new XMLHttpRequest();
        xhr.onload = function () {
            resolve(xhr.response);
        };
        xhr.onerror = function () {
            reject(new Error('uriToBlob failed'));
        };
        xhr.responseType = 'blob';
        xhr.open('GET', uri, true);
        xhr.send(null);
    });
}

try {
    const res = await axios.get(`${baseURL}/uploadUrl`, {
        params: {
            contentType: fileObject.photoObject.type,
            type: fileObject.type
        }
    })

    const { url } = res.data;

    // console.log(fileObject.photoObject);

    const file = await uriToBlob(fileObject.photoObject.uri);

    const xhr = new XMLHttpRequest();
    xhr.open('PUT', url);
    xhr.onreadystatechange = function () {
        if (xhr.readyState === 4) {
            if (xhr.status === 200) {
                console.log("uploded");
            } else {
                alert("Enable to upload the image! please try again.")
            }
        }
    };
    xhr.send(file);

    return { imageName: res.data.imageName };
}
catch (err) {
    alert("Enable to upload the image! please try again.");
    return;
}}

Answer 2

The problem might be the passing format of image in Body, as it is not getting passed as expected by s3.upload parameters (It says Body key must be Buffer to be passed).

So, The simple solution is pass the Body as buffer, if your file is present in any location in the directory then don't pass it like

// Wrong Way 

const params = {
   Bucket: 'Your-Buket-Name',
   Key: 'abc.png', // destFileName i.e the name of file to be saved in s3 bucket
   Body: 'Path-To-File'
}

Now, the problem is the file gets uploaded as Raw Text, which is corrupted format and will not be readable by the OS on downloading.

So, to get it working pass it like

// Correct Way According to aws-sdk library

const fs = require('fs'); 
const imageData = fs.readFileSync('Path-To-File'); // returns buffer
const params = {
   Bucket: 'Your-Buket-Name',
   Key: 'abc.png', // destFileName i.e the name of file to be saved in s3 bucket
   Body: imageData // image buffer
}
const uploadedFile = await s3.upload(params).promise();

Note: During answer i was using -> "aws-sdk": "^2.1025.0"

Hope this will help you or somebody else. Thanks!

Answer 3

I got it working by adding the base64 string in JSON format like so

and then sent

let decodedImage = Buffer.from(encodedImage, 'base64'); as the Body param.

updated upload.js

const AWS = require('aws-sdk');
var s3 = new AWS.S3();


exports.handler = async (event) => {

    let encodedImage = JSON.parse(event.body).base64Data;
    let decodedImage = Buffer.from(encodedImage, 'base64');
    var filePath = "user-data/" + event.queryStringParameters.username + ".jpg"
    var params = {
        "Body": decodedImage,
        "Bucket": process.env.UploadBucket,
        "Key": filePath
    };

    try {
        let uploadOutput = await s3.upload(params).promise();
        let response = {
            "statusCode": 200,
            "body": JSON.stringify(uploadOutput),
            "isBase64Encoded": false
        };
        return response;
    }
    catch (err) {
        let response = {
            "statusCode": 500,
            "body": JSON.stringify(eerr),
            "isBase64Encoded": false
        };
        return response;
    }

};

I found this article to be super helpful

Answer 4

When building the params you should add content encoding, otherwise you're just uploading the text data:

const params = {
  Bucket: s3Bucket,
  Key: objectName,
  Body: objectData,
  ContentType: objectType,
  ContentEncoding: 'base64'
};

edit

Okay I have checked the file, I think you might be misunderstanding what will happen when you store the image in base64.

Windows or a browser for that matter can't read a jpg file in base64 (as far as I know), it must be converted first. When you have an image in the browser with a base64 source, the browser handles this conversion on the fly but the base64 data inside the "helloworld.jpg" container is useless in windows without converting it.

There's two options, either convert once it reaches your server then upload directly as utf8 or have a layer in between, converting the image as it's requested.