CODEWITHSUNDEEP
node.jscsvlambdaaws-api-gateway
Meet
Meet

Reputation: 243

AWS Api Gateway & CSV Upload Issue

I am getting some meta-information / junk in CSV in lambda function. I need to remove that junk. If I directly save the file to s3 the junk is included. Could anyone guide me on how to remove this?

----------------------------362648820336892682391117   ***// remove this***
Content-Disposition: form-data; name="file"; filename="Book1.csv" ***// remove this***
Content-Type: text/csv ***// remove this***

o;?name,age // remove this o;?
andy,33
hello,34
----------------------------362648820336892682391117--  ***// remove this***

I can also upload directly to s3 using pre-signed URL however, that is not what I am looking for.

const AWS = require('aws-sdk');

exports.handler = async (event) => {
    try {
            console.log(JSON.stringify(event, 2, null));
            const data = new Buffer(event.body, 'base64');
            const text = data.toString('ascii');
            const s3 = new AWS.S3();
            const params = {Bucket: 'bucket', Key: 'key', Body: text};
        const d = await s3.upload(params).promise();
        return {
            statusCode: 200,
            body: JSON.stringify('uploaded successfully'),
        };
    } catch (e) {
        return {
            statusCode: 200,
            body: JSON.stringify('uploaded successfully'),
        };
    }
};

Thanks

Upvotes: 1

Views: 613

Answers (1)

Josh Aguilar
Josh Aguilar

Reputation: 2281

I assume you are uploading the file using multipart/form-data. If so, you will need to do further processing of the request body. You can either do something very rudimentary like manually parsing the contents using regex or you could use a library like busboy which helps process HTML form data.

A quick example for your scenario could be something like this.

const Busboy = require('busboy');
const AWS = require('aws-sdk');

// This function uses busboy to process the event body and
// return an object containing the file data and other details.
const parseFile = (event) => new Promise((resolve, reject) => {
  let contentType = event.headers['content-type']
  if (!contentType) {
    contentType = event.headers['Content-Type'];
  }

  const busboy = new Busboy({ headers: { 'content-type': contentType } });
  const uploadedFile = {};
  busboy.on('file', (fieldname, file, filename, encoding, mimetype) => {
    file.on('data', data => {
      uploadedFile.data = data;
    });

    file.on('end', () => {
      uploadedFile.filename = filename;
      uploadedFile.contentType = mimetype;
    });
  });

  busboy.on('error', error => {
    reject(error);
  });

  busboy.on('finish', () => {
    resolve(uploadedFile);
  });

  busboy.write(event.body, event.isBase64Encoded ? 'base64' : 'binary');
  busboy.end();
});

exports.handler = async (event) => {
  // Use the parse function here
  const { data } = await parseFile(event);

  const s3 = new AWS.S3();
  const params = { Bucket: 'bucket', Key: 'key', Body: data };
  await s3.upload(params).promise();
  return {
    statusCode: 200,
    body: 'uploaded successfully',
  };
};

Upvotes: 1

Related Questions