CODEWITHSUNDEEP
pythonamazon-web-servicesfileaws-lambda
Jérémie RPK
Jérémie RPK

Reputation: 317

AWS Lambda Python 3 handle POST file

I'm working with Python 3.8 on AWS Lambda, and I would like to handle posted file, like requests.files with Django. But it's impossible with AWS Lambda. I want to put this file in the S3 with :

s3.put_object(
    Body=fileAsString,
    Bucket=MY_BUCKET,
    Key='my-file.jpg'
)

When I return directly the received event, with the file :

enter image description here

First try : with cgi.parse_multipart : I my handler :

c_type, c_data = parse_header(event['headers']['Content-Type'])
boundary = c_data['boundary'].encode('latin1')
body = event['body'].encode('utf8')

fp = BytesIO(body)
pdict = {
    'boundary': boundary,
    'CONTENT-LENGTH': str(len(body))
}
form_data = parse_multipart(fp, pdict)
fileBytes: bytes = form_data['file'][0]
return ({'statusCode': 200, 'body': json.dumps(str(fileBytes))}

I receive :enter image description here I also tried with form_data['file'][0].decode('utf-8') but I receive : enter image description here and I have always the "?"

I should get this, because it's the original image, opened in edition : enter image description here

Second try : I follow this tutorial : https://medium.com/swlh/upload-binary-files-to-s3-using-aws-api-gateway-with-aws-lambda-2b4ba8c70b8e

So, I tried :

file_content = base64.b64decode(event['body'])

but I get :

string argument should contain only ASCII characters

(the same with validate=False) and when I try to load body as dict with :

body2 = json.loads(event['body'])

I get :

loads failed : Expecting value: line 1 column 1 (char 0)

I tried to add "image/png" and "image/*" to "Binary Media Types" in the settings of the API, but no change.

Any idea ?

Upvotes: 1

Views: 3347

Answers (5)

Awwal15
Awwal15

Reputation: 11

I solved this easily just by adding multipart/form-data to Binary media type an i handled the file extensions in my code to disallow some file types

Upvotes: 0

Sebgenie
Sebgenie

Reputation: 1

The solution for me was to do:

event['body'] = str(event['body']).encode('utf-8)

Upvotes: 0

user1773219
user1773219

Reputation: 21

upload_file = request.files['file']
img = re.sub(b"\xef\xbf\xbd", b'', upload_file.stream.read())

when i upload to the flask, the AWS Lambda add UTF-8 BOM,this is just a trick to cleanup utf-8 bom bytes, it works fine for me.

split-function-add-xef-xbb-xbf-n-to-my-list

Upvotes: 0

redPanda
redPanda

Reputation: 797

This is an old post but I was going crazy trying to solve it.

It turns out that the solution for me was to add ALL binary types to the API Gateway as so: AWS API Gateway settings

My issue was that my code was working for a .png file but not a .jpg file. Once the correct MIME type was added "image/jpeg" all worked OK and the Error

string argument should contain only ASCII characters

went away. NOTE: that the API needs to be re-deployed after this change (which is not instantaneous)

Bottom line: this code is my working code:

import json
import boto3
import base64
  
s3 = boto3.client('s3')

def lambda_handler(event, context):
if event['httpMethod'] == 'POST' : 
    fileName = event['multiValueQueryStringParameters']['fileName']
    bodyData = event['body']
    decodedFile = base64.b64decode(bodyData)
    s3.put_object(Bucket=<BUCKET_NAME>, Key=fileName[0], Body=decodedFile)
    return {'statusCode': 200, 
        'body': json.dumps({'message': 'file saved successfully'}), 
        'headers': {'Access-Control-Allow-Origin': '*'}}

Upvotes: 4

msc
msc

Reputation: 489

When you use requests.files, it will upload images as multipart. It's a standard way to upload multiple content-type files at once but you'll need to parse it when receive. If your application is not sensitive with content-type, just use base64 enc/dec to post and receive like below pseudo code:

Uploader:

import json
import requests
import base64

with open("/path/to/your/file/image.png", "rb") as image_file:
        encoded_string = base64.b64encode(image_file.read())

    payload = {
                'name': 'image.png',
                'file': encoded_string.decode('utf-8')
                }

    url = 'https://api_url/upload'
    r = requests.post(url, data = json.dumps(payload), headers = {})

Lambda side:

import json
import boto3
import base64

def upload(event, context):
    body = json.loads(event['body'])

    s3 = boto3.client("s3", region_name='us-east-1')

    decodedFile = base64.b64decode(body['file'])
    rs = s3.put_object(
                        Bucket='bucket_name',
                        Key=body['name'],
                        Body=decodedFile
                        )

    response = {
        "statusCode": 200,
    }

    return response

Upvotes: 0

Related Questions