CODEWITHSUNDEEP
python-3.xamazon-web-servicesamazon-s3
FlyingZebra1
FlyingZebra1

Reputation: 1346

Python Boto3 - upload images to S3 in one put request

I have a script in python built out, that uploads images to s3, one image|put request at a time. Is it possible to upload all images to s3 at the same time, using one put request, to save $$ on requests?

for image_id in list_of_images:
#upload each image
    filename = id_prefix+"/"+'{0}.jpg'.format(image_id)
    s3.upload_fileobj(buffer, bucket_name, filename, ExtraArgs={ "ContentType": "image/jpeg"})

Upvotes: 0

Views: 179

Answers (2)

FlyingZebra1
FlyingZebra1

Reputation: 1346

Good news - it looks like several companies came out with boto3-like apis, with much better storage + download (per gb) pricing.

As of a few days ago - Backblaze came out with S3 compatible storage + API .

We did a few tests on our application, and everything seems to be working as advertised!

Upvotes: 0

John Rotenstein
John Rotenstein

Reputation: 270134

No.

The Amazon S3 API only allows creation of one object per API call.

Your options are to loop through each file (as you have done) or, if you wish to make it faster, you could use multi-threading to upload multiple files simultaneously to take advantage of more networking bandwidth.

If your desire is simply to reduce requests costs, do not panic. It is only $0.005 per 1000 requests.

Upvotes: 1

Related Questions