CODEWITHSUNDEEP
pythonamazon-web-servicesamazon-s3boto3
Rishabh Jain
Rishabh Jain

Reputation: 469

Multiple image upload to amazon AWS using CLI

I want to upload 100 images to s3 bucket using CLI. Rather than uploading image one by one is there any other way to upload the the 100 images together? My code goes like this

import boto3
import sys
#s3-rish = argv[0]
filen = sys.argv[1]
s3 = boto3.resource('s3')
for bucket in s3.buckets.all():
   print(bucket.name)
#filen='messi.jpeg'
data = open(filen, 'rb')
x=s3.Bucket('bucket-rishabh').put_object(Key=""+filen, Body=data,Metadata=         {'FullName': sys.argv[2]})
x.Acl().put(ACL='public-read')
print x
# Works Properly enter code here
# Takes in image input as command line arguments first image path than image name'

What modifications can be made to these lines of code to 100 images input ?

Upvotes: 0

Views: 1628

Answers (2)

Mausam Sharma
Mausam Sharma

Reputation: 892

You can store all your images in one particular folder and then use this kind of python script to upload all images found in your folder :

import os
import boto3

s3 = boto3.resource('s3')

directory_in_str="E:\\streethack\\hold"   # change directory path to your images folder

directory = os.fsencode(directory_in_str)

for file in os.listdir(directory):
    filename = os.fsdecode(file)
    if filename.endswith(".jpeg") or filename.endswith(".jpg") or filename.endswith(".png"):

        strg=directory_in_str+'\\'+filename
        print(strg)     
        file = open(strg,'rb')
        object = s3.Object('bucketname',filename)
        object.put(Body=file,ContentType='image/jpeg')



    else:
        continue

Basically , this script will iterate over each file in your folder, if it's an image then it will upload that image to your bucket.

if you want custom names to be given to your images before uploading, you can use this with some modifications :

import os
import boto3

s3 = boto3.resource('s3')

directory_in_str="E:\\streethack\\hold"

directory = os.fsencode(directory_in_str)

for file in os.listdir(directory):
    filename = os.fsdecode(file)
    if filename.endswith(".jpeg") or filename.endswith(".jpg") or filename.endswith(".png"):

        strg=directory_in_str+'\\'+filename
        print(strg)
        print("Enter name for your image : ")
        inp_val = input()

        strg2=inp_val+'.jpeg'
        file = open(strg,'rb')
        object = s3.Object('bucketname',strg2)
        object.put(Body=file,ContentType='image/jpeg',ACL='public-read')



    else:
        continue

Upvotes: 0

John Rotenstein
John Rotenstein

Reputation: 269370

Your code is using the AWS SDK for Python ("boto3"), not the AWS Command-Line Interface (CLI). The AWS CLI is also written in Python and uses boto to call AWS.

There is no Amazon S3 API call to upload multiple files. You will need to upload each file individually. With some clever programming you can use multiple threads to upload multiple files simultaneously but each API call will only upload a single file.

The AWS CLI has said clever programming included, so you can use aws s3 cp or aws s3 sync to specify multiple files and it will perform parallel uploads.

Upvotes: 1

Related Questions