CODEWITHSUNDEEP
python-3.xamazon-web-servicesamazon-s3boto3
Taukheer
Taukheer

Reputation: 1211

Python - Download data from specific folder / sub - folder AWS S3

I am trying to extract data from AWS S3. The code below helps me download a file. However if the bucket has multiple folders then the below code tries to extract the complete bucket and not a specific folder / sub-folder. How could I modify the below code to extract only a specific file in a folder / sub-folder

from boto3.session import Session
import boto3

ACCESS_KEY = 'ABC'
SECRET_KEY = 'XYZ'

session = Session(aws_access_key_id=ACCESS_KEY,
          aws_secret_access_key=SECRET_KEY)
s3 = session.resource('s3')
your_bucket = s3.Bucket('bucket_name')

for s3_file in your_bucket.objects.all():
    print(s3_file.key)

your_bucket.download_file('rawdata.xlsx' ,/Users/user/Desktop/folder/')

Thanks.

Upvotes: 0

Views: 6421

Answers (1)

BugHunter
BugHunter

Reputation: 1204

To download specific files, you should create a list of the files to be downloaded. then, you can check if the file matches, then download it.

files_to_downloaded = ["folder1/key.png", "folder2/other.png", "folderRandom/Randomkey.png"]
for fileObject in bucket.objects.all():
    file_name = str(fileObject.key)
    if file_name in files_to_downloaded:
        bucket.download_file(file_name, 'destination_path')

Upvotes: 3

Related Questions