CODEWITHSUNDEEP
pythonamazon-s3boto3
Ryan Williams
Ryan Williams

Reputation: 9

S3 File Download Returns Error

I wrote the following Python script to download ALL files within an S3 Bucket into my current directory:

import boto3
import botocore
import os

from boto3.session import Session

ACCESS_KEY='AWS_IAM_AccessKey'
SECRET_KEY='AWS_IAM_SecretKey'

session = Session(aws_access_key_id=ACCESS_KEY, 
aws_secret_access_key=SECRET_KEY)

s3 = session.resource('s3')
myBucket = s3.Bucket('S3_bucketName')

for object in myBucket.objects.all():
    myBucket.download_file(object.key, os.path.join(os.curdir, object.key))

I am receiving the following error: 

FileNotFoundError: [Errno 2] No such file or directory: './raw_cdr/s3_fileName'

How can I fix this? Despite the fact that the s3_fileName is in fact an actual file within the raw_cdr directory, its saying that no such file/directory exists (and it only finds that one file amongst several).

Also, as the S3 bucket size grows, I'd like to further enhance this script to only pull down S3 files generated within the last 24 hours (using the Last Modified column value) as opposed to all of them. Any and all suggestions are much appreciated.

Upvotes: 0

Views: 2048

Answers (1)

Ashish Acharya
Ashish Acharya

Reputation: 3399

You're getting this error because object.key contains /raw_cdr/s3_fileName. Since /raw_cdr/ does not exist, it can't create the file. Try doing it like this:

for object in myBucket.objects.all():
    myBucket.download_file(object.key, os.path.join(os.curdir, os.path.basename(object.key)))

Upvotes: 2

Related Questions