CODEWITHSUNDEEP
amazon-web-servicesamazon-s3boto3
zendevil
zendevil

Reputation: 999

No such file or directory when downloading a file using boto3 from an was s3 bucket

I want to download the whole bucket "my-bucket" from aws s3 storage, and save it in the path "./resources/my-bucket". So, I'm doing the following, importing boto3, creating a resource with the access_key_id and secret_access_key, then iterating over all the objects in the bucket and downloading them using the download_file api call:

import boto3
s3 = boto3.resource(
service_name="s3",
region_name="us-west-1",
aws_access_key_id="AWUHWIEDFB",
aws_secret_access_key="AOIERJGOERWHJO"
)

for o in s3.Bucket("my-bucket").objects.all():
    filename = o.key
    s3.Bucket("my-bucket").download_file(Key=filename, Filename="./resources/my-bucket/" + filename)

The s3 bucket my-bucket itself has a folder called "input" with only png and jpg files, and also has another .pt file in the root path.

But I'm getting the error:

FileNotFoundError: [Errno 2] No such file or directory: './resources/my-bucket/input/.F89bdcAc'

I don't know what .F89bdcAc means, and why boto3 is trying to find that file.

How to fix this?

Upvotes: 7

Views: 16665

Answers (2)

Nomi
Nomi

Reputation: 325

It seems you don't have a directory structure that you are providing in the Filename param.

Try creating provided directory structure, i-e './resources/my-bucket/input/' and then the name of the file at the end, along with the extension for better.

Upvotes: 1

Julien
Julien

Reputation: 528

See the answer of this post : Boto3 to download all files from a S3 Bucket

It separates the processing of the directories in your bucket and the processing of the files (keys). Directories on your system are created on the fly with os.makedirs, you won't get this error anymore. And you ought to change your access key / secret !

Upvotes: 4

Related Questions