CODEWITHSUNDEEP
amazon-s3
Timothee A
Timothee A

Reputation: 175

AWS S3: Bulk move files within bucket

I have many files (millions) spread around numerous folders (hundreds of thousands) in a S3 bucket and I need to rename all folders according to a custom mapping.

I'm aware there's no rename method so I'll have to copy and delete. I can currently do that by looping on all my objects and copying with a new prefix, but that's way to slow.

I guess I'm looking to perform a bulk operation somehow, but I couldn't find a way to do so.

Upvotes: 4

Views: 2411

Answers (1)

Andrey Lebedenko
Andrey Lebedenko

Reputation: 1978

Will aws s3 mv command help?

To see if it will do the trick, try it with dryrun first:

aws s3 mv s3://bucketname/the/original/path/ s3://bucketname/the/new/path/ --recursive --exclude "*" --include "*whatever_common_part_of_name*.*" --dryrun

Upvotes: 3

Related Questions