CODEWITHSUNDEEP
amazon-web-servicesamazon-s3
Sanjay M. P.
Sanjay M. P.

Reputation: 1009

Remove empty files from S3 bucket

We have 500 gb data set in S3 bucket, where we have some empty files, need to remove the empty files is there a better than copying to a linux machine and running the find cmd to delete the empty files ?

Upvotes: 1

Views: 1168

Answers (2)

Sanjay M. P.
Sanjay M. P.

Reputation: 1009

I mounted the bucket on to an ec2 instance using s3fs and ran the empty file/dir check, this method was more convenient.

Upvotes: 0

Marcin
Marcin

Reputation: 238199

If you don't know which files are empty, you could request S3 inventory. It is provided once a day or week in a CSV format. One of its fields is:

Size – Object size in bytes.

Thus, having the inventory file, you will be able to very efficiently identify, and then remove, empty files from your bucket.

Upvotes: 4

Related Questions