CODEWITHSUNDEEP
amazon-s3http-headers
user2615911
user2615911

Reputation: 21

Amazon S3: change HTTP headers of a million files

My Amazon S3 bucket has 2 million files.

I wish to change the HTTP header "Cache-Control" to a longer time period.

The advice seems to be to copy the object, setting the new headers in the process. But this takes around 0.5 seconds per object, and so will take an infinite amount of time, metaphorically.

Is there another way of changing HTTP headers on S3 objects?

Upvotes: 2

Views: 187

Answers (1)

sthede
sthede

Reputation: 928

There is no way to change anything about an object once it is uploaded. It seems like overkill to have to make a copy of an object just to change something so small, but that is the way it is with S3.

A way you can optimize the copy object procedure is to make several calls to S3 at the same time. I noticed a huge improvement in speed when I multithreaded our application, and made several calls to S3 at once, instead of performing them sequentially.

Additionally, launching an EC2 instance to do this would no doubt be even faster.

Amazon S3 can be so very rigid, for example S3 does not offer a rename feature, but it has been faked for so long via a copy object procedure, that most do not realize that rename is not possible in the traditional sense, on S3.

Upvotes: 1

Related Questions