CODEWITHSUNDEEP
amazon-web-servicesamazon-s3http-headersamazon-cloudfront
vanilla_skies
vanilla_skies

Reputation: 415

Setting HTTP headers for all files in AWS S3 bucket

Ive have been able to figure out how to set the http header, cache-control: max_age on a single file using the AWS console. However, I would like to do it for all files in a S3 bucket.

How do I go about doing that for say 500 files in one bucket? I can't do it for each individual file using the AWS console.

If you can't do it through AWS. can you recommend a third party app?

Thanks, Jeff

Upvotes: 5

Views: 6334

Answers (3)

SomeGuyOnAComputer
SomeGuyOnAComputer

Reputation: 6208

The awscli allows this. I wouldn't recommend running this on a bucket where you have mixed metadata. For instance, if you have some objects that are KMS encrypted, then the below example will rewrite all your objects without those credentials.

This will set content type header on every object in the bucket if you added the --recursive switch.

aws s3 cp \
  s3://bucket-name/ \
  s3://bucket-name/ \
  --metadata-directive REPLACE \
  --content-type text/plain

To safely do this, I'd do it for each item, one at a time, either using a python script or being selective on the CLI to make sure the metadata is the same before and after with only the additional information.

You can use aws s3api for that.

aws \
  s3api head-object \
  --bucket s3://bucket-name/ \
  --key some-secret
{
    "AcceptRanges": "bytes",
    "LastModified": "2020-06-17T21:07:35+00:00",
    "ContentLength": 56,
    "ETag": "snip",
    "VersionId": ".aWa93h._.g_NEYcJgt_7MXJWzALyqSa",
    "ContentType": "text/plain",
    "ServerSideEncryption": "aws:kms",
    "Metadata": {},
    "SSEKMSKeyId": "arn:aws:kms:us-east-1:snip:key/snip"
}

So if the first replace awscli command was run, it would remove the SSEKMSKeyId key of my policy. For that, we'd also have to add the following switches to the first command at the top.

  --sse-kms-key-id "arn:aws:kms:us-east-1:snip:key/snip" \
  --sse aws:kms

Upvotes: 2

jmng
jmng

Reputation: 2568

This has since been improved, you can edit folder metadata in the console, just select the folder(s) or file(s), press Actions, choose Change Metadata and you'll be presented with a panel like this:

enter image description here

in https://docs.aws.amazon.com/AmazonS3/latest/user-guide/add-object-metadata.html

Upvotes: 8

Michael - sqlbot
Michael - sqlbot

Reputation: 179074

The only way to modify object metadata is to make copy of the object and set the metadata. In the copy operation you set the same object as the source and target.

— http://docs.aws.amazon.com/AmazonS3/latest/dev/CopyingObjectsExamples.html

This is the operation that the console is doing for you, behind the scenes. There is no batch operation provided.

You'll need to write some code to iterate through the objects in the bucket and call the api for each object you want to modify.

Or, find a tool or library that can do that for you. Which tool and where to find it are questions outside the scope of Stack Overflow.

Upvotes: 2

Related Questions