CODEWITHSUNDEEP
file-uploadamazon-s3boto
user3776364
user3776364

Reputation: 183

Setting metadata on S3 multipart upload

I'd like to upload a file to S3 in parts, and set some metadata on the file. I'm using boto to interact with S3. I'm able to set metadata with single-operation uploads like so:

Is there a way to set metadata with a multipart upload? I've tried this method of copying the key to change the metadata, but it fails with the error: InvalidRequest: The specified copy source is larger than the maximum allowable size for a copy source: <size>

I've also tried doing the following:

key = bucket.create_key(key_name)
key.set_metadata('some-key', 'value')
<multipart upload>

...but the multipart upload overwrites the metadata.

I'm using code similar to this to do the multipart upload.

Upvotes: 9

Views: 4299

Answers (2)

Kirill Vereshchagin
Kirill Vereshchagin

Reputation: 21

Faced such issue earlier today and discovered that there is no information on how to do that right. The code example on how we solved that issue provided below.

        $uploader = new MultipartUploader($client, $source, [
        'bucket' => $bucketName,
        'key' => $filename,
        'before_initiate' => function (\Aws\Command $command) {
            $command['ContentType'] = 'application/octet-stream';
            $command['ContentDisposition'] = 'attachment';
        },
    ]);

Unfortunately, documentation https://docs.aws.amazon.com/aws-sdk-php/v3/guide/service/s3-multipart-upload.html#customizing-a-multipart-upload doesn't make it clear and easy to understand that if you'd like to provide alternative meta data with multipart upload you have to go this way.

I hope that will help.

Upvotes: 2

user3776364
user3776364

Reputation: 183

Sorry, I just found the answer:

Per the docs:

If you want to provide any metadata describing the object being uploaded, you must provide it in the request to initiate multipart upload.

So in boto, the metadata can be set in the initiate_multipart_upload call. Docs here.

Upvotes: 6

Related Questions