CODEWITHSUNDEEP
amazon-web-servicesamazon-s3amazon-glacier
Thagor
Thagor

Reputation: 900

S3 Glacier Objects aren't restored

So about 14h ago I did run a restore command on 2 folders (both relativly large 10.5 TiB and 200 GiB) in my S3 bucket

s3cmd restore --recursive --restore-days=4 -restore-priority=bulk s3://mybucket/folder1/
s3cmd restore --recursive --restore-days=4 -restore-priority=bulk s3://mybucket/folder2/

now I want to copy both folders to a different bucket with

aws s3 sync s3://mybucket/folder1/ s3://mybucket2/

and

aws s3 sync s3://mybucket/folder2/ s3://mybucket2/

but the aws cli tells me that all my Object are of storage class GLACIER and that I have to restore them first. That is what I thought I did. When I take a look at them in the web interface they seem to be restored and they already have an expiration date. The expiration date is at Apr 14, 2018 2:00:00 AM GMT+0200 so more than 4 days from now (it is when I'm writing this Apr 9, 2018 10:30:00 AM GMT+0200) does that mean the objects only become available on Apr 10, 2018 2:00:00 AM GMT+0200 to conform with the 4 days of restore time?

A couple of edits:

a)

I at least think why the expiration date is further than 4 Days away, because AWS always rounds the date up to the next UTC midnight which in this case is at Apr 14, 2018 2:00:00 AM GMT+0200

b)

Also It tried to download the files through the web interface which works without a problem

c)

when I fetch the Metadata with aws s3api head-object --bucket mybucket --key folder1/part1.file this is the response:

{
    "Restore": "ongoing-request=\"false\", expiry-date=\"Sat, 14 Apr 2018 00:00:00 GMT\"", 
    "AcceptRanges": "bytes", 
    "ContentType": "binary/octet-stream", 
    "LastModified": "Wed, 20 Dec 2017 15:02:48 GMT", 
    "ContentLength": 59408357, 
    "ETag": "\"49b981b95eaf62abed0880c0db7c1200-8\"", 
    "StorageClass": "GLACIER", 
    "Metadata": {}
}

which seems to be how it should be.

d)

when I try copy with aws s3 cp s3://mybucket/folder1/part1.file s3://mybucket2/ it works but --recursive does not work aws s3 cp --recursive s3://mybucket/folder1/ s3://mybucket2/

Upvotes: 3

Views: 1252

Answers (2)

snehab
snehab

Reputation: 744

I have faced the same issue and the following command worked for me:

aws s3 cp s3://buckt_name/dir1/ s3://bucket_name/dir1/ --recursive --force-glacier-transfer

Upvotes: 3

coppaste
coppaste

Reputation: 176

If the object is already glacier restored, s3 sync with --force-glacier-transfer should work

Upvotes: 3

Related Questions