With Amazon S3 Intelligent Tiering, how do you know which tier an object is in?

Question

https://docs.aws.amazon.com/AmazonS3/latest/userguide/archived-objects.html says:

When you archive Amazon S3 objects to the S3 Glacier or S3 Glacier Deep Archive storage class, or when objects are archived to the S3 Intelligent-Tiering Archive Access or Deep Archive Access tiers, the objects are not accessible in real time.

I understand this, but how can you tell if an object is in the Archive Access or Deep Archive Access tiers? I've gone through much of the AWS S3 documentation and I can't seem to find this information. I'm writing a UI for S3 and want to display either a "download" or a "restore" button depending on which tier the object is in. I'm just starting development on this so I don't have any files in those two tiers yet and I can't wait 90 days before anything gets moved there.

When you do a list-objects request, you get the storage class for each item, which comes back as INTELLIGENT_TIERING, but it doesn't tell you which tier it is.

A comment on Testing S3 Intelligent Tiering says you can fake it by changing the storage class to Glacier, but I have my doubts that this is an accurate test because Glacier and Intelligent Tiering are two different storage classes, and it doesn't make sense that Intelligent Tiering objects would suddenly change their storage class.

Does anyone have any insight? Thanks in advance.

UPDATE FROM AMAZON: (tl;dr, this is not possible)

Unfortunately we cannot identify the access tier for an object stored in Intelligent Tiering class the only way to determine the access tier of an object in by identifying the access pattern of an object. The access pattern can be identified either by analyzing AccessLogs or a user can review access pattern using S3 Storage Lens.

Based on your requirement, the best option for you would be to keep Intelligent-Tiering Archive configuration disabled. With this config disabled the objects in Intelligent-Tiering class will not move in "Archive Access tier" and "Deep Archive Access tier". With this configuration you can create a logic in your application that if the object is in class "Intelligent-Tiering" then the download button will be enabled and user will be able to download the object.

Now for moving objects to "Glacier" and "Glacier Deep Archive" you can configure a life cycle rule. Lets say you can configure a life cycle rule to transition objects in "Intelligent-Tiering" class to Glacier after 90 days and to "Glacier Deep Archive" after 180 days.

This way using ListObjects API call you can identify the correct storage class of the objects and you can program a logic where the "Restore" button will only be enabled if the storage class is "Glacier" or "Glacier Deep Archive". Appropriately a message will be displayed to users to download the object after 3-5 hours.

Answer 1

You can check if by going to your bucket -> metrics. The plot showing the data amoount segregates the data into different plots as per the tier their are stored in.

Answer 2

Likewise in HTTP HEAD request, it's now available in AWS CLI too.

aws s3api head-object --bucket xxxxxx --key input/2021-06-16-xxxxxx.xml

{
    "AcceptRanges": "bytes",
    "ArchiveStatus": "DEEP_ARCHIVE_ACCESS",
    "LastModified": "Wed, 16 Jun 2021 19:43:26 GMT",
    "ContentLength": 533096,
    "ETag": "\"d9a61a886e50199806664cdbd9xxxxxx\"",
    "VersionId": "null",
    "ContentType": "application/xml",
    "ServerSideEncryption": "AES256",
    "Metadata": {},
    "StorageClass": "INTELLIGENT_TIERING"
}

Official Docs: https://docs.aws.amazon.com/AmazonS3/latest/userguide/intelligent-tiering-managing.html

Answer 3

From the official documentation:

You can also use a HEAD object request to view an object's archive status. If an object is stored using the S3 Intelligent-Tiering storage class and is in one of the archive tiers, the HEAD object response shows the current archive tier. It does this using the x-amz-archive-status header.

Example:

HTTP/1.1 200 OK
x-amz-id-2: FSVaTMjrmBp3Izs1NnwBZeu7M19iI8UbxMbi0A8AirHANJBo+hEftBuiESACOMJp
x-amz-request-id: E5CEFCB143EB505A
Date: Fri, 13 Nov 2020 00:28:38 GMT
Last-Modified: Mon, 15 Oct 2012 21:58:07 GMT
ETag: "1accb31fcf202eba0c0f41fa2f09b4d7"
x-amz-storage-class: 'INTELLIGENT_TIERING'
x-amz-archive-status: 'ARCHIVE_ACCESS'
x-amz-restore: 'ongoing-request="true"'
x-amz-restore-request-date: 'Fri, 13 Nov 2020 00:20:00 GMT'
Accept-Ranges: bytes
Content-Type: binary/octet-stream
Content-Length: 300
Server: AmazonS3

Answer 4

@howcheng You can use S3 Inventory to get list of all of your objects and their corresponding metadata, including the S3 Intelligent-Tiering access tiers (Frequent, Infrequent, Archive, Deep Archive). Amazon S3 Inventory provides CSV, ORC, or Parquet output files that list your objects and their corresponding metadata on a daily or weekly basis for an S3 bucket or a shared prefix. You can also make a HEAD request on your objects to report the S3 Intelligent-Tiering archive access tiers.

Answer 5

After some back-and-forth with AWS support, I was able to come up with a working solution. For each item in the bucket, I can do a GetObject request and ask for a small byte range (like 10 bytes), so that's pretty much the equivalent of doing a HeadObject request. If the file is in the Archive or Deep Archive access tiers, then AWS will return an exception:

HTTP/1.1 403 Forbidden
x-amz-request-id: 43XTDH6Q7E9F2N5J
x-amz-id-2: 3Fn/qzcZwOxaCg1LLDsczSXF7stl7sbOwwDR1ajHIc+guZgxAfFHm1WfRoXWEsFLWPJzKt9Ynaw=
Content-Type: application/xml
Transfer-Encoding: chunked
Date: Wed, 23 Jun 2021 16:36:24 GMT
Server: AmazonS3

<?xml version="1.0" encoding="UTF-8"?>
<Error>
  <Code>InvalidObjectState</Code>
  <Message>The operation is not valid for the object's access tier</Message>
  <StorageClass>INTELLIGENT_TIERING</StorageClass>
  <RequestId>43XTDH6Q7E9F2N5J</RequestId>
  <HostId>3Fn/qzcZwOxaCg1LLDsczSXF7stl7sbOwwDR1ajHIc+guZgxAfFHm1WfRoXWEsFLWPJzKt9Ynaw=</HostId>
</Error>

So based on whether the request succeeds or not, I can determine whether to show the "download" or "restore" button for the item.

Answer 6

You can do a try catch. Try calling the restore and catch the exception. In your catch you can then do a GetObject.