Automatically changing Azure blob storage tier to cool?

Question

We are considering moving archived data after some retention period to the newer cool tier of Azure Storage (https://azure.microsoft.com/en-us/blog/introducing-azure-cool-storage/).

Can I programatically set up something that will automatically change the tier or move content to a cool tier storage after some period of time?

Answer 1

Can I programmatically set up something that will automatically change the tier or move content to a cool tier storage after some period of time?

Yes, (since August 2021) it's now possible to have Azure automatically move blobs between Hot, Cool, and Archive tiers - this is done using a Lifecycle management policy which you can configure.

This is documented in this page: https://learn.microsoft.com/en-us/azure/storage/blobs/lifecycle-management-overview

Some caveats:

• You can't force-run a policy: once you've set-up the policy you'll simply have to wait... up to 48 hours or even longer.

  • So if you want to quickly archive or cool your blobs immediately then you should use another approach. Unfortunately I can't make any recommendations right at the time of me writing this.

• Your blobs need to be readable by Azure itself, so this won't work with super secure or inaccessible storage accounts.

• You need to enable blob access-tracking first (I describe how to do this in the section below).

  • Note that blob access tracking costs extra money, but I can't find any pricing information available on access-tracking.

• The policy rules you can create are somewhat restrictive as you can only filter the set of blobs to auto-archive or automatically move to the Cool tier based on only these parameters:

  • The Blob's Last-Modified time and Last-Accessed time - but only in relative days-ago (so you can't use absolute dates or complex interval/date-range expressions).
  • The blob type (only Block blobs can be auto-moved between tiers; Append blobs can only be auto-deleted (not auto-moved); while Page blobs aren't supported at all)
  • The blob container name, and optionally a blob name prefix match.
    • But you can't filter by blob-name prefix match without specifying a container name.
  • And by exact-matching Blob Tags already indexed by a Blob Index.
    • And Blob Indexes have their own limitations too.

• The only possible rule actions are to move blobs between tiers or to delete blobs. You can't move blobs between containers or accounts or edit any blob metadata or change a blob's access-policy and so on...

  • While there is an action that will move a Cool blob back to the Hot tier on the first access/request (which is nice), I'd have personally liked to see an option to promote the blob back to hot only after 2 or 3 (or more) requests in a given time-window, instead of always moving it on the first request, which means junk HTTP requests (e.g. web spiders, your own blob scanner tools, etc) might incur the more expensive Cool-tier data transfer/read fees - and you can’t move blobs to cool without a 30-day wait.

---

To set up a blob lifecycle policy to automatically move Hot blobs to the Cool tier if not accessed for 90 days (as an example), do this:

_{(Azure Portal screenshots taken as I wrote this, in August 2022 - if the screenshots or instructions are outdated leave a comment and I'll update them)}

1. Open the Azure Portal (https://portal.azure.com) and navigate to your Storage Account's "blade".

   • BTW, if your storage account is older than a few years then ensure your Storage account is upgraded to GeneralPurposeV2 or Blob-only.

2. Look for "Lifecycle Management" in the sidebar:

   • 

3. Ensure "Enable access tracking" is enabled.

   • The tooltip says access tracking has additional charges, but I can't find any information about what those charges are.

     • NOTE: The access tracking feature is distinct from Azure Blob Inventory, but lots of pages and Google results incorrectly report Blob Inventory pricing

   • 

4. Click "Add a rule" and complete the wizard:

   1. Enter a Rule name like move-blobs-not-accessed-for-90-days-to-cool-tier
   2. If you only want to auto-move blobs in specific containers (instead of the entire account) then check the "Limit blobs with filters".
   3. On the "Base blobs" tab of the wizard you can specify the Last-Modified and Last-Accessed rules and actions as described above.
      1. Note that you can have up to 3 conditions in a single policy, for the 3 possible actions: move-to-cool (tierToCool), move-to-archive (tierToArchive), and delete-blob (delete).

5. And that's it - so now you have to wait up to 48 hours for the policy to start to take effect, and maybe wait even longer for Azure to actually finish moving the blobs between tiers.

   • As I write this, I've had my policy set-up for about 2 hours now and there's still no-sign that it's even started yet. YMMV.

Answer 2

In additional, we can change the Blob Storage characteristic at any point. But when change from cool to hot, you have to pay for a lot of I/O for converting the blob type. Converting from hot to cool is free. We could find more details at this document.