Madushan
Madushan

Reputation: 7458

Azure Blobs billing break down of "All Other Operations"

I am looking at the billing for my Azure Storage Account and trying to understand managing its cost.

Currently my blobs cost is mostly under the "All Other Operations" category. Is there a way to see what operations these are?

I would like to reduce this cost, so the goal is to update my app so these operations are performed less, but I need to first identify what they are.

Below is the graph I get from cost analysis. (Storage accounts, Accumulated cost, grouped by meter)

eazure cost analysis

Upvotes: 5

Views: 2841

Answers (2)

KrishnaG
KrishnaG

Reputation: 3484

Short answer to your question is Yes.

Analysis: According to my observation, I get "all other operations" when I group by "Meter" as shown in below screenshot.

enter image description here

And then if I export the results by clicking on "Export" and then when I filter the results for "Meter" column with "all other operations" then I observe that column named "ServiceTier" has "tiered block blob" as value (in my case). For reference, see below screenshot.

enter image description here

And then if I group by "Meter subcategory" as shown in below screenshot then I see "tiered block blob" (in my case).

enter image description here

And then if I export the results by clicking on "Export" and then when I filter the results for "Meter subCategory" column with "tiered block blob" then I observe that column named "ServiceTier" also has "tiered block blob". For reference, see below screenshot.

enter image description here

So based on above analysis, I believe that we can figure out the break down of "Meter" column with "all other operations" as "tiered block blob" in my case with the help of "Meter subcategory" and "ServiceTier". Similarly you would be able to figure out the break down of "Meter" column with "all other operations".

Other related references: As per this and this Azure documents, there are many other operations on blobs excluding write, read, list operations so in your case any such operations might have fallen under "all other operations" category.

Upvotes: 1

Madushan
Madushan

Reputation: 7458

After a support call with Azure, they pointed me to some of the (somewhat hidden) tracing capabilities.

First and easiest is to check the type of transactions.

  • Go to the storage account > Metrics
  • Select Transactions as the metric
  • Click Add Filter and select API Name as the property
  • Select the API names you think are the suspects

Azure storage transaction metrics

Unfortunately selecting multiple doesn't show them separately, so you have to try every API individually and see if anything sticks out.


Second option is to enable Diagnostics Logging for the storage type you're interested in.

If the above doesn't yield any good results, or you're curious about the exact calls at exact times .etc. you can enable this feature, and wait for logs to be collected, usually over a few days, so you have a good sample set to reason.

  • Go to the storage account > Diagnostic settings (classic).
    • This is under Monitoring (classic) doesn't seem to have a replacement in the new Monitoring section.
  • Enable logging and metrics type (hour or minute)
  • Click Save

azure storage diagnostic logs setup

These logs are written to a blob storage in the same account to a container named $logs. According to docs, this container cannot be deleted after enabling, but the content can be deleted when you're done.

Note that if your storage account gets a lot of traffic, this log can get very big very quick. You're charged the same rates for reads, writes and storage in this container as usual, including the log writes the platform does when these settings are enabled.

See documentation here

After setting this up, give it some time to collect data.

Use storage explorer or other means to navigate and download the logs and inspect them.

Logs contain every request made to storage, with details such as timestamp, API name, result, whether the operation was authenticated, and if you're looking at blobs it also shows the url, the user agent and more.

(turns out my app did close to 100,000 calls to GetBlobProperties and GetContainerProperties per day🎈)

Upvotes: 9

Related Questions