Expensive use of storage account from Azure Functions

Question

I'm running a single Azure Function on the consumption plan. I've picked the consumption plan for the serverless feature as well as to minimize cost. The function consumes messages from a service bus topic and writes some output to blob storage.

Keeping the function running for the last 30 days is around $10. That's very acceptable, since the function has a lot of messages to consume. Writing the output to blob storage is around $20. Also acceptable. What I don't understand is, that the charge for the Function's underlying storage account is around $70 for the same period. The consumption is primarily hitting File Write Operation Units and File Protocol Operation Units. The storage account is created as locally redundant general purpose v1.

Anyone able to explain what's going on here? When looking at the storage account, there's a few blobs. I believe the problem is with tables storage. When inspecting the storage account, there are tables looking like this:

$MetricsCapacityBlob $MetricsHourPrimaryTransactionBlob AzureWebJobsHostLogs201804

I've disabled logging in my function, by removing the AzureWebJobsDashboard app setting. After doing so, the AzureWebJobsHostLogs* tables no longer seems to receive new rows. But the $Metrics* tables still receive new data. I have no clue if writes to these tables are causing all of the file write activity I see in the Costs Management view in the Portal, though.

What's going on here? Is maintaining these tables from serverless code really required and does it sound normal that the price for table access is x7 the price of the function itself?

Answer 1

You should go to Metrics in Azure Portal for this storage account and check the patterns of how the File storage transactions are consumed. If it's consistently high, it's something with your application (e.g. too much logging to file).

In my case, it appears to be a bug in Azure Functions, and I filed a bug here.

The function starts consuming tens of thousands of read and write transactions after any code change, however minor. So basically each code change or deployment costs me perhaps around $0.20, and it could be more in your case.

This is easy to see in the Metrics diagram because it looks like a huge spike in transactions.

So the solution is: don't write logs to the filesystem and don't deploy often.

Answer 2

It is interesting and unusual that your storage cost is so much higher. I think the dashboard logging is a likely culprit, so it would be good understand if you see a drop over the next few days with it turned off.

I would spend a bit more time in the cost analysis section of the Azure Portal to see if you can get more details about exactly which aspect of your storage usage is driving the majority of the cost. i.e. is it about table operations, blob operations, etc. This screenshot shows the Cost History view with a breakdown per meter. Note the tooltip in this screenshot:

The $Metrics tables are not written by Azure Functions, they are generated by Azure Storage itself. I would be surprised if these metrics were contributing significantly to your overall cost. But if you want to experiment, I think you can disable them through this UX:

To give you a baseline on what sort of ratio of storage costs to functions execution cost is expected, you might want to take a look at the cost write up I did in this blog post: https://blogs.msdn.microsoft.com/appserviceteam/2017/09/19/processing-100000-events-per-second-on-azure-functions/

You'll notice that the storage costs were less than functions, and that includes a significant number of storage operations due to event hubs processing requiring checkpoints written to storage. I'll note that these tests were run with dashboard logging off (again making me suspect that as the main cost driver). So no, it is NOT normal for your storage costs to be 7x your functions cost!