Azure files has high list transactions

Question

We are in the process of moving fully into the cloud and do away with our VPN and on-premise file server. We are looking to put the company 100% remote. I have chosen Azure Files because we are microsoft shop for most our stuff. I used the calculators ahead of time to determine our storage costs. Everything seemed like it was going to be just hunky dory. However, now that I have it all live, data moved, and am preparing to push it out to everyone, I have run into a cost issues I wasn't expecting. I knew that the initial sync with our on prem file server was going to incur a high write transaction because of 6TB of data. That being said, I was not expecting that LIST transaction were going to be high. Right now, I am quite litterally the only one connecting to the share with my tests, and we have incurred more than 24 million list transactions. Over the weekend where literally nothing was being touched, we incurred another 2.1 million list transactions $20 worth. I have asked microsoft what exactly is a list transaction and how does one happen, and they are not being of help at all. They keep just copy and pasting the definition from their website ("List transactions are any operations which enumerate parts of the file share, such as listing files and directories within a share.") and then dodging question on what actions incur a list transaction. So I am asking my dear friends of the IT community, what is a list transaction and how can I avoid so many? Is it avoidable?

I mean, the weekend hit and nobody was touching anything and it is just casually racking up list transactions.

Answer 1

I found my answer, though I don't have a solution for it. I found this in the FAQ about Azure file Sync:

"To detect changes to the Azure file share, Azure File Sync has a scheduled job called a change detection job. A change detection job enumerates every file in the file share, and then compares it to the sync version for that file. When the change detection job determines that files have changed, Azure File Sync initiates a sync session. The change detection job is initiated every 24 hours. Because the change detection job works by enumerating every file in the Azure file share, change detection takes longer in larger namespaces than in smaller namespaces. For large namespaces, it might take longer than once every 24 hours to determine which files have changed. "

I went into my metrics and found that every day at between 10:15 PM and about 1:00 AM the next day, I was having a spike of about 50k transactions every few moments, which totals about 1 million transactions over the few hours. Looking at my Azure Sync, sure enough, there is a scheduled enumeration that starts at 10:15 PM every night and ends at about 12:50 or so. This means that with this scheduled enumeration, I will be incurring about $10/day in list transactions. The good news is that when we go full time with this and shut off azure file sync and the old on prem server, this won't be an issue. However, this also means we will incur these list costs every day. Unless someone knows how to change this enumeration schedule, the only solution I can think of is to delete the sync and just recreate it every couple of weeks.