CODEWITHSUNDEEP
c#.netazure-blob-storage
Haydar Can Kubilay Gümüş
Haydar Can Kubilay Gümüş

Reputation: 1

Performance issue uploading even small files to Azure Blob Storage in .Net 7

We are using Azure.Storage.Blobs library to upload files in .net7 environment. Some requests take 100 second which is default HttpClient timeout i think. I've read about blob storage client api uses a shared HttpClient object and it says that is thread safe but the const 100 second makes me thinking there is a bottleneck about HttpClient inside the library. I am confused at this point.

In our business, we have almost 20 containers in azure blob storage and the request comes with the container and blob name. Therefore I need to get the BlobContainerClient and BlobClient Objects in every request. This is the part that I'm suspicious. According to New Relic metrics, I understand that getting BlobContainerClient object takes 100 second "packing" is one of our containers name. It seems this is waiting a connection to finish I don't know.

This is our upload file method.

public async Task<string> UploadFileFromByteArrayAsync(string containerName, string blobName, byte[] fileBytes, CancellationToken cancellationToken = default)
{
    BlobClient blob = await GetBlobInstance(containerName, blobName, true);

    await blob.UploadAsync(new BinaryData(fileBytes), overwrite:true, cancellationToken);

    return blob.Uri.AbsoluteUri;
}


private async Task<BlobClient> GetBlobInstance(string containerName, string blobName, bool crateIfNotExist = false)
{
    BlobContainerClient container = await GetContainerInstance(containerName, crateIfNotExist);

    BlobClient blob = container.GetBlobClient(blobName);

    return blob;
}

private async Task<BlobContainerClient> GetContainerInstance(string containerName, bool crateIfNotExist = false)
{
    BlobContainerClient container = _blobServiceClient.GetBlobContainerClient(containerName);

    if (crateIfNotExist)
    {
        await container.CreateIfNotExistsAsync(PublicAccessType.BlobContainer);
    }

    return container;
}

And I've created a singleton lifetime for BlobServiceClient which is recommended.

builder.Services.AddSingleton(_ =>
{
    return new BlobServiceClient(builder.Configuration.GetSection("azureStorageConfiguration").GetValue<string>("ConnectionString"));
});

Note * all uploading files size are small like 15-20 Kib.

Upvotes: 0

Views: 674

Answers (1)

Gaurav Mantri
Gaurav Mantri

Reputation: 136126

According to New Relic metrics, I understand that getting BlobContainerClient object takes 100 second "packing" is one of our containers name. It seems this is waiting a connection to finish I don't know.

I believe this is the problem. I noticed that you are passing true for crateIfNotExist parameter and what your code end up doing is trying to create the blob container on every request to upload blob.

If you know that your containers are going to be present in the storage account, you can pass false for crateIfNotExist parameter and that should speed up the uploading process.

In short, try to change the following line of code:

BlobClient blob = await GetBlobInstance(containerName, blobName, true);

to

BlobClient blob = await GetBlobInstance(containerName, blobName);

Upvotes: 0

Related Questions