CODEWITHSUNDEEP
azure.net-coreazure-blob-storageazure-storage
Maciej Kachniarz
Maciej Kachniarz

Reputation: 231

How to upload big files to Azure Blob Storage (.NET Core)

I try to upload 2GB file to Azure Storage, but my code fails and I have an exception. For smaller files it works fine. How to change timeouts and file size limits? I use Azure.Storage.Blobs version 12.10.0 on .NET Core.

My code:

    public FileRepository(AppSettings appSettings)
    {
        var blobServiceClient = new BlobServiceClient(appSettings.StorageConnectionString);

        var containers = blobServiceClient.GetBlobContainers();
        if (!containers.Any(x => x.Name == containerName))
        {
            blobServiceClient.CreateBlobContainer(containerName);
        }
        _blobContainerClient = blobServiceClient.GetBlobContainerClient(containerName);
    }

    public async Task<string> Upload(string fileName, Stream content)
    {
        var blobName = $"{Guid.NewGuid()}/{fileName}";
        await _blobContainerClient.UploadBlobAsync(blobName, content);
        return blobName;
    }

Exception I have:

System.AggregateException
  HResult=0x80131500
  Message=Retry failed after 6 tries. (The operation was canceled.) (The operation was canceled.) (The operation was canceled.) (The operation was canceled.) (The operation was canceled.) (The operation was canceled.)
  Source=Azure.Core
  StackTrace:
   at Azure.Core.Pipeline.RetryPolicy.<ProcessAsync>d__11.MoveNext()
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Azure.Core.Pipeline.HttpPipelineSynchronousPolicy.<ProcessAsync>d__1.MoveNext()
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Azure.Core.Pipeline.HttpPipelineSynchronousPolicy.<ProcessAsync>d__1.MoveNext()
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Azure.Core.Pipeline.HttpPipelineSynchronousPolicy.<ProcessAsync>d__1.MoveNext()
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Azure.Storage.Blobs.BlobRestClient.BlockBlob.<UploadAsync>d__0.MoveNext()
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Threading.Tasks.ValueTask`1.get_Result()
   at Azure.Storage.Blobs.Specialized.BlockBlobClient.<UploadInternal>d__26.MoveNext()
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.ConfiguredTaskAwaitable`1.ConfiguredTaskAwaiter.GetResult()
   at Azure.Storage.Blobs.Specialized.BlockBlobClient.<>c__DisplayClass48_0.<<GetPartitionedUploaderBehaviors>b__0>d.MoveNext()
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.ConfiguredTaskAwaitable`1.ConfiguredTaskAwaiter.GetResult()
   at Azure.Storage.PartitionedUploader`2.<UploadInternal>d__19.MoveNext()
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.ConfiguredTaskAwaitable`1.ConfiguredTaskAwaiter.GetResult()
   at Azure.Storage.Blobs.BlobClient.<StagedUploadInternal>d__29.MoveNext()
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.ConfiguredTaskAwaitable`1.ConfiguredTaskAwaiter.GetResult()
   at Azure.Storage.Blobs.BlobContainerClient.<UploadBlobAsync>d__83.MoveNext()
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter`1.GetResult()
   at C:\...\FileRepository.<Upload>d__3.MoveNext() in C:\...\FileRepository.cs:line 38

Inner Exception 1:
TaskCanceledException: The operation was canceled.

Inner Exception 2:
HttpRequestException: Error while copying content to a stream.

Inner Exception 3:
IOException: Unable to read data from the transport connection: The I/O operation has been aborted because of either a thread exit or an application request..

Inner Exception 4:
SocketException: The I/O operation has been aborted because of either a thread exit or an application request.

Upvotes: 1

Views: 4419

Answers (1)

Gaurav Mantri
Gaurav Mantri

Reputation: 136356

I think it is not possible to control the timeout option and file size limit when uploading using BlobContainerClient.

You will need to make use of UploadAsync method in BlobClient class and specify appropriate values in BlobUploadOptions.TransferOptions.

If you want even more fine grained control over the upload process, you will need to make use of upload methods in specific blob clients. For example, if you're uploading block blobs, you can make use of BlockBlobClient.StageBlockAsync and BlockBlobClient.CommitBlockListAsync.

Upvotes: 2

Related Questions