CODEWITHSUNDEEP
azurexamarinazure-storageazure-blob-storage
user246392
user246392

Reputation: 3019

Azure Storage: BlobClient keeps resetting itself while uploading large files

I'm trying to upload a 200 MB video file from a mobile phone that's connected to a fast Wifi connection. I'm using v12 of Azure Storage SDK for .NET, but the below code keeps resetting itself after about 30% upload progress. When the reset occurs, progress begins from 0 and there is no exception thrown.

await blobClient.UploadAsync(stream, progressHandler: new Progress<long>(progress =>
{
     // show progress bar
}), cancellationToken: cancellationToken);

Does v12 support uploading large files? If I remember correctly, the old APIs had the ability to upload individual blocks. I was under the impression that the above upload method would handle chunking implicitly.

How can I upload large files using the latest sdk?

P.S. I tried passing a StorageTransferOptions with very high concurrency and 1 MB transfer size, but it made no difference.

EDIT: After waiting for a long time, I was able to get an exception thrown. I see that multiple tasks get canceled because of

Cannot access a disposed object.
Object name: 'MobileAuthenticatedStream'.

  at Mono.Net.Security.MobileAuthenticatedStream.StartOperation (Mono.Net.Security.MobileAuthenticatedStream+OperationType type, Mono.Net.Security.AsyncProtocolRequest asyncRequest, System.Threading.CancellationToken cancellationToken) [0x00245] in /Users/builder/jenkins/workspace/archive-mono/2020-02/android/release/mcs/class/System/Mono.Net.Security/MobileAuthenticatedStream.cs:410 
  at System.Net.Http.HttpConnection.WriteAsync (System.ReadOnlyMemory`1[T] source) [0x00118] in /Users/builder/jenkins/workspace/archive-mono/2020-02/android/release/external/corefx/src/System.Net.Http/src/System/Net/Http/SocketsHttpHandler/HttpConnection.cs:1008 
  at System.IO.Stream.CopyToAsyncInternal (System.IO.Stream destination, System.Int32 bufferSize, System.Threading.CancellationToken cancellationToken) [0x000e7] in /Users/builder/jenkins/workspace/archive-mono/2020-02/android/release/external/corert/src/System.Private.CoreLib/shared/System/IO/Stream.cs:152 
  at Azure.Core.RequestContent+StreamContent.WriteToAsync (System.IO.Stream stream, System.Threading.CancellationToken cancellation) [0x00094] in <7b1dc95b0b4841539beb48023c1128d3>:0 
  at Azure.Core.Pipeline.HttpClientTransport+PipelineRequest+PipelineContentAdapter.SerializeToStreamAsync (System.IO.Stream stream, System.Net.TransportContext context) [0x0007c] in <7b1dc95b0b4841539beb48023c1128d3>:0 
  at System.Net.Http.HttpContent.CopyToAsyncCore (System.Threading.Tasks.ValueTask copyTask) [0x00022] in /Users/builder/jenkins/workspace/archive-mono/2020-02/android/release/external/corefx/src/System.Net.Http/src/System/Net/Http/HttpContent.cs:361

So it seems there is parallelization built into blobClient, but it's failing still. Additionally, I cannot use https://www.nuget.org/packages/Microsoft.Azure.Storage.DataMovement because it doesn't work with Xamarin Forms.

Upvotes: 2

Views: 570

Answers (2)

Jim Xu
Jim Xu

Reputation: 23111

If you want to upload file to Azure blob storage in chunk, please refer to the following code

public async Task upload(Stream stream){
            string connectionString = "";
            string containerName = "upload";
            string blobName = "";
            BlockBlobClient blobClient = new BlockBlobClient(connectionString, containerName, blobName);
           
            List<string> blockList = new List<string>();

                while (true) {
                    byte[] b = new byte[1024 * 1024];
                    var n = await stream.ReadAsync(b, 0, 1024 * 1024);
                    if (n == 0) break;
                    string blockId = Guid.NewGuid().ToString();
                    string base64BlockId = Convert.ToBase64String(Encoding.UTF8.GetBytes(blockId));
                    await blobClient.StageBlockAsync(base64BlockId, new MemoryStream(b, true));
                    blockList.Add(base64BlockId);
                }

                await blobClient.CommitBlockListAsync(blockList);

   
}

Update

If you want to use sas token, please refer to the following code

var uri = new Uri($"https://{storageAccountName}.blob.core.windows.net/{containerName}/{blobName}?{sasToken}");
BlockBlobClient blobClient = new BlockBlobClient(uri);

Upvotes: 3

Shiraz Bhaiji
Shiraz Bhaiji

Reputation: 65391

v12 has both BlobClient and BlockBlobClient. It is BlockBlobClient that has the functionality of splitting into blocks and uploading each block.

You could try Microsoft.Azure.Storage.DataMovement https://www.nuget.org/packages/Microsoft.Azure.Storage.DataMovement

"Microsoft Azure Storage DataMovement Library offers a set of APIs extending the existing Azure Storage .Net Client Library to help customer transfer Azure Blob and File Storage with high-performance, scalability and reliability."

Upvotes: 1

Related Questions