CODEWITHSUNDEEP
c#parallel-processingmessage-queueazure-blob-storage
Nitin Bourai
Nitin Bourai

Reputation: 458

Parallel Uploading images to Azure blob what can be a better option?

i have a List and Every Message has a Image and List now i need to update the blob Url for Image and Attachment for Every Message

solution 1 : call UploadBlob for image and and a call to UploadBlob for each attachment and then update the Message Object in DB with Blob url

Solution 2 upload the image and All attachments in one go and use parallel uploading and updates the Blob URl and save Message in DB,Every Message has a Unique id so while uploading to blob i can set this id to create a blob url which is unique,so to update Message i need to map Message Ids with Blob Url which contains message Id

if you have any other solution to increase code readability and performance pls Advice

Upvotes: 0

Views: 949

Answers (2)

Ming Xu - MSFT
Ming Xu - MSFT

Reputation: 2116

Base on my understanding, looks like solution 2 is better.

We could upload multiple files in parallel with the help of tasks parallel library: http://msdn.microsoft.com/en-us/library/dd460717.aspx. We could also use async pattern (call the Begin/End methods). But it would be better if we would not upload too many files at the same time. If too many pending web requests occurs, some of them might fail. For example, we can choose to create 10 tasks, wait for 1 to finish, and then create the 11th.

Best Regards,

Ming Xu.

Upvotes: 0

David Makogon
David Makogon

Reputation: 71120

Not sure if I fully understand the question, but... it seems like each message will have an image and zero or more attachments, and that the message object needs to have a reference to the image and related attachments.

If that's a correct interpretation: Each upload would go in its own blob. Even with solution #2 above, you'd still end up with multiple blob URI's after all the uploads.

I can't tell where this fits in your app, whether it's a desktop app doing the uploads, or maybe a web app that's taking images and attachments from an end-user, then uploading them to blobs (along with a database record being written). If it's the latter: I'd suggest the following:

  1. Write the message record (which would presumably give you some type of unique ID back)
  2. When receiving each image or attachment, upload to blob storage and then update the message with resulting URI

You can kick off parallel tasks to upload each image or attachment and then update the Message record. As long as you have the bandwidth, parallel uploading is going to make more use of it, up to a point. Each storage account does have a throughput cap (5,000 transactions per second, and 3Gbps).

There are certainly alternative approaches, such as introducing a queue for Message updates, using a CQRS pattern, etc. Part of the decision will be based on the app's architecture (as I stated, I can't tell if this is a desktop app doing the upload. to blob storage, or a web server doing the uploads).

Upvotes: 1

Related Questions