CODEWITHSUNDEEP
azure-storage
shiva kumar
shiva kumar

Reputation: 19

How to copy many terabytes of data to Azure?

I am trying to copy 25 TB of data to Azure. Do we have any option to move the date?

Tried to copy but it has taken 1 hr for 1 GB Data, do we have any better solution so that I can do it more quickly?

Upvotes: 0

Views: 2059

Answers (3)

John Rusk - MSFT
John Rusk - MSFT

Reputation: 643

If you open the target Storage account in the Azure Portal, there's now a calculator that will accept basic details (how much data etc) and then recommend the best options to you. Its under the heading "Data transfer".

Upvotes: 0

Michael B
Michael B

Reputation: 12228

The best way to upload large datasets into the cloud is still the sneakernet

Azure do a thing called the Azure Import/Export Service Basically you buy a SATA hard drive, encrypt it with a numerical bitlocker key, copy data to it, create an Azure import job, then ship the hard drive to them.

This ends up being considerably quicker than trying to upload.

An alternative you might want to look into, would be the AWS Import/Export Snowball for which they will ship you an appliance to copy the data to which you ship back to them when complete. It might be worth considering copying data into AWS via Snowball then copying it across their much faster internet pipes into Azure instead of buying the hardware required to transfer that much data.

Upvotes: 1

ASB
ASB

Reputation: 101

The problem statement is very general. I would start with asking, how are you transferring the data?

The speed is dependent on so many factors, a few being: 1. Location of the data. 2. Location of the storage account you're writing to. 3. Network speed and bandwidth on the client side. 4. Network speed and bandwidth on the azure storage side. (expected to be good)

If you're writing the data to a Azure Storage account which is in a region closer to you, you're expected to get better speed.

As for the options to write the data: 1. Look at AzCopy. https://azure.microsoft.com/en-us/documentation/articles/storage-use-azcopy/

  1. Use Import\Export service. https://azure.microsoft.com/en-us/pricing/details/storage-import-export/

Upvotes: 2

Related Questions