CODEWITHSUNDEEP
azureazure-blob-storageazcopy
Reddspark
Reddspark

Reputation: 7559

Copying from Azure file to BLOB

Does Azure have a way of moving files between Storage containers without downloading it back down to your laptop? I tried AzCopy but that seems to be downloading then re-uploading the file.

Upvotes: 1

Views: 1210

Answers (2)

silent
silent

Reputation: 16108

Azcopy does indeed provide the feature for server-side copy. It is in fact the default behaviour. You can find an example of that here: AzCopy /Source:https://myaccount.blob.core.windows.net/mycontainer1 /Dest:https://myaccount.blob.core.windows.net/mycontainer2 /SourceKey:key /DestKey:key /Pattern:abc.txt

However, be warned: As you are getting the required compute basically for free as spare capacity from Azure, you do not get any performance SLA. So this is most cases way slower than for example using a fast VM in the same Azure region and using what's called synchronous copy (down- and upload to copy, using the /SyncCopy parameter in azcopy).

So: If you have time and what to save money, use server-side copy. If you want your copy to be done quickly, use /SyncCopy

Upvotes: 3

Sage
Sage

Reputation: 4937

You could use a Runbook to accomplish this. Here is link to an article on Microsoft's site that accomplishes this;

https://blogs.msdn.microsoft.com/cloud_solution_architect/2016/05/09/azure-storage-account-backup-azure-automation-runbook-azure-functions/

here is the relevant code, slightly modified to make ResourceGroup a variable. 

$primary = Get-AutomationVariable -Name 'Log-Storage-Primary'
$secondary = Get-AutomationVariable -Name 'Log-Storage-Secondary'
$ResourceGroupName = Get-AutomatioNVariable -Name 'LogStorageResourceGroup'

$Conn = Get-AutomationConnection -Name AzureRunAsConnection
Add-AzureRMAccount -ServicePrincipal -Tenant $Conn.TenantID -ApplicationId $Conn.ApplicationID -CertificateThumbprint $Conn.CertificateThumbprint

$primarykey = Get-AzureRmStorageAccountKey -ResourceGroupName $ResourceGroupName -Name $primary
$secondarykey = Get-AzureRmStorageAccountKey -ResourceGroupName $ResourceGroupName -Name $secondary

$primaryctx = New-AzureStorageContext -StorageAccountName $primary -StorageAccountKey $primarykey.Key1
$secondaryctx = New-AzureStorageContext -StorageAccountName $secondary -StorageAccountKey $secondarykey.Key1

$primarycontainers = Get-AzureStorageContainer -Context $primaryctx

# Loop through each of the containers
foreach($container in $primarycontainers) {
    # Do a quick check to see if the secondary container exists, if not, create it.
    $secContainer = Get-AzureStorageContainer -Name $container.Name -Context $secondaryctx -ErrorAction SilentlyContinue

    if (!$secContainer) {
        $secContainer = New-AzureStorageContainer -Context $secondaryctx -Name $container.Name
        Write-Host "Successfully created Container" $secContainer.Name "in Account" $secondary
    }

    # Loop through all of the objects within the container and copy them to the same container on the secondary account
    $primaryblobs = Get-AzureStorageBlob -Container $container.Name -Context $primaryctx
    foreach($blob in $primaryblobs) {
        $copyblob = Get-AzureStorageBlob -Context $secondaryctx -Blob $blob.Name -Container $container.Name -ErrorAction SilentlyContinue

        # Check to see if the blob exists in the secondary account or if it has been updated since the last runtime.
        if (!$copyblob -or $blob.LastModified -gt $copyblob.LastModified) {
            $copyblob = Start-AzureStorageBlobCopy -SrcBlob $blob.Name -SrcContainer $container.Name -Context $primaryctx -DestContainer $secContainer.Name -DestContext $secondaryctx -DestBlob $blob.Name
            $status = $copyblob | Get-AzureStorageBlobCopyState

            while ($status.Status -eq "Pending") {
                $status = $copyblob | Get-AzureStorageBlobCopyState
                Start-Sleep 10
            }
            Write-Host "Successfully copied blob" $copyblob.Name "to Account" $secondary "in container" $container.Name
        }
    }
}

Upvotes: 0

Related Questions