Copy Azure blob content to local server with specific structure

Question

I want to copy Azure blob content to local server

https://<blob>/prod/backup/node1 -> f:\backup\cluster
https://<blob>/prod/backup/node2/<RandomFolderName>/forests/* -> f:\backup\cluster\forests
https://<blob>/prod/backup/node3/<RandomFolderName>/forests/* -> f:\backup\cluster\forests

I want final folder structure on server to be:

f:\backup\cluster\
f:\backup\cluster\forests\<forest2-content-folder>
f:\backup\cluster\forests\<forest3-content-folder>

Answer 1

I end up creating a PowerShell script to execute post AzCopy task to restructure folders

Answer 2

Copy Azure blob content to local server with specific structure

You can achieve this by using Azcopy tool to copy azure blob content to local server with specific structure.

Here is the command:

azcopy copy "https://venkat326123.blob.core.windows.net/prod/backup/node1/*?<Your-sas-token>" "C:\backup\cluster" --recursive=true

azcopy copy "https://venkat326123.blob.core.windows.net/prod/backup/node2/results/forests/*?<Your-sas-token>" "C:\backup\cluster\forests" --recursive=true

azcopy copy "https://venkat326123.blob.core.windows.net/prod/backup/node3/example/forests/*?<Your-sas-token>" "C:\backup\cluster\forests" --recursive=true

Command and Output:

PS C:\Users\xxxx> azcopy copy "https://venkat326123.blob.core.windows.net/prod/backup/node1/*?<Your-sas-token>" "C:\backup\cluster" --recursive=true
INFO: Scanning...
INFO: azcopy.exe 10.22.2: A newer version 10.25.1 is available to download

INFO: Any empty folders will not be processed, because source and/or destination doesn't have full folder support

Job ea769875-1zzzd3e0c7ef4 has started
Log file is located at: C:\Users\zzzz.azcopy\ea7698xxx-884d-5166-71ad3e0c7ef4.log

100.0 %, 3 Done, 0 Failed, 0 Pending, 0 Skipped, 3 Total, 2-sec Throughput (Mb/s): 0.0633


Job ea769875-15zzzzsummary
Elapsed Time (Minutes): 0.0334
Number of File Transfers: 3
Number of Folder Property Transfers: 0
Number of Symlink Transfers: 0
Total Number of Transfers: 3
Number of File Transfers Completed: 3
Number of Folder Transfers Completed: 0
Number of File Transfers Failed: 0
Number of Folder Transfers Failed: 0
Number of File Transfers Skipped: 0
Number of Folder Transfers Skipped: 0
TotalBytesTransferred: 15823
Final Job Status: Completed

PS C:\Users\zzzz>  azcopy copy "https://venkat326123.blob.core.windows.net/prod/backup/node2/results/forests/*?<Your-sas-token>" "C:\backup\cluster\forests" --recursive=true

INFO: Scanning...
INFO: azcopy.exe 10.22.2: A newer version 10.25.1 is available to download

INFO: Any empty folders will not be processed, because source and/or destination doesn't have full folder support

Job 3b63fezzzzz9c50ce has started
Log file is located at: C:\Users\zzz\.azcopy\3b63xxxxxf-0445-4e19-c116a29c50ce.log

100.0 %, 4 Done, 0 Failed, 0 Pending, 0 Skipped, 4 Total, 2-sec Throughput (Mb/s): 0.0396


Job 3b63fe9d-czzz29c50ce summary
Elapsed Time (Minutes): 0.0335
Number of File Transfers: 4
Number of Folder Property Transfers: 0
Number of Symlink Transfers: 0
Total Number of Transfers: 4
Number of File Transfers Completed: 4
Number of Folder Transfers Completed: 0
Number of File Transfers Failed: 0
Number of Folder Transfers Failed: 0
Number of File Transfers Skipped: 0
Number of Folder Transfers Skipped: 0
TotalBytesTransferred: 9940
Final Job Status: Completed

PS C:\Users\zzz> azcopy copy "https://venkat326123.blob.core.windows.net/prod/backup/node3/example/forests/*?<Your-sas-token>" "C:\backup\cluster\forests" --recursive=true
INFO: Scanning...
INFO: azcopy.exe 10.22.2: A newer version 10.25.1 is available to download

INFO: Any empty folders will not be processed, because source and/or destination doesn't have full folder support

Job 176d8769-bec9-184zzzc9 has started
Log file is located at: C:\Users\zzzzu\.azcopy\176zzec9-1846-zz.log

100.0 %, 3 Done, 0 Failed, 0 Pending, 0 Skipped, 3 Total, 2-sec Throughput (Mb/s): 0.5394


Job 176d8769zzcf46a460c9 summary
Elapsed Time (Minutes): 0.0336
Number of File Transfers: 3
Number of Folder Property Transfers: 0
Number of Symlink Transfers: 0
Total Number of Transfers: 3
Number of File Transfers Completed: 3
Number of Folder Transfers Completed: 0
Number of File Transfers Failed: 0
Number of Folder Transfers Failed: 0
Number of File Transfers Skipped: 0
Number of Folder Transfers Skipped: 0
TotalBytesTransferred: 135843
Final Job Status: Completed

In Cluster folder:

In forests folder:

Reference: azcopy copy | Microsoft Learn