CODEWITHSUNDEEP
azcopy
Aparna
Aparna

Reputation: 845

How to use azcopy version 10.3.0 to copy and then delete file from Blob storage to VM using powershell

Some csv files keep getting placed in an azure storage container. I need to continuously move the files to a VM. I am using the following powershell script running on my VM. 

while($true){
.\azcopy sync "source blob" "destination folder on VM" --include-pattern "*.csv" --log-level ERROR
Start-Sleep -Seconds 60
}

The files are getting copied, but how do I delete the files from the source. They are not needed anymore after copying to the VM.

Upvotes: 0

Views: 11718

Answers (2)

Rick Taylor
Rick Taylor

Reputation: 11

I scoured the interwebs for this and still none of the "solutions" were sufficient.
SO -- I had to write my own. Below is the code I wrote that will do the following:
a. Download AZCopy if it doesnt exist.
b. Copy information from an Azure blob storage
c. Upon successful copy, delete information from source
 https://github.com/SlkRck/demoscripts/blob/master/copy-blobstorage.ps1

Upvotes: 1

Ivan Glasenberg
Ivan Glasenberg

Reputation: 30005

You can use azcopy remove command after the azcopy sync operation is completed. And here I need to mention that the azcopy sync operation is thread-blocking, so it's safe to use azcopy rm command at the end of the azcopy sync operation.

Note that if you want to just remove all the .csv file, you should add --include-pattern="*.csv" in the command.

I'm using the latest version of azcopy, v10.3.1. If you prefer to use v10.3.0, then first you need to use this command azcopy remove --help for the details of this command and it's parameters.

A sample command like below:

while($true){
.\azcopy sync "source blob" "destination folder on VM" --include-pattern "*.csv" --log-level ERROR
Start-Sleep -Seconds 60

#after the copy operation is completed, use remove command as below.
azcopy.exe rm "https://xxx.blob.core.windows.net/test4?sastoken" --include-pattern="*.csv" --recursive=true

}

One thing need to mention, in my test, I use the sas token for my testing purpose.

Upvotes: 0

Related Questions