How to download multiple files with powershell

Question

Okay, so, here is what I ended up editing from my original answer. Kudos to @Matt for pointing out that I should be more descriptive with my answer and explain clearly what my edits were so that other users might be able to benefit from my answer in the futre. Users like @Matt are a great part of this community and put emphasis on keeping the standards high here.

The first thing I edited/added is the ability to delete the previous log from each run. Since this script will be scheduled it is important to remove the previous logs in order to prevent utilizing up too much disk space. This can be noted under the comment: "delete log files from prev run"

# delete log files from prev Run
Remove-Item C:\alerts\logs\*.*

The next thing I edited/added is the ability to switch between host names. I did this to prevent the overwriting of the files. You can see this under the comment "change filename in order to prevent overwriting of log file". I accomplished this by checking the index of "$url" in the foreach loop, and checked to see if it was at the position where I needed to change the host name. I suspect there was a much more intuitive way to do this and I would just love it if someone chimed in with a better way to do this as its driving me crazy that I don't know a better way. It should be noted that there are a total of 44 urls where I'm downloading from, hence the magic numbers (11, 22, 33) where I change the host name. Again, if you know a better way please don't hesitate to let me know.

If ($urls.IndexOf($url) -eq 11){
                $currentDir = "goxsd1704"
           }
           ElseIf ($urls.IndexOf($url) -eq 22){
                $currentDir = "goxsd1705"
           }
           ElseIf ($urls.IndexOf($url) -eq 33){
                $currentDir = "goxsd1706"
           }

The next thing I edited/added, thanks to @Matt for the recommendation is the try catch blocks which are clearly noted in the code. I should of had these to start with as by not having them before I was assuming that the script was always going to work. Rookie mistake and point taken.With that being said, these are all my edits. The code is working fine, but improvement is always possible. Thank you for your time and answers.

# set date
$date = Get-Date -UFormat "%Y-%m-%d-%H_EST"

# delete log files from prev Run
Remove-Item C:\alerts\logs\*.*

# setup download links
$urls = "http://subdomain.domain.com:portnumber/LogA/API_DBG_CS_Logs/dbg_a.$date.log"



function DownloadFMOSLogs() 
{
    try
    {
         # assign working dir to currentDir
         $currentDir = "goxsd1703"

         # instantiate web-client.
         $wc = New-Object System.Net.WebClient


          # loop through each url
          foreach ($url in $urls)
          {    
               # change filename to prevent overwriting of log file
               If ($urls.IndexOf($url) -eq 11){
                    $currentDir = "goxsd1704"
               }
               ElseIf ($urls.IndexOf($url) -eq 22){
                    $currentDir = "goxsd1705"
               }
               ElseIf ($urls.IndexOf($url) -eq 33){
                    $currentDir = "goxsd1706"
               }

               # get file name
               $fileName = $url.SubString($url.LastIndexOf('/')+1)

               # create target file name
               $targetFileName = "C:\alerts\logs\" + $currentDir + "_" + $fileName 
               $wc.DownloadFile($url, $targetFileName)
               Write-Host "Downloaded $url to file location $targetFileName"             
          }
     } catch [System.Net.WebException],[System.IO.IOException]
       {
            "An error occurred. Files were not downloaded."
       }
  }

DownloadFMOSLogs
Write-Host ""
Write-Host "Download of application log files has successfully completed!"

Matt · Accepted Answer

You have a couple of problems and an issue or two.

$urls is not an array like you think it is. It is actually one whole string. Try something like this instead:
```
$urls = "http://subdomain.domain.com:port/LogA/API_DBG_CS_Logs/dbg_a.$date.log",
    "http://subdomain.domain.com:port/LogA/API_DBG_CS_Logs/dbg_b.$date.log"
```
The variable will expand in that string just fine. The issue before is that you were concatenating the string starting from the first part because of the order of operations. When you add an array to a string on the left hand side the array gets converted to a space delimited string. Have a look at a smaller example which is exactly what you tried to do.
```
"hello" + 2,2 + "there"
```
You could have made what you had work if you wrapped each one in a set of brackets first.
```
("hello" + 2),(2 + "there")
```
This code might make sense elsewhere but as others have pointed out you have a useless loop about lines in a file. foreach($line in Get-Content .\hosts.txt). If you don't use it get rid of it.
You don't really use $targetDir to its full potential. If you are going to use the working directory of the script at least use some absolute paths. Side note the comments don't really match what is happening which is likely related to 2. above
```
# preprend host to file name to keep file names diff.
$targetFilePath = [io.path]::combine($pwd,"test.txt")

# download the files.
$wc.DownloadFile($link, $targetFilePath)
```
You should try and make that unique somehow since the files will overwrite eachother as you have it coded.

I would also wrap that in a try block in case the download fails and you can report properly on that. As of now you assume it will work every time.

How to download multiple files with powershell

Answers (2)

Related Questions