CODEWITHSUNDEEP
powershellpowershell-3.0powershell-ise
CM2K
CM2K

Reputation: 883

Using wildcards to download a file with Invoke-WebRequest

I have an url in which there are 6 digits which are changing daily.

Sample website: https://www.ecb.europa.eu/paym/coll/assets/html/dla/ea_MID/ea_csv_160126.csv

This is the part which changes: 160126

I don't know the correct syntax but as a form of pseudo code:

$url = "https://www.ecb.europa.eu/paym/coll/assets
/html/dla/ea_MID/ea_csv_" + [0-9][0-9][0-9][0-9][0-9][0-9]+ ".csv"

How can I write this string?

To answer the comments, I use it to download that file to a folder, like this:

"https://www.ecb.europa.eu/paym/coll/assets/html/dla/ea_MID/ea_csv_" + [0-9]+[0-9]+[0-9]+[0-9]+[0-9]+[0-9] +".csv"
$output = "C:\MyFolder\SomeSubFolder\ScriptDownload"
$start_time = Get-Date

Invoke-WebRequest -Uri $url -OutFile $output  
Write-Output "Time taken: $((Get-Date).Subtract($start_time).Seconds) second(s)"

Upvotes: 2

Views: 4341

Answers (3)

Martin
Martin

Reputation: 1963

You can parse the download page for your filenames by downloading the page or use get-ElementById etc. I assume, this is the original download page

This is your Download URL:

    $Url = "https://www.ecb.europa.eu/paym/coll/assets/html/list-MID.en.html"
$page = Invoke-WebRequest -Uri $Url
$a = ($page.ParsedHtml.getElementsByTagName('table') | ? {$_.classname -eq 'ecb-contentTable'}).textContent
$filename =  $a.Substring($a.IndexOf('ea_csv_'), 17)
$DLURL =  "https://www.ecb.europa.eu/paym/coll/assets/html/dla/ea_MID/" + $filename

Gives:

$DLURL
https://www.ecb.europa.eu/paym/coll/assets/html/dla/ea_MID/ea_csv_160126.csv

Complete that with your

$output = "C:\MyFolder\SomeSubFolder\ScriptDownload\" + $filename
Invoke-WebRequest -Uri $DLURL -OutFile $Output

and its done.

Upvotes: 2

Matt
Matt

Reputation: 46710

What you are asking for cannot be done. However there are better, more reliable ways to get the same result you are looking for.

I'm with Martin. I also found the download page he did. The better way to do this is get the link. Now this is probably not the best way to get the information but it is a start in the right direction.

Note this is slow as hell. Mostly because of Invoke-WebRequest

$start_time = Get-Date
$output = "C:\MyFolder\SomeSubFolder\ScriptDownload"
# Browse to the page hosting the csv file.
$request = Invoke-WebRequest "https://www.ecb.europa.eu/paym/coll/assets/html/list-MID.en.html"
# Locate the uncompressed CSV file name from the page
$filename = $request.ParsedHtml.getElementsByTagName("a") | Where-Object{$_.nameProp -match "^ea_csv_\d{6}\.csv$"} | Select -ExpandProperty nameProp
$fileurl = "https://www.ecb.europa.eu/paym/coll/assets/html/dla/ea_MID/$filename"

# Get the file the is hosted today.
Invoke-WebRequest -Uri $fileurl -OutFile "$output\$filename" 
Write-Output "Time taken: $((Get-Date).Subtract($start_time).Seconds) second(s)"

The way we find the right file name is with ^ea_csv_\d{6}\.csv$ which matches a name where it is exactly "ea_csv_[6 digits].csv".

Upvotes: 1

RB.
RB.

Reputation: 37192

The 6 digits are the date encoded as YYMMDD, correct? If so, you can generate a URL for the current day with:

$currentDay = $(get-date).ToString("yyMMdd")
$url = "https://www.ecb.europa.eu/paym/coll/assets/html/dla/ea_MID/ea_csv_$currentDay.csv"

Upvotes: 1

Related Questions