CODEWITHSUNDEEP
powershell
Matt Roberts
Matt Roberts

Reputation: 11

Read Invoke-WebRequest line by line

I'm trying to keep a central list of log file locations where my log file cleanup script can grab the most up to date list.

$logpaths = (Invoke-WebRequest -UseBasicParsing -Uri 'http://10.7.58.99/logpaths.txt').Content

foreach($logpath in $logpaths)
{
    "line"
    $logpath
}

My script was sort of working but I was seeing some strange behavior so when I broke it down I found that within the foreach loop it just loops once and dumps the entire contents.

If I download the file the a text file on the local machine I can then use [System.IO.File]::ReadLines and it steps through perfectly. However, I don't want to download the file each time I run it or store it on the local server at all for that matter. How can I step through the content of Invoke-WebRequest line by line?

Upvotes: 1

Views: 1651

Answers (1)

marsze
marsze

Reputation: 17055

Based on this example from the .NET docs, you could read a response stream line-by-line like this, which should have better performance.

$url = 'http://10.7.58.99/logpaths.txt'
& {
    $myHttpWebRequest = [System.Net.WebRequest]::Create($url)
    $myHttpWebResponse = $myHttpWebRequest.GetResponse()
    $receiveStream = $myHttpWebResponse.GetResponseStream()
    $encode = [System.Text.Encoding]::GetEncoding("utf-8")
    $readStream = [System.IO.StreamReader]::new($receiveStream, $encode)
    while (-not $readStream.EndOfStream) {
        $readStream.ReadLine()
    }
    $myHttpWebResponse.Close()
    $readStream.Close()
} | foreach {
    $logPath = $_
}

You might want to turn this into a nice little function. Let me know if you need help.

Upvotes: 1

Related Questions