Powershell v2.0 Using multiple threads

Question

Basic script idea: Hello. I've created a powershell script which I use to check the filesizes of certain executables, and then keep them in a text file. Next time the script runs, if a filesize differs it will replace the one in the text file with the new one.

The structure: I have a main script and a folder which contains many scripts, each for every executable of which I want to check the filesize. So the scripts in the folder will return a string containing the link to the executable, which will be fed to the main script.

The code:

$progdir = "C:\script\programms"
$items = Get-ChildItem -filter *.ps1 -Path $progdir
$webclient = New-Object System.Net.WebClient

$filesizes = get-content C:\updatechecker\programms\filesizes
if ($filesizes.length -ne $items.length) { 
    if ($filesizes.length -eq $null) {
        Write-Host ("Building filesize database...") -nonewline
    } 
    else { 
        Write-Host ("Rebuilding filesize database...") -nonewline 
    }
    clear-content  C:\programms\filesizes

    for ($i=0; $i -le $items.length-1; $i++) {
        $command = "c:\programms\" + $items[$i].name
        $link = & $command
        $webclient.OpenRead($link) | Out-Null
        $filesize = $webclient.ResponseHeaders["Content-Length"]
        $filesize >> C:\programms\filesizes
    }
    echo "Done." 
} 
else {
    ...

Question: This for loop is the one I want to run in parallel. I need your advice on how to do this since I'm new to powershell. I tried to implement a few things I found but they didn't work correctly (took very long to finish, output errors, multiple entries of filesizes in my filesizes file). I suspect it's a synchronization issue and somehow I need to lock the critical parts. Isn't there anything like omp parallel for in powershell? :P

Any help,advice on how to achieve this would be appreciated :)

edit:

Get-Job | Remove-Job -Force
$progdir = "C:\programms"
$items = Get-ChildItem -filter *.ps1 -Path $progdir
$webclient = New-Object System.Net.WebClient
$filesizes = get-content C:\programms\filesizes

$jobWork = {
    param ($MyInput)
    $command = "c:\programms\" + $MyInput
    $link = & $command
    $webclient.OpenRead($link) | Out-Null
    $filesize = $webclient.ResponseHeaders["Content-Length"]
    $filesize >> C:\programms\filesizes

}
foreach ($item in $items) {

    Start-Job -ScriptBlock $jobWork -ArgumentList $item.name | out-null
}

Get-Job | Wait-Job
Get-Job | Receive-Job | Out-GridView | out-null
echo "Done."

Edit 2: Used code I found here: http://ryan.witschger.net/?p=22

$mutex = new-object -TypeName System.Threading.Mutex -ArgumentList $false, “RandomGlobalMutexName”;
$MaxThreads = 4
$SleepTimer = 500


$jobWork = {
    param ($MyInput)
    $webclient = New-Object System.Net.WebClient
    $command = "c:\programms\" + $MyInput
    $link = & $command
    $webclient.OpenRead($link) | Out-Null

    $result = $mutex.WaitOne();

    $file = $webclient.ResponseHeaders["Content-Length"]
    $file >> C:\programms\filesizes

    $mutex.ReleaseMutex();
}

$progdir = "C:\programms"
$items = Get-ChildItem -filter *.ps1 -Path $progdir
$webclient = New-Object System.Net.WebClient
$filesizes = get-content C:\programms\filesizes

Get-Job | Remove-Job -Force

$i = 0

ForEach ($item in $items){
    While ($(Get-Job -state running).count -ge $MaxThreads){

        Start-Sleep -Milliseconds $SleepTimer
    }

    $i++
    Start-Job -ScriptBlock $jobWork -ArgumentList $item.name | Out-Null


}

Answer 1

You can run each iteration of the loop in a background job which is not the same a seperate thread in that it is a whole other PowerShell.exe process. Data is passed from the background processes through serialization.

To approach it using background jobs you'll need to define a script block that will do that actual work and then call the script block with parameters in each iteration of the loop. The script block can report back status via Write-Output or by throwing an exception.

You'll probably want to throttle how many concurrent background jobs are running. Here's an example of how to throttle:

$jobItems = "a", "b", "c", "d", "e"
$jobMax = 2
$jobs = @()

$jobWork = {
    param ($MyInput)
    if ($MyInput -eq "d") {
        throw "an example of an error"
    } else {
        write-output "Processed $MyInput"
    }
}

foreach ($jobItem in $jobItems) {
    if ($jobs.Count -le $jobMax) {
        $jobs += Start-Job -ScriptBlock $jobWork -ArgumentList $jobItem
    } else {
        $jobs | Wait-Job -Any
    }
}
$jobs | Wait-Job

As an alternative you might try eventing. Take a look at this thread for some examples of how to implement concurrency using events.

PowerShell: Runspace problem with DownloadFileAsync

You might be able to replace DownloadFileAsync with OpenReadAsync