CODEWITHSUNDEEP
powershellforeachbatch-processingsynchronousstreamwriter
Jensen010
Jensen010

Reputation: 445

Tell Powershell to wait for foreach to finish writing to txt file

I have a .ps1 script that takes some of our internal SQL data and syncs it to a Google API.

It does this by running a foreach against our data, using logic to generate the API commands, and writing those commands to a text file via System.IO.StreamWriter. Then the API processor runs a batch job on the file.

The issue I'm having is that the batch job portion appears to be firing before the foreach has finished writing to the file, which makes the whole script fail.

Here's a simplified version of my code:

$stream = [System.IO.StreamWriter] "C:\GAM\uploads\stream\gamBatch$today.txt";

## Loop csv and generate Google OU path
Write-Host "Generating location strings, OU paths, and update commands..."

Import-CSV "C:\uploads\Upload$today.csv" | ForEach-Object {

  ## Define fancy vars here
  ## Do fancy logic here

  ## Stream command list to log file
  $line = "update $deviceId assetid $Barcode location $location ou $ouPath";
  $stream.WriteLine($line);
};

## Trim top line of batch file (Removes header line)
(Get-Content "C:\uploads\stream\Batch$today.txt" | Select-Object -Skip 1) | Set-Content "C:\uploads\stream\Batch$today.txt";

## Close Stream instance and bulk run commands
Write-Host "Running batch command..."

$stream.WriteLine("commit-batch");
$stream.close();

apiBatch "C:\uploads\stream\Batch$today.txt";

This generates this error in my logs:

PS>TerminatingError(Set-Content): "The process cannot access the file 
'C:\uploads\stream\Batch2020-05-14.txt' because it is being used by another process."

How can I make Powershell wait for the txt file to finish being written before firing the batch command?

Upvotes: 0

Views: 1383

Answers (1)

Mathias R. Jessen
Mathias R. Jessen

Reputation: 175085

There's no need to write the first line to the file, only to then remove it again immediately after (and having to rewrite the rest of the file).

If you're not doing anything meaningful with it inside the ForEach-Object loop, skip it immediately:

Import-CSV "C:\uploads\Upload$today.csv" | Select-Object -Skip 1 | ForEach-Object {
    # ...
    $stream.WriteLine($line)
}

If you need to inspect the first line inside the ForEach-Object body, make sure you skip calling WriteLine() on the first iteration:

$first = $true
Import-CSV "C:\uploads\Upload$today.csv" | ForEach-Object {
    # ...
    if($first){
        $first = $false
    }
    else{
        $stream.WriteLine($line)
    }
}

Alternatively, close the StreamWriter before re-writing the file with Set-Content, and then either use Add-Content, or a new stream writer to write the last line:

Import-CSV "C:\uploads\Upload$today.csv" | ForEach-Object {
    # ...
    $stream.WriteLine($line)
}

# dispose of the current writer (will close the file stream)
$stream.Dispose()

... | Set-Content "C:\uploads\stream\Batch$today.txt"

# open a new writer and append the last string
$stream = (Get-Item "C:\uploads\stream\Batch$today.txt").AppendText()
$stream.WriteLine("commit-batch")

# or use `Add-Content`
"commit-batch" | Add-Content "C:\uploads\stream\Batch$today.txt"

Upvotes: 1

Related Questions