Powershell Script - Merge Multiple Nessus Scans -

Question

So I am attempting to automate the process of merging multiple Nessus scans, following the manual guide defined at Ryker Exum. The challenge I'm having is the part where I have to find and delete lines within files up to and including a certain point (once a specific string has been found). My goal is to do this as efficiently as possible given some of these Nessus scan results (XML files) can be over 100MB. Thus my approach was to:

1. Put some logic in place to identify the first and last file, and act accordingly on them.
2. Remove the last 33 characters of all but the first scan file I come across.
3. Get the content of each file and read each object in one at a time. if there is not a match, delete the line and move on to the next object. If there is a match, delete the line and stop (thus the do until).

At this point, I've not had any success getting step three to work. The code is as follows:

$first = Get-ChildItem ".\" -Filter *.nessus | Select-Object -first 1
$last = Get-ChildItem ".\" -Filter *.nessus | Select-Object -last 1

if ($first -ne $last)
{
    Get-ChildItem ".\" -Filter *.nessus | Foreach-Object {

        $filepath = $_.FullName

        if ($first -eq  $_ -and $last -ne $_)
        {
            $stream = [System.IO.File]::OpenWrite($_.FullName)
            $stream.SetLength($stream.Length - 33)
            $stream.Close()
            $stream.Dispose()
        }

        if ($first -ne  $_ -and $last -ne  $_)
        {
            $stream = [System.IO.File]::OpenWrite($_.FullName)
            $stream.SetLength($stream.Length - 33)
            $stream.Close()
            $stream.Dispose()

            $found = ""

            do
            {
                Get-Content $_.FullName | Foreach-Object {

                    $found = $_.Contains("<Report name=")

                    if ($found)
                    {
                        Where-Object {$_ -match '<Report name='} | Set-Content $filepath
                    } else {
                        Where-Object {$_ -notmatch '<Report name='} | Set-Content $filepath
                    }
                }
            } until ($found)
        }

        if ($last -eq  $_ -and $first -ne $_)
        {
            $found = ""

            do
            {
                Get-Content $_.FullName | Foreach-Object {

                    $found = $_.Contains("<Report name=")

                    if ($found)
                    {
                        Where-Object {$_ -match '<Report name='} | Set-Content $filepath
                    } else {
                        Where-Object {$_ -notmatch '<Report name='} | Set-Content $filepath
                    }
                }
            } until ($found)
        }
    }
}

Thoughts or comments anybody?

Answer 1

I think that looks really complicated. Instead I would check if it's the first file, if it is I'd tell it not to skip any lines, if it isn't the first file I'd have it find that <Report name= string, and skip all lines up to and including that.

Then I'd check if it's the last file, and if it is I would have it read the whole rest of the file, but if it's not the last file I'd have it read all but the last 2 lines of the file beyond what it is potentially reading.

Once I knew how many lines to skip at the beginning and/or end of the file I'd have it read the appropriate lines, and output to a new file, appending as needed. The code for all of that looks like this:

$First = GCI ".\" -Filter *.nessus|Select -First 1
$Last = GCI ".\" -Filter *.nessus|Select -Last 1
GCI GCI ".\" -Filter *.nessus|%{
    If($_.Name -ne $First.Name){$SkipLines = (Select-String -Path $_.FullName -SimpleMatch "<Report name="|select -expand LineNumber)+1}else{$SkipLines = 0}
    If($_.Name -ne $Last.Name){$ReadLines = (GC $_.FullName).Count - 2 - $SkipLines} else {$ReadLines = (GC $_.FullName).Count - $SkipLines}
    GC $_.FullName | Select -First $ReadLines -Skip $SkipLines | Out-File ".\Merged.nessus" -Append
}