CODEWITHSUNDEEP
powershellreplacescriptingwindows-scripting
David Lago
David Lago

Reputation: 357

Filter a file content to exclude all the lines with matching strings from another file in PowerShell

I need to filter the content of a variable to exclude all the lines with matching strings from a SubnetExceptions.txt file as filter, just as a grep -v command.

The $Valuevariable suffered lots of treatments in order to achieve the format I need.

The working part of the code is as follows:

$Value=netsh dhcp server show mibinfo | findstr "Subnet Addresses"
$Value=$Value -replace "Subnet","% `n Subnet"
$Value=$Value -replace "No. of Addresses in use","AddressesInUse"
$Value=$Value -replace "No. of free Addresses","AddressesFree"
$Value=$Value -replace '^.|.$', ' '
$Value=$Value -replace '    ', ""
$Value=$Value -replace '  ', " "
$Value= -join $Value 
$Value=$Value -replace '\n', ''
$Value=$Value -replace "% ", "`n"
$Value=$Value -replace ' = ', '='

These first 11 lines of the ps1 treats the output ($Value) of that command from this:

    Subnet = 10.1.8.0.
        No. of Addresses in use = 11.
        No. of free Addresses = 18.
    Subnet = 10.1.9.0.
        No. of Addresses in use = 1.
        No. of free Addresses = 201.
    Subnet = 10.1.11.0.
        No. of Addresses in use = 188.
        No. of free Addresses = 61.
    Subnet = 10.1.12.0.
        No. of Addresses in use = 207.
        No. of free Addresses = 44.
    Subnet = 10.1.13.0.
        No. of Addresses in use = 149.
        No. of free Addresses = 100.

to this:

Subnet=10.1.8.0  AddressesInUse=11  AddressesFree=18  
Subnet=10.1.9.0  AddressesInUse=1  AddressesFree=201  
Subnet=10.1.11.0  AddressesInUse=188  AddressesFree=61  
Subnet=10.1.12.0  AddressesInUse=207  AddressesFree=44  
Subnet=10.1.13.0  AddressesInUse=149  AddressesFree=100

These replace lines work on linebreaks, dots, spaces, and "variable=value" formatting. I'll need the subnet values to sit in the same line so I can filter them out, hence the string treatments.

The actual output of the command is 278 lines long with more of the same, so I trimmed it to 5 lines in order to keep it minimal to allow reproducing it in lab.

The content of the filter file (C:\Scripts\SubnetExceptions.txt) reads as follows: 

10.1.12.0
10.1.13.0

These are the last two subnet values, that I want to filter out. That's what has been tested so far (adding it right below the treatments above):

$Filter = (Get-Content '.\SubnetExceptions.txt' |
          ForEach-Object {[regex]::Escape($_)}) -join '|'
($Value) -notmatch $Filter | Set-Content '.\output.txt'

The expected result should be:

Subnet=10.1.8.0  AddressesInUse=11  AddressesFree=18
Subnet=10.1.9.0  AddressesInUse=1  AddressesFree=201
Subnet=10.1.11.0  AddressesInUse=188  AddressesFree=61

Where the last 2 lines would be removed due to the filter, but instead the output file writes only the value "false".

Upvotes: 1

Views: 4228

Answers (2)

Ansgar Wiechers
Ansgar Wiechers

Reputation: 200293

Build a regular expression from the content of your second file:

$re = (Get-Content '.\filter.txt' | ForEach-Object {[regex]::Escape($_)}) -join '|'

Then use that filter to exclude matching lines from your first file:

(Get-Content '.\MyList.txt') -notmatch $re | Set-Content '.\output.txt'

Use Tee-Object instead of Set-Content if you need the output written to both a file and STDOUT.

Edit:

Now that we can see the broader picture we might be able to offer some more thorough advice. For one thing I would suggest to change your handling of the netsh output to something like this:

$Value = netsh dhcp server show mibinfo | findstr "Subnet Addresses" | Out-String
$Value = $Value -replace '(?ms)\.\r?\n\s+No\. of ', "`t"
$Value = $Value -replace 'Addresses in use', 'AddressesInUse'
$Value = $Value -replace 'free Addresses', 'AddressesFree'
$Value = $Value -replace ' '
$Value = $Value -split '\r?\n'
$Value = $Value -replace '\.$'

That will give you a list of tab-separated rows that are probably easier to handle than the single string you have now.

Note that you could also daisy-chain the operations listed above in a single statement:

$Value = (netsh dhcp server show mibinfo | findstr "Subnet Addresses" | Out-String) `
         -replace '(?ms)\.\r?\n\s+No\. of ', "`t" `
         -replace 'Addresses in use', 'AddressesInUse' `
         ...

I would, however, stick with the form you chose despite the multiple assignments, because it provides better maintainability. You can easily disable or enable individual operations simply by commenting/uncommenting a line.

With your data in the form of a list of rows the regular expression filter will now work as expected:

$Value -notmatch $Filter | Tee-Object -FilePath '.\output.txt'

The reason why that didn't work before is that you had a single string instead of an array of strings.

With all of that said, I would strongly recommend to re-design your solution. While PowerShell is perfectly capable of parsing and handling strings you're ignoring many features that could make your life a lot easier.

For one thing, there are DHCP cmdlets that will give you the data in object form, removing the need to parse the string output of netsh. But even if you can't use them for some reason you can still transform the string output into objects, e.g. like this:

...
$Value = $Value -split '\r?\n'
$Value = $Value -replace '\.$'
$obj = $Value | ForEach-Object {
    $ht = $_ -replace '\t', "`n" | ConvertFrom-StringData
    New-Object -Type PSObject -Property $ht
}

or like this:

$Value = netsh dhcp server show mibinfo | findstr "Subnet Addresses" | Out-String
$Value = $Value -replace '(?ms)\.\r?\n\s+No\. of ', "`t"
$Value = $Value -replace ' '
$obj = $Value -split '\r?\n' | Where-Object {
    $_ -match '=(\d+\.\d+\.\d+\.\d+)\t.*?=(\d+)\t.*?=(\d+)'
} | ForEach-Object {
    New-Object -Type PSObject -Property @{
        'Subnet' = $matches[1]
        'Used'   = [int]$matches[2]
        'Free'   = [int]$matches[3]
    }
}

With your data in the form of a list of objects like that you can simplify your filtering to something like this, because you can now compare individual properties instead of having to match partial strings:

$Filter = Get-Content '.\SubnetExceptions.txt'
$obj | Where-Object {
    $Filter -notcontains $_.Subnet
} | Export-Csv '.\output.csv' -NoType

You can also export the data in a structured format like CSV that will allow for simplified transfer/import should further processing of the data be required.

Upvotes: 7

2 B
2 B

Reputation: 114

I would recommend you to use compare-object cmdlet.

$refference  = get-content C:\text_file1.txt
$difference = get-content C:\text_file2.txt
Compare-Object -ReferenceObject $refference -DifferenceObject $difference -IncludeEqual | Where sideindicator -ne "==" | select -ExpandProperty inputobject

Upvotes: -1

Related Questions