CODEWITHSUNDEEP
powershell
petwolfe
petwolfe

Reputation: 158

How to remove blank lines from text file using powershell

I am building a parser in powershell for converting vmstat log dumps to CSV files as an input to a graphing framework (Rickshaw). I have repeating 'headers' in the file which I would like to remove. Data sample is as below:

Tue Sep  1 14:03:26 2015: procs -----------memory---------- ---swap-- -----io---- --system-- -----cpu-----
    Tue Sep  1 14:03:26 2015:  r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa st
    Tue Sep  1 14:03:26 2015:  0  1 224412 358316 248772 63286912    0    0   388   267    1    1  8  0 91  1  0    
    Tue Sep  1 14:03:36 2015:  0  0 224412 357572 248796 63286916    0    0     0     8  220  261  0  0 100  0  0   
    Tue Sep  1 14:03:46 2015:  0  0 224412 357696 248808 63286916    0    0     0    14  276  293  0  0 100  0  0   
    Tue Sep  1 14:03:56 2015:  0  0 224412 357688 248808 63286916    0    0     0    13  231  269  0  0 100  0  0   
    Tue Sep  1 14:04:06 2015:  0  0 224412 357300 248812 63286920    0    0     0    17  266  283  0  0 100  0  0   
    Tue Sep  1 14:06:56 2015: procs -----------memory---------- ---swap-- -----io---- --system-- -----cpu-----
    Tue Sep  1 14:06:56 2015:  r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa st
    Tue Sep  1 14:06:56 2015:  1  0 224412 357348 248976 63286928    0    0     0     1  182  231  0  0 100  0  0   
    Tue Sep  1 14:07:06 2015:  0  0 224412 357348 248980 63286928    0    0     0     9  211  251  0  0 100  0  0   
    Tue Sep  1 14:07:16 2015:  0  0 224412 357136 248988 63286928    0    0     0    19  287  279  0  0 100  0  0   
    Tue Sep  1 14:07:26 2015:  0  0 224412 357012 249004 63286928    0    0     0     9  199  244  0  0 100  0  0   
    Tue Sep  1 14:07:36 2015:  0  0 224412 357080 249012 63286928    0    0     0     7  235  258  0  0 100  0  0   
    Tue Sep  1 14:10:26 2015: procs -----------memory---------- ---swap-- -----io---- --system-- -----cpu-----
    Tue Sep  1 14:10:26 2015:  r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa st
    Tue Sep  1 14:10:26 2015: 12  0 224400 351832 265992 62560000    6    0    15 25262 8579  617 96  4  0  0  0    
    Tue Sep  1 14:10:36 2015: 12  0 224400 379200 266064 62444728    0    0     2 16727 8418  761 97  3  0  0  0

I use this bit of code to get that done.

Get-Content "C:\Projects\Play\Garage\Data_Processing\Sampler.log" | select-string -pattern 'procs|swpd' -notmatch | Out-File "C:\Projects\Play\Garage\Data_Processing\Refined.log"

The resulting file has the desired lines removed but instead have blank lines inserted at the beginning and towards the end. Because of this, I am unable to send this data/file to the next step of parsing. What could I be doing wrong?

Resultant File data:

> [BLANK LINE]
    Tue Sep  1 14:03:26 2015:  0  1 224412 358316 248772 63286912    0    0   388   267    1    1  8  0 91  1  0    
    Tue Sep  1 14:03:36 2015:  0  0 224412 357572 248796 63286916    0    0     0     8  220  261  0  0 100  0  0    
    Tue Sep  1 14:03:46 2015:  0  0 224412 357696 248808 63286916    0    0     0    14  276  293  0  0 100  0  0    
    Tue Sep  1 14:03:56 2015:  0  0 224412 357688 248808 63286916    0    0     0    13  231  269  0  0 100  0  0    
    Tue Sep  1 14:04:06 2015:  0  0 224412 357300 248812 63286920    0    0     0    17  266  283  0  0 100  0  0    
    Tue Sep  1 14:06:56 2015:  1  0 224412 357348 248976 63286928    0    0     0     1  182  231  0  0 100  0  0    
    Tue Sep  1 14:07:06 2015:  0  0 224412 357348 248980 63286928    0    0     0     9  211  251  0  0 100  0  0    
    Tue Sep  1 14:07:16 2015:  0  0 224412 357136 248988 63286928    0    0     0    19  287  279  0  0 100  0  0    
    Tue Sep  1 14:07:26 2015:  0  0 224412 357012 249004 63286928    0    0     0     9  199  244  0  0 100  0  0    
    Tue Sep  1 14:07:36 2015:  0  0 224412 357080 249012 63286928    0    0     0     7  235  258  0  0 100  0  0    
    Tue Sep  1 14:10:26 2015: 12  0 224400 351832 265992 62560000    6    0    15 25262 8579  617 96  4  0  0  0    
    Tue Sep  1 14:10:36 2015: 12  0 224400 379200 266064 62444728    0    0     2 16727 8418  761 97  3  0  0  0    
>[BLANK LINE]
>[BLANK LINE]
>[BLANK LINE]

Upvotes: 2

Views: 3493

Answers (1)

Bluecakes
Bluecakes

Reputation: 2149

Not sure why Select-String is making empty lines but you could replace Select-String with a simple Where-Object which would not return the empty lines

Here's how i would do it:

Get-Content "C:\Projects\Play\Garage\Data_Processing\Sampler.log" | Where-Object -FilterScript {$_ -notmatch 'procs|swpd'} | Out-File "C:\Projects\Play\Garage\Data_Processing\Refined.log"

Upvotes: 4

Related Questions