CODEWITHSUNDEEP
powershelldata-processing
Subhantech
Subhantech

Reputation: 67

How to process Custom log data using Powershell?

I have a log file which has data separated with "|" symbol. Like 

"Username|servername|access|password|group"
"Username|servername|access|password|group"

I need to validate the data. And, If the group column(record) is missing information or empty. I need to write only that row into another file. Please help me. Thanks in Advance.

Upvotes: 1

Views: 63

Answers (3)

Mark Wragg
Mark Wragg

Reputation: 23395

You could use 'Import-CSV with -Delimiter '|'. If your file doesn't have a header line, you would also need to use -Header to define it. You could then use Where to filter for the empty Group lines and Export-CSV with -Delimiter again to create a new file of just those lines.

For example:

Import-CSV 'YourLog.log' -Delimiter '|' -Header 'Username','Servername','Access','Password','Group' |
    Where {$_.'Group' -eq ''} |
    Export-CSV 'EmptyGroupLines.log' -Delimiter '|'

Upvotes: 2

mjolinor
mjolinor

Reputation: 68331

If you're just checking for missing data, you can run a quick check using a regex of '(\S+\|){4}\S+'. Use Get-Content with the -ReadCount parameter, and you can work in batches of a few thousand records at a time, minimizing disk i/o and memory usage without going through them one record at a time.

Get-Content $inputfile -ReadCount 2000 |
foreach { 
          $_ -notmatch '(\S+\|){4}\S+' |
          Add-Content $outputfile
         }

Upvotes: 3

Tim Haintz
Tim Haintz

Reputation: 676

If your group column is always in the same place, which it looks like it is, you could use the split method. You can certainly neaten the code up. I have used the below as an example as to how you could use split.

The foreach statement is to iterate through each line in your file. if (!$($groupstring.Split('|')[4])) checks if it is null.

$groupstring = 'Username|servername|access|password|group'
$groupstring.Split('|')[4]


foreach ($item in $collection)
{
    if (!$($groupstring.Split('|')[4])) 
    {
        Write-Host "variable is null" 
    }
}

Hope this helps. Thanks, Tim.

Upvotes: 0

Related Questions