CODEWITHSUNDEEP
powershellloggingsplit
BCon995
BCon995

Reputation: 135

Powershell - Splitting one log file into multiple by Date

I have a log file that generates as a mix of dates, I need to segment this one file out into multiple files only containing entries by date. As an example, one file would be named 1-2-23 and only contain entries for Jan. 2nd, 2023. Next file would be 1-3-23 and only contain entries for Jan. 3rd, 2023. etc...

I am not sure what route would be the most efficient as they contain 1,000s of entries, however the entries are separated by the last line of the entry being:

-----------------------

EXAMPLE OF LOG FILES

entry is HOSTFIT 0
inserted
group is 0
entry is HOSTFIT 0
code entered
chose no
184357 ejected
184400 taken
184404 presented
USD
16X20 
== NOV 09, 2022 COMPNAME ==
184405 taken
 11/09/22 18:43 COMPNAME
 
12345******6789
SEQ 4201
RANDOMTEXT 320.00
FROM ****882
 
COMPLETED SUCCESSFULLY
 
 
AVAIL        32456
 
A1234567891011
PROCESS
-----------------------

inserted
group is 1
entry is HOSTFIT 1
ntry is HOSTFIT 1
192153 inserted
group is 1
entry is HOSTFIT 1
code entered
chose receipt
192221 ejected
192222 aken
192226 presented
240 USD
12X20 
192227 taken
 11/09/22 19:22 12274267
 
45678******9101
SEQ 4202
WITH %240.00
FROM

 
COMPLETED SUCCESSFULLY
 
 
AVAIL           936.77
 
A1234567891011
US 
-----------------------

ATTEMPTED CODE 1:

get-content  $Log |
select-string -SimpleMatch '-----------------------' |
set-content "C:\Users\User\Desktop\Projects\Archive\Temp\TestAudit.log"

ATTEMPTED CODE 2:

Get-ChildItem $Log -recurse | ForEach-Object { 
    Get-Content $_ | ForEach-Object { 
        $fields = $_.split('-----------------------')
        [pscustomobject]@{
            Field7 = $fields[7]
            Field8 = $fields[8]
            Field10 = $fields[10]
        }
    }
} | Export-CSV "C:\Users\User\Desktop\Projects\Archive\Temp\TestAudit.csv" -NoTypeInformation

Upvotes: 1

Views: 264

Answers (2)

BCon995
BCon995

Reputation: 135

I was able to find a solution with the direction given by @TheMadTehcnician

$InPC = "C:\Users\User\Desktop\Projects\Archive\Temp\"
Get-ChildItem -Path $InPC -Filter *.LOG | ForEach-Object -Process {

        $basename= $_.BaseName   
        $m = ( ( Get-Content $_.FullName | Where-Object { $_ | Select-String "-----------------------" -Quiet } | 
        Measure-Object | 
        ForEach-Object { $_.Count } ) -ge 2) 
        $a = 1
        if ($m) {
  Get-Content $_.FullName | ForEach-Object {

    If ($_ -match "-----------------------") {
        $OutputFile = "$InPC\$basename-$a.LOG"
        $a++
    }    
    Add-Content $OutputFile $_
    }
  Remove-Item $_.FullName 
  }
  }

Upvotes: 0

TheMadTechnician
TheMadTechnician

Reputation: 36297

I think I would use Get-Content with the -Raw parameter to get the entire file, and split it on the dashes. Then use RegEx to match either the mm/dd/yy or MMM dd, yyyy formatted date, and parse that with [DateTime]::Parse() method, and group on that date. Then for each group you can append a log for that date if you archive the logs after processing.

$InputFiles = Get-ChildItem $Log -recurse
ForEach($File in $InputFiles){
    $Records = (Get-Content $File.FullName -Raw) -split '(?ms)^-+\s*$'|?{$_}
    $Records | Group-Object {[datetime]::Parse(([regex]::Matches($_,'\b\d\d/\d\d/\d\d\b|\b\w{3} \d\d, \d{4}\b')[0]))} | ForEach-Object {
        Add-Content -Path "C:\Users\User\Desktop\Projects\Archive\Temp\$($_.Name.ToString('mm-dd-yyyy')).log" -Value $_.Group
    }
}

If you have to run the same logs over and over then that will get you a lot of duplicate data, and you'd need to de-dupe the records, which could be challenging.

Upvotes: 1

Related Questions