CODEWITHSUNDEEP
powershell
user2361820
user2361820

Reputation: 449

How to parse filenames to determine the newest file in each of multiple folders

I have logs that are getting written from various Linux servers to a central windows NAS server. They're in E:\log in the format:

E:\log\process1\log20140901.txt,

E:\log\process2\20140901.txt,

E:\log\process3\log-process-20140901.txt,

etc.

Multiple files get copied on a weekly basis at the same time, so created date isn't a good way to determine what the newest file is. Therefore I wrote a powershell function to parse the date out, and I'm attempting to iterate through and get the newest file in each folder, using the output of my function as the "date". I'm definitely doing something wrong.

Here's the Powershell I've written so far:

Function ReturnDate ($file)
{
    $f = $file
    $f = [RegEx]::Matches($f,"(\d{8})") | Select-Object -ExpandProperty Value
    $sqlDate = $f.Substring(0,4) + "-" + $f.substring(4,2) + "-" + $f.substring(6,2)
    return $sqlDate
}

Get-ChildItem E:\log\* |
 Where {$_.PsIsContainer} |
  foreach-object { Get-ChildItem $_ -Recurse |
   Where {!$_.PsIsContainer} |
    ForEach-Object { ReturnDate $_}|
     Sort-Object ReturnDate -Descending |
      Select-Object -First 1 | Select Name,ReturnDate
       }

I seem to be confounding properties and causing "You cannot call a method on null-valued expression errors", but I'm uncertain what to do from here.

Upvotes: 0

Views: 152

Answers (3)

Aaron Jensen
Aaron Jensen

Reputation: 26739

The Sort-Object cmdlet supports sorting by a custom script block and will sort by whatever the script block returns. So, use a regular expression to grab the timestamp and return it.

Get-ChildItem E:\log\* -Directory |
    ForEach-Object { 
        Get-ChildItem $_ -Recurse -File |
        Sort-Object -Property {
            if( $_.Name -match '(\d{8})' )
            {
                return $Matches[1]
            }
            Write-Error ('File ''{0}'' doesn't contain a timestamp in its name.' -f $_.FullName)
        } | 
        Select-Object -Last 1 | 
        Select Name,ReturnDate
    }

Note that Select-Object -First 1 was changed to Select-Object -Last 1, since dates would be sorted from oldest to newest.

Upvotes: 1

Keith Hill
Keith Hill

Reputation: 201652

I suspect your $f variable is null and you're trying to invoke a method (Substring) on a null value. Try this instead:

Get-ChildItem E:\Log -File -Recurse | Where Name -Match '(\d{8})\.' | 
    Foreach {Add-Member -Inp $_ NoteProperty ReturnDate ($matches[1]) -PassThru} | 
    Group DirectoryName | 
    Foreach {$_.Group | Sort ReturnDate -Desc | Select -First 1}

This does require V3 or higher. If you're on V1 or V2 change it to this:

Get-ChildItem E:\Log -Recurse | 
    Where {!$_.PSIsContainer -and $_.Name -Match '(\d{8})\.'} | 
    Foreach {Add-Member -Inp $_ NoteProperty ReturnDate ($matches[1]) -PassThru} |
    Group DirectoryName | 
    Foreach {$_.Group | Sort ReturnDate -Desc | Select -First 1}

Upvotes: 3

Matt
Matt

Reputation: 46710

Your code was ok for me when i tried it up until you did a select you were requesting name and returndate when those properties did not exist. Creating a custom object with those values would make your code work. Also i removed some of the logic from your pipes. End result should still work though (I just made some dummy files to test with like your examples).

Working with your original code you could have something like this. This would only work on v3 or higher. Simple changes could make it work on lower if need be. Mostly where [pscustomobject] is concerned. 

Function ReturnDate ($file)
{

    $f = $file
    $f = [RegEx]::Matches($f,"(\d{8})") | Select-Object -ExpandProperty Value
    $sqlDate = $f.Substring(0,4) + "-" + $f.substring(4,2) + "-" + $f.substring(6,2)
    [pscustomobject] @{
        'Name' = $file.FullName
        'ReturnDate' = $sqlDate
    }
}

Get-ChildItem C:\temp\E\* -Recurse |
   Where-Object {!$_PSIsContainer} | 
   ForEach-Object{ReturnDate $_} |
   Sort-Object ReturnDate -Descending |
   Select-Object -First 1

Upvotes: 1

Related Questions