CODEWITHSUNDEEP
powershell
Raggamuffin
Raggamuffin

Reputation: 11

Powershell - Efficient Way to Return Line Numbers from Logs

I have an extremely large log file (max 1GB) which is appended to throughout the day. There are various strings within this log which I would like to search for (that I can already achieve using Select-String) however I am scanning the whole file on every sweep which is inefficient and a tad unnecessary.

Ideally I want to scan only the last 5 minutes of the log for these strings on each sweep. Unfortunately not every row of the log file contains a timestamp. I initially thought of doing a wildcard select-string for the last 5 mins timestamps combined with the strings of interest will miss some occurrences. My only other idea at the moment is to determine the line numbers of interest, $FromLineNumber (5 mins before current system time) and $ToLineNumber (the very last line number of log file) and then only Select-String between those two line number ranges.

As an example, to search between line 50 and the final line of the log. I am able to return the line number of $FromLineNumber but I'm struggling with grabbing $ToLineNumber for final row of log.

Q. How do I return only the line number of the final row of a log file?

So far I have tried returning this with Get-Content $path -tail -1 (object type linenumber) however this always returns blank values even with various switches and variations. I can only return line numbers via the Select-String cmdlet however I do not have a specific string to use that relates to the final row of the log. Am I misusing this cmdlet per its original design and if so...is there any other alternative to return the last line number?

Continued...Once I have determined the line number range to search between would I isolate those rows using a Get-Content loop between
$FromLineNumber and $ToLineNumber first to filter down to this smaller selection and then pipe this into a Select-String or is there a more efficient way to achieve this? I suspect that looping through thousands of lines would be demanding on resources so I'm keen to know if there is a better way.

Upvotes: 1

Views: 753

Answers (1)

Novaterata
Novaterata

Reputation: 4781

Here is the answer to the first question

From https://blogs.technet.microsoft.com/heyscriptingguy/2011/10/09/use-a-powershell-cmdlet-to-count-files-words-and-lines/

If I want to know how many lines are contained in the file, I use the Measure->Object cmdlet with the line switch. This command is shown here:

Get-Content C:\fso\a.txt | Measure-Object –Line

Upvotes: 1

Related Questions