CODEWITHSUNDEEP
unixcommand-linegrep
user1345658
user1345658

Reputation: 25

Alternative to slow grep

Currently I have a system log (Growl.app in this case) that I'd like to see from time to time. I use grep to filter out the lines I do not wish to see.

cat /Users/Daniel/Library/Logs/Growl.log | grep -vE 'registered|Display frame|Reserving|Used rects|Beginning a pass|Successfully reserved|Adjusted display frame|---|User went idle|User returned|positionDisplay|primaryDirection|Bowtie:|secondaryDirection' | tail -20 > /Users/Daniel/Library/Logs/Growl-log.txt
sleep 2
qlmanage -p /Users/Daniel/Library/Logs/Growl-log.txt

I'm not sure if there's a better way to filter out the information but as it is my script takes quite a bit of time to process that command. I may as well add the my hardware is definitely not holding it back. Another thing to note is the 'qlmanage' part is an OS X specific command that displays the text file's contents in a 'quick look' window. It is instantaneous when displaying requests.

Any ideas?

Upvotes: 0

Views: 676

Answers (1)

jim mcnamara
jim mcnamara

Reputation: 16389

You are asking the system to read a really big, probably continuously growing file, remove a bunch of records based on a large number of searches( each alternation : |z|c|V ..etc). And every time you ask it to run it has to plow through ever more records.

Does the log have a timestamp? In conjunction with a timestamp (or even better,tail) , grep, and you can have a job run for you every day or every hour that has what you want right there in a far smaller file.

Basically you cannot read 100's of MB doing all that you ask in a few seconds. Let's start with tail, which works by positioning a file pointer instead of reading the whole file. Faster.

tail -20000 /Users/Daniel/Library/Logs/Growl.log | 
grep -vE 'registered|Display frame|Reserving|Used rects|Beginning a pass|Successfully reserved|Adjusted display frame|---|User went idle|User returned|positionDisplay|primaryDirection|Bowtie:|secondaryDirection' |
> /Users/Daniel/Library/Logs/Growl-log.txt
qlmanage -p /Users/Daniel/Library/Logs/Growl-log.txt

Adjust the 20000 to get a reasonable nunber of samples that pass your filter. In other words prefilter the file by removing early records which you will pitch anyway.

Another speed up is to do a positive search - grep for what you want, rather than what you do not want.

Congratulations.You also won the UUOC award. Try Mr google for the the term UUOC (has to do with cat). Everyone has won the award before they learned about it

Upvotes: 2

Related Questions