CODEWITHSUNDEEP
linuxbashshelltail
Germinate
Germinate

Reputation: 2098

How to use shell to continuously process tail -f new lines in real time？

There is a file on the linux server, which new lines appended to it aperiodically. I need to process new line, parse it and use other command or script to process it . Check the file periodically is not acceptable, I need a real time solution. No other language (Python, Perl) is available on the server, only shell.

Now, I'm trying to assign the new line to a shell variable, and then process it. But can not find a good way to do that. Another problem is I need to rely on some result former when processing new lines. For example, when I process the 11th line, maybe the 5th line result is required. So some variables needed to store the result before, and I need to use them in the loop.

Any solution or any better suggestion for my case?

Upvotes: 4

Views: 2554

Answers (2)

shuerguo
shuerguo

Reputation: 64

Try the code below

#!/bin/bash
process_new_line() {
    local var= '' #declare some local variables here 
    read
    while true
    do
        #process $REPLY 
        #The new line content is in the variable $REPLY 
        #store the result in the local variable according to your rules.
        read
    done
}

tail -f the_file | process_new_line

Use a function will solve your problem. You could use local variable to store the result, and $REPLY hold the new line content.

Upvotes: 4

John1024
John1024

Reputation: 113944

Try:

tail -f filename | while read -r line; do
    : # process the line
done

This will be real-time except to the extent that lines are buffered. If the buffering bothers you, there are utilities like stdbuf to shrink the buffers.

You didn't mention whether your system has awk. It is required by the POSIX standard for Unix systems. If you have it, it is excellent for "calculating, replacing some characters", etc.:

tail -f filename | awk 'awk code'

Upvotes: 3

Related Questions