CODEWITHSUNDEEP
linuxperlloggingfedora
Thomas ryolas
Thomas ryolas

Reputation: 43

How to handle updates from an continuous process pipe in Perl

I am trying to follow log files in Perl on Fedora but unfortunately, Fedora uses journalctl to read binary log files that I cannot parse directly. This, according to my understanding, means I can only read Fedora's log files by calling journalctl.

I tried using IO::Pipe to do this, but the problem is that $p->reader(..) waits until journalctl --follow is done writing output (which will be never since --follow is like tail -F) and then allows me to print everything out which is not what I want. I would like to be able to set a callback function to be called each time a new line is printed to the process pipe so that I can parse/handle each new log event.

use IO::Pipe;

my $p = IO::Pipe->new();
$p->reader("journalctl --follow"); #Waits for process to exit

while (<$p>) {
  print;
}

Upvotes: 4

Views: 522

Answers (2)

Borodin
Borodin

Reputation: 126742

I have no access to journalctl, but if you avoid IO::Pipe and open the piped output directly then the data will not be buffered

use strict;
use warnings 'all';

open my $follow_fh, '-|', 'journalctl --follow' or die $!;

print while <$follow_fh>;

Upvotes: 0

Sebastian
Sebastian

Reputation: 2550

I assume that journalctl is working like tail -f. If this is correct, a simple open should do the job:

use Fcntl; # Import SEEK_CUR

my $pid = open my $fh, '|-', 'journalctl --follow'
    or die "Error $! starting journalctl";
while (kill 0, $pid) {
    while (<$fh>) {
        print $_; # Print log line
    }
    sleep 1; # Wait some time for new lines to appear
    seek($fh,0,SEEK_CUR); # Reset EOF
}

open opens a filehandle for reading the output of the called command: http://perldoc.perl.org/functions/open.html

seek is used to reset the EOF marker: http://perldoc.perl.org/functions/seek.html Without reset, all subsequent <$fh> calls will just return EOF even if the called script issued additional output in the meantime.

kill 0,$pid will be true as long as the child process started by open is alive.

You may replace sleep 1 by usleep from Time::HiRes or select undef,undef,undef,$fractional_seconds; to wait less than a second depending on the frequency of incoming lines.

AnyEvent should also be able to do the job via it's AnyEvent::Handle.

Update:

Adding use POSIX ":sys_wait_h"; at the beginning and waitpid $pid, WNOHANG) to the outer loop would also detect (and reap) a zombie journalctl process:

while (kill(0, $pid) and waitpid($pid, WNOHANG) != $pid) {

A daemon might also want to check if $pid is still a child of the current process ($$) and if it's still the original journalctl process.

Upvotes: 2

Related Questions