CODEWITHSUNDEEP
perlfile-iotruncate
AndyB
AndyB

Reputation: 513

Reading a file that might be truncated in Perl

I'm using Perl to read through a log file that might be truncated at any point. If this happens, I want to start reading the file at the beginning again, but the default Perl behaviour seems to be to wait until the file pointer has caught up. For example, if I run the following:

perl -e 'open FILE, "test"; while (1) { $line = <FILE>; print "$line"; }'

and then do:

for i in 1 2 3; do echo $i >> test; done
:>test
for i in 4 5 6 7; do echo $i >> test; done

the output I get is:

1
2
3
7

But if I were to do the same thing, replacing the Perl code with:

tail -f test

then (ignoring tail output to stderr) I get the output I want, i.e.:

1
2
3
4
5
6
7

Obviously, I can't just use tail because I want to do some processing on the line itself. One thought I did have was combining the two:

tail -f test | perl -e 'while (1) { $line = <STDIN>; print "$line"; }'

which works on my Linux development machine but unfortunately not on the Solaris target platform.

Any ideas?

As requested, an example of using File::Tail is as follows:

perl -e 'use File::Tail; $file = File::Tail->new("test"); while (defined($line=$file->read)) { print "$line"; }'

If I then enter data into test as before, the output I get is just:

7

which is obviously not as desired. I've tried adjusting the maxinterval that File::Tail will wait as follows:

perl -e 'use File::Tail; $file = File::Tail->new(name => "test", maxinterval => 1); while (defined($line=$file->read)) { print "$line"; }'

but in that case, entering data into the test file too quickly, like so:

for i in 1 2 3; do echo $i >> test; done; :>test; for i in 4 5 6 7; do echo $i >> test; done

results in just:

4
5
6
7

Unfortunately, that's a realistic scenario for our (very busy) application. For the sake of comparison, Linux tail appears to handle that sort of data entry speed properly, so it's fairly clear that it can be done (although to be fair, maybe not in Perl....?)

Upvotes: 3

Views: 516

Answers (2)

Ilmari Karonen
Ilmari Karonen

Reputation: 50368

The Linux tail command uses the inotify mechanism to monitor changes to the file without having to repeatedly poll it for updates. This allows it to respond quickly and reliably to any changes in the file.

To implement similar functionality in Perl, you could use the Linux::Inotify2 module from CPAN. Unfortunately, as the name indicates, this module only works on Linux, so it won't help you in porting your script to Solaris.

You might be able to use FAM with the SGI::FAM module instead; it may still internally fall back to polling if there's no native file change notification mechanism available, but at least it should provide a reasonably good and tested implementation of file polling.

Upvotes: 2

DVK
DVK

Reputation: 129481

It sounds like you wish to emulate tail in Perl.

  • If you have access to CPAN, you can use File::Tail.

    It's pure Perl (not XS) so you can see how it's implemented in the source if no CPAN access.

  • Also, see PerlFAQ5 ("How do I do a tail -f in perl?")

Upvotes: 1

Related Questions