CODEWITHSUNDEEP
linuxtail
user3164187
user3164187

Reputation: 1432

tail dynamically created files in linux

I am having a list of files under a directory as below,

file1
file2
file3
....
....

files will get created dynamically by a process.

now when i do tail -f file* > data.txt,

file* takes only the existing files in the directory.

for (e.g)

existing files:

file1
file2

i do : tail -f file* > data.txt

when tail in process a new file named file3 got created,

(here i need to include file3 as well in the tail without restarting the command)

however i need to stop tail and start it again so that dynamically created files also tailed.

Is there a way to dynamically include files in tail whenever there is a new file created or any workaround for this.

Upvotes: 4

Views: 2885

Answers (2)

Neil Masson
Neil Masson

Reputation: 2689

You could use inotifywait to inform you of any files created in a directory. Read the output and start a new tail -f as a background process for each new file created.

Upvotes: 1

Chris Maes
Chris Maes

Reputation: 37752

I have an anwser that satisfies most but not all of your requirements:

You can use

tail -f --follow=name --retry file1 file2 file3 > data.txt

This will keep opening the files 1,2,3 until they become available. It will keep printing the output even if on of the files disappears and reappears again.

example usage:

first create two dummy files:

echo a >> file1
echo b >> file2

now use tail (in a separate window):

tail -f --follow=name --retry file1 file2 file3 > data.txt

now append some data and do some other manipulations:

echo b >> file2
echo c >> file3
rm file1
echo a >> file1

Now this is the final output. Remark that all three files are taken into account, even though they weren't present at a certain moment:

==> file1 <==
a

==> file2 <==
b
tail: cannot open ‘file3’ for reading: No such file or directory

==> file2 <==
b
tail: ‘file3’ has become accessible

==> file3 <==
c
tail: ‘file1’ has become inaccessible: No such file or directory

==> file1 <==
a

remark: this won't work with file*, because that is a glob pattern that is expanded before execution. Suppose you do:

tail -f file*

and only file1 and file2 are present; then tail gets as input:

tail -f file1 file2

The glob expansion cannot know which files would eventually match the pattern. So this is a partial answer: if you know all the possible names of files that will be created; this will do the trick.

Upvotes: 2

Related Questions