CODEWITHSUNDEEP
linuxbash
VickiT05
VickiT05

Reputation: 101

Check directory daily for new files - linux bash script

I'd like to monitor a directory for new files daily using a linux bash script.

New files are added to the directory every 4 hours or so. So I'd like to at the end of the day process all the files. By process I mean convert them to an alternative file type then pipe them to another folder once converted.

I've looked at inotify to monitor the directory but can't tell if you can make this a daily thing.

Using inotify I have got this code working in a sample script:

#!/bin/bash

while read line
do
        echo "close_write: $line"
done < <(inotifywait -mr -e close_write "/home/tmp/")

This does notify when new files are added and it is immediate.

I was considering using this and keeping track of the new files then processing them at all at once, at the end of the day.

I haven't done this before so I was hoping for some help.

Maybe something other than inotify will work better.

Thanks!

Upvotes: 1

Views: 1788

Answers (4)

ryenus
ryenus

Reputation: 17381

To collect the files by the end of day, just use find:

find $DIR -daystart -mtime -1 -type f

Then as others pointed out, set up a cron job to run your script.

Upvotes: 1

Niall Cosgrove
Niall Cosgrove

Reputation: 1303

Your cron job (see other answers on this page) should keep a list of the files you have already processed and then use comm -3 processed-list all-list to get the new files. man comm

Its a better alternative to
awk 'FNR==NR{a[$0];next}!($0 in a)' processed-list all-list

and probably more robust than using find since you record the ones that you have actually processed.

Upvotes: 1

drewyupdrew
drewyupdrew

Reputation: 1609

Definitely should look into using a cronjob. Edit your cronfile and put this in:

0 0 * * * /path/to/script.sh

That means run your script at midnight everyday. Then in your script.sh, all you would do is for all the files, "convert them to an alternative file type then pipe them to another folder once converted".

Upvotes: 1

Anton Krosnev
Anton Krosnev

Reputation: 4132

You can use a daily cron job: http://linux.die.net/man/1/crontab

Upvotes: 1

Related Questions