CODEWITHSUNDEEP
phpmysqlxmlcron
Edward144
Edward144

Reputation: 493

Risks of running a cronjob every minute

I have a php script designed to check a certain folder for xml files and then import the information from each file to a MySQL database.

I would like to set up a cronjob to run every minute so that anytime new files are added they will almost instantly be imported without me having to manually ssh in and run the script.

I have an if statement which checks if files exist and only runs the code if they do, otherwise 'No files' is echoed.

I would like to know if there are any risks to having this run constantly, will excessive resources be taken up? etc

Upvotes: 6

Views: 2828

Answers (1)

Bret Weinraub
Bret Weinraub

Reputation: 2283

There's really nothing wrong with running a process every minute ... except the usual pitfalls [which I include with ways to mitigate]. I do want to say that a minute is a really really long time for a modern computer. If you are short of cycles, a few extra system calls per minute is the wrong place to look.

  • pitfall #1 is that something goes "wrong" with the script and for some reason it doesn't exit. Symptom: box crashes as it can't create anymore processes and/or open a file descriptor, etc.

How to solve: make the script grab an exclusive lock to a file. You could write your pid to a file, but that's hacky. If you can't grab the exclusive lock, a previous version is running, so you should just exit.

Here's the PHP interface to flock(): PHP flock()

  • pitfall #2: it really should be a daemon.

If something needs to be "done all the time", maybe it should really be "done all the time". You can use the file locking recipe to make sure your script stays up, or you can use something like monit to start it. But you can also just insure it stays up by using cron, and file locking.

  • Pitfall #3: you convert to a daemon, but there's a memory leak and the thing just keeps expanding up like the girl in Willie Wonka on too many blueberries. Symptom: OOM error, swapping, etc. This is PHP after all.

Solution: exit after 1000 [or some #] iterations, and then use cron and the file locking model to start a new version [or monit or equivalent].

Upvotes: 5

Related Questions