CODEWITHSUNDEEP
pythoncronflock
Hyperboreus
Hyperboreus

Reputation: 32429

Running python script with cron only if not running

I need to run a python script (job.py) every minute. This script must not be started if it is already running. Its execution time can be between 10 seconds and several hours.

So I put into my crontab:

* * * * * root cd /home/lorenzo/cron && python -u job.py 1>> /var/log/job/log 2>> /var/log/job/err

To avoid starting the script when it is already running, I use flock().

This is the script (job.py):

import fcntl
import time
import sys

def doIncrediblyImportantThings ():
    for i in range (100):
        sys.stdout.write ('[%s] %d.\n' % (time.strftime ('%c'), i) )
        time.sleep (1)

if __name__ == '__main__':
    f = open ('lock', 'w')
    try: fcntl.lockf (f, fcntl.LOCK_EX | fcntl.LOCK_NB)
    except:
        sys.stderr.write ('[%s] Script already running.\n' % time.strftime ('%c') )
        sys.exit (-1)
    doIncrediblyImportantThings ()

This approach seems to work.

Is there anything I am missing? Are there any troubles I can run into using this approach?

Are there more advised or "proper" ways of achieving this behaviour?

I thank you for any suggestion.

Upvotes: 19

Views: 7835

Answers (3)

Nate Ferrero
Nate Ferrero

Reputation: 1458

I ran into this exact problem last week, and although I did find some good solutions, I decided to make a very simple and clean python package and uploaded it to PyPI.

Install with: pip install quicklock

Using it is extremely simple:

[nate@Nates-MacBook-Pro-3 ~/live] python
Python 2.7.6 (default, Sep  9 2014, 15:04:36)
[GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.39)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from quicklock import singleton
>>> # Let's create a lock so that only one instance of a script will run
...
>>> singleton('hello world')
>>>
>>> # Let's try to do that again, this should fail
...
>>> singleton('hello world')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/nate/live/gallery/env/lib/python2.7/site-packages/quicklock/quicklock.py", line 47, in singleton
    raise RuntimeError('Resource <{}> is currently locked by <Process {}: "{}">'.format(resource, other_process.pid, other_process.name()))
RuntimeError: Resource <hello world> is currently locked by <Process 24801: "python">
>>>
>>> # But if we quit this process, we release the lock automatically
...
>>> ^D
[nate@Nates-MacBook-Pro-3 ~/live] python
Python 2.7.6 (default, Sep  9 2014, 15:04:36)
[GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.39)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from quicklock import singleton
>>> singleton('hello world')
>>>
>>> # No exception was thrown, we own 'hello world'!

Take a look: https://pypi.python.org/pypi/quicklock

Upvotes: 2

Jean-Paul Calderone
Jean-Paul Calderone

Reputation: 48315

The only suggestion I would make is to make your exception handling a little more specific. You don't want to accidentally delete the fcntl import one day and hide the NameError that results. Always try to catch the most specific exception you want to handle. In this case, I suggest something like:

import errno

try:
    fcntl.lock(...)
except IOError, e:
    if e.errno == errno.EAGAIN:
        sys.stderr.write(...)
        sys.exit(-1)
    raise

This way, any other cause of the lock being unobtainable shows up (probably in your email since you're using cron) and you can decide if it's something for an administrator to look at, another case for the program to handle, or something else.

Upvotes: 15

Mel
Mel

Reputation: 6157

You're in trouble when the machine reboots or freezes with the script running (and thus an active lock). Simple way to counter this is to use the @reboot cron timestamp to run rm /path/to/lock.

Upvotes: 3

Related Questions