CODEWITHSUNDEEP
pythonlinux
Lostsoul
Lostsoul

Reputation: 25999

How can I monitor a Python process and relaunch if it dies?

I have a program that I'm running that listens to a queue (it's not multi-threaded so I want to run several instances of it). I've tried my best to catch errors but in the event the app crashes due to a error or bad incoming data, I want to be able to respawn the Python app (after I log the stacktrace) so it continues to work.

I felt this might be a common problem for people who run python based services so I thought I would ask but I was thinking of writing some code to do a ps -ef and count the instances of the name of the Python program (if less than a threshold then I would have the program relaunch it).

Before I build this, I wanted to know if there was perhaps a better way or a existing tool/module that did this?

Upvotes: 1

Views: 2147

Answers (3)

jaime
jaime

Reputation: 2334

Checkout supervisord. I use it regularly to launch, monitor all types of things.

Here is how I set it up to launch a wsgi app on my server:

[program:quizzes]
directory = /var/www/quizzes.seasources.net
command = /home/jaime/code/virtualenv/quizzes/bin/uwsgi uwsgi.ini
process_name = quizzes
autostart = true
startsecs = 5
user = www-data
redirect_stderr = true
stdout_logfile = /var/www/quizzes.seasources.net/logs/supervisor-console.log
environment = PYTHON_EGG_CACHE=/tmp/python-eggs

The configuration file format is easy to understand and it even logs stdout/stderr to a file. Above it's /var/www/quizzes.seasources.net/logs/supervisor-console.log You can read more about configuration here.

Upvotes: 2

joaoricardo000
joaoricardo000

Reputation: 4959

If you are looking for something more simple, you can use the subprocess module (python default) to start and check your processes... A basic version would look like this:

# run.py

import subprocess, time

# add your listener processor call here
_PROCESS_ARGS = ['python','/path/to/listener.py']
_PROCESS_TOTAL = 10

process_list = []

# start the processes...
for i in range(_PROCESS_TOTAL):     
    process_list.append(subprocess.Popen(_PROCESS_ARGS))

while True:     
    for i in range(_PROCESS_TOTAL):         
        p = process_list[i]         
            if p.poll() != None: # check if process is running                      
                process_list[i] = subprocess.Popen(_PROCESS_ARGS) # if not, replace with new one
    time.sleep(1) # check only every second...

Upvotes: 1

Webthusiast
Webthusiast

Reputation: 1026

You could use a supervisor. A well known one that's written in Python would be supervisord, a more recent one also in Python would be Circus, and then there are Monit or daemontools and probably many more.

Upvotes: 2

Related Questions