CODEWITHSUNDEEP
pythonmultiprocessing
alonisser
alonisser

Reputation: 12068

Python multiprocessing - watch a process and restart it when fails

Consider this working code:

from multiprocessing import Process
from updaters import app1, app2

if __name__ == '__main__':

    apps = [ app1, app2]  
    for app in apps:
        instance = app()
        p = Process(target=instance.start_listener)
        p.start()
        p.join()

This works fine until one process fails (lets say some exception) - how Can I programmatically monitor the process and restart it if it fails

Upvotes: 16

Views: 17706

Answers (1)

Steve Barnes
Steve Barnes

Reputation: 28370

Poll the Process.is_alive() and if it returns False delete the process and start a new one, e.g.:

from multiprocessing import Process
from updaters import app1, app2
from time import sleep

if __name__ == '__main__':

    apps = [app1, app2]  
    processes = {}
    n = 0
    for app in apps:
        instance = app()
        p = Process(target=instance.start_listener)
        p.start()
        processes[n] = (p, app) # Keep the process and the app to monitor or restart
        n += 1

    while len(processes) > 0:
        for n in processes.keys():
            (p, a) = processes[n]
            sleep(0.5)
            if p.exitcode is None and not p.is_alive(): # Not finished and not running
                 # Do your error handling and restarting here assigning the new process to processes[n]
                 print(a, 'is gone as if never born!')
            elif p.exitcode < 0:
                print ('Process Ended with an error or a terminate', a)
                # Handle this either by restarting or delete the entry so it is removed from list as for else
            else:
                print (a, 'finished')
                p.join() # Allow tidyup
                del processes[n] # Removed finished items from the dictionary 
                # When none are left then loop will end
print ('FINISHED')

Upvotes: 21

Related Questions