CODEWITHSUNDEEP
pythonerror-handlingmultiprocessing
Davoud Taghawi-Nejad
Davoud Taghawi-Nejad

Reputation: 16826

Exiting a multiprocessing program on error

I have an application that creates one multiprocessing process for communication and n for workers. I would need to find a way to exit the whole application when there is an error somewhere or the user presses control-C.

// the agents are connected via zmq

Upvotes: 0

Views: 42

Answers (1)

Binnut
Binnut

Reputation: 90

If the error occurs in the communication process, you could use your existing socket to send a special task that asks the receiver to close. This would be received by one worker, that notifies the communication process (through some socket) that it is exiting, and then exits.

The communication process could basically fill the pipeline with n sockets if no queuing at workers are done, or k*n + one in a while if queuing is done. This is done until all workers are done.

An error at a worker could be sent to the communication process through the earlier mentioned channel that an error has occurred. The channel could for example be a push/pull socket, or maybe a pub/sub with a proxy. (I guess you are using a push/pull to get the results back. In that case use that.)

Upvotes: 1

Related Questions