CODEWITHSUNDEEP
rparallel-processingdoparallel
agnivesh
agnivesh

Reputation: 115

R parallel backend: What happens when one process faces an exception?

I am using foreach + %dopar% to achieve parallelism over multiple cores. I know some of the tasks will face exceptions. When an exception occurs:

  1. Will the remaining tasks that were already parallelly started still complete?
  2. Will the tasks that were not scheduled (I don't know if that's the correct term) be scheduled and eventually complete? If so, will it still be able to utilize all the cores?

I tried finding resources on this, but couldn't find any. Looks like I'm using the wrong keywords. If you have any resources, please direct me to them.

Upvotes: 1

Views: 237

Answers (1)

Random Cotija
Random Cotija

Reputation: 673

There is a parameter in foreach called .errorhandling, it could have the values of stop (default), remove or pass. their behaviour is like this:

  1. stop: The function will be stopped.
  2. remove: The result of this specific task will not be returned.
  3. pass: The error object will be included with the results.

So addressing your specific question, if you have many task running in parallel and one of the task in one worker raised an exception, then it stop the process and will pass to the next task "scheduled" (that is because of the default value stop). The other task will continue as normal in parallel.

Please see this answer that explains better how are handled the errors in foreach and %dopar%.

I hope this will clarify a little your problem.

Upvotes: 3

Related Questions