CODEWITHSUNDEEP
rwindowsfor-loopparallel-processingiteration
Sinusx
Sinusx

Reputation: 111

Output of parallel computing in R on Windows

I would like to do some parallel computing in R on Windows. 

library(doSNOW)
library(foreach)
cl<-makeCluster(4)
registerDoSNOW(cl)

nn = seq(2,20, by=2)
nn2 = rep(-1,10)
vector.accessed.within.iteration = rep(0, 10)

y = foreach(i=1:10) %dopar% {

   print(paste0('i = ', i))
   nn2[i] = (nn[i])^2
   vector.accessed.within.iteration[i] = nn[i] + nn2[i]
   nn2[i]
} 

stopCluster(cl)

When running the code, y is returned as the list of squares from 2 to 20, as intended. However, neither vector nn2 nor vector.accessed.within.iteration are modified within foreach loop. Furthermore, the printing print(paste0('i = ', i)) does not occur.

Is it possible to run iterations of a loop in parallel on windows while also displaying messages and modifying variables from global environment?

Upvotes: 0

Views: 578

Answers (1)

HenrikB
HenrikB

Reputation: 6805

Is it possible to run iterations of a loop in parallel on windows while ... modifying variables from global environment?

No(*). All values set inside a foreach() %dopar% { ... } construct must be returned at the end of each iteration. This is by design and philosophy of the foreach package. See 'Example 2: A slightly complicated for-loop' in blog post 'Parallelize a For-Loop by Rewriting it as an Lapply Call' from 2019-01-11 (disclaimer: I'm the author) for how to return multiple values.

(*) There exist various workarounds that communicate using other channels, e.g. the file system. However, such solutions should only be used if all other alternatives have been exhausted and failed. I highly recommend such alternative solutions because you will run into other types of problems with those approached. In your case, you definitely don't need it.

Is it possible to run iterations of a loop in parallel on windows while also displaying messages ...?

Yes, if you use the doFuture foreach adaptor (disclaimer: I'm the author). When you use that, you will get access to all of the future framework, which includes relaying of output, messages, warnings, and other condition types. You can read about this in blog post 'future 1.9.0 - Output from The Future' from 2018-07-23 (disclaimer: I'm the author).

> library(doFuture)
> registerDoFuture()  ## tell foreach to use the future framework

> ## Create a local cluster of two local workers
> plan(multisession, workers = 2L)

> library(foreach)
> y <- foreach(x = 1:3) %dopar% {
    cat(paste0("x = ", x, "\n"))
    sqrt(x)
  }
x = 1
x = 2
x = 3
>

Upvotes: 2

Related Questions