CODEWITHSUNDEEP
rforeachparallel-processing
DOSMarter
DOSMarter

Reputation: 1523

Parallel execution and writing results into matrix using foreach package

I wrote the following code for parallel for loop using (foreach) library and the code seems to be working fine (the parallel execution). However, I face a problem when writing to the variable BestLoops as after the execution I always fine the matrix empty so it seems that it doesn't write anything during the parallelization, any help on how to overcome this problem is highly appreciated the following is a simplified version of my script:

library("foreach")
library("doSNOW") 
N=10000
cl=5
BestLoops=matrix(0,N,2) 
registerDoSNOW(makeCluster(cl, type="SOCK"))
foreach(i=1:N) %dopar% {
BestLoops[i,]=c(2,3)
#plus many other codes in this part, but just consider this one line   (before) just for simplicity
}

Upvotes: 0

Views: 825

Answers (1)

Christian Borck
Christian Borck

Reputation: 1862

Using doParallel will allow you to combine the results using rbind:

library(doParallel)
N=100000
nworkers <- detectCores()
cl <- makePSOCKcluster(nworkers)
registerDoParallel(cl)

step <- ceiling(N/nworkers)

BestLoops <- foreach(start_from=seq(1, N, step),
                     .combine='rbind') %dopar% {
                       mat <- matrix(0,step,2)
                       i <- 1
                       for(j in start_from:(start_from+step-1)) {
                         mat[i,]=c(j,3)
                         i <- i +1
                       }
                       mat
                     }

You just perform the code on each of the workers but with each using a smaller number of iterations.

Upvotes: 2

Related Questions