CODEWITHSUNDEEP
roptimizationparallel-processingparallel.foreach
Maryam
Maryam

Reputation: 97

How to export global variable with modified values when using parallelization in R with parSapply()

I am using parSapply from the "parallel" package in R to optimize a computationally expensive code. The code works well and the final answer is the same as the serial version of my code. The only problem is that I have a global variable whose values are changing during the code, but parSapply function can not apply the changes on the global variables. Here is a simple example:

M = matrix(1,5,5)
test = function(x){
M[x,x] <<- x
}

sapply(c(1:5), function(x) test(x))
M
no_of_cores = detectCores()
cl = makeCluster(no_of_cores)
clusterExport(cl, varlist = c("test","M"), envir=environment())
parSapply(cl, c(1:5), function(x) test(x))
M

when I use sapply function, the values of M are modified correctly, but when I use parSapply, M does not change! I appreciate any suggestions to fix this issue! Thanks.

Upvotes: 1

Views: 229

Answers (1)

akrun
akrun

Reputation: 887108

We may use foreach

library(doSNOW)
library(parallel)
no_of_cores = detectCores()
cl <- makeSOCKcluster(no_of_cores)
registerDoSNOW(cl)
M <- matrix(1,5,5)

comb <- function(m, ...) {
  for (r in list(...)) {
    m[r$k, r$k] <- r$k
  }
  m
}

out <- foreach(k=1:5, .combine='comb', .init=M,
          .multicombine=TRUE) %dopar% {
    m <- M  
     list(k=k, m=m)
  }

-output

out
     [,1] [,2] [,3] [,4] [,5]
[1,]    1    1    1    1    1
[2,]    1    2    1    1    1
[3,]    1    1    3    1    1
[4,]    1    1    1    4    1
[5,]    1    1    1    1    5

Upvotes: 1

Related Questions