CODEWITHSUNDEEP
rfor-loopparallel-processingnested-loops
bdeonovic
bdeonovic

Reputation: 4220

Parallelizing a double for loop in R

I've been using the parallel package in R to do loops like:

cl <- makeCluster(getOption("cl.cores", 6))
result <- parSapply(cl,1:k,function(i){ ... })

Is there a natural way to parallelize a nested for loop in R using this package? Or perhaps another package? I know there are several ways to implement parallelism in R.

My loop looks something like this. I simplified a bit but it gets the message across:

sup_mse <- matrix(0,nrow=k,ncol=length(sigma))
k <- 100000 #Number of iterations
sigma <- seq(from=0.1,to=10,by=0.2)

for(i in 1:k){
  for(j in 1:length(sigma)){
    sup<-supsmu(x,y)
    sup_mse[i,j] <- mean((m(x)-sup$y)^2)
  }
}

Upvotes: 2

Views: 516

Answers (1)

James Tobin
James Tobin

Reputation: 3110

Thanks for making the reproducible example! I prefer snowfall for my parallel processing, so here's how it looks in there.

install.packages('snowfall')
require(snowfall)

### wasn't sure what you were using for x or y
set.seed(1001)
x <- sample(seq(1,100),20)
y <- sample(seq(1,100),20)
k <- 100
sigma <- seq(0.1, 10, 0.2)

### makes a local cluster on 4 cores and puts the data each core will need onto each
sfInit(parallel=TRUE,cpus=4, type="SOCK",socketHosts=rep("localhost",4))
sfExport('x','y','k','sigma')

answers <- sfSapply(seq(1,k), function(M) 
  sapply(seq(1,length(sigma)), function(N)
    mean((mean(x)-supsmu(x,y)$y)^2)  ## wasn't sure what you mean by m(x) so guessed mean
  )
)

sup_mse <- t(answers) ## will give you a matrix with length(sigma) columns and k rows
sfStop()

I remember reading somewhere that you only want to use sfSapply in the outer loops and then use your regular apply functions inside of that loop. Hope this helps!

Upvotes: 1

Related Questions