CODEWITHSUNDEEP
rforeachraster
jim
jim

Reputation: 647

Use a raster Stack in foreach loop with doParallel in R

I want to perform a series of calculations on each layer in a large raster stack in R, and save the results for each layer as a separate raster for later use. I am trying to speed up the process using foreach and doParallel. Here is the basic code, with a trivial calculation (*3) for each layer in the stack:

library(raster)
library(doParallel)

RasterStack<- stack(system.file("external/rlogo.grd", package="raster")) 
RasterStack<-addLayer(RasterStack,RasterStack)

cl <- makeCluster(2)
registerDoParallel(cl)

RasterStack<- stack(system.file("external/rlogo.grd", package="raster")) 
RasterStack<-addLayer(RasterStack,RasterStack)

foreach(rasname=iter(names(RasterStack)),packages="raster") %dopar%{
  ras<-RasterStack[[rasname]]*3
  save(ras,file=paste0(rasname,"_new.Rras"))
}

It mostly works, but the first N layers are not processed, where N is the number of nodes (e.g. 2 in above). I get the following error: Error in { : task 1 failed - "this S4 class is not subsettable". The error mostly only happens after all other layers have been processed correctly (i.e. its only the first N layers that don't work, and this mostly doesn't stop the other layers being processed -though occasionally it does). All layers get processed correctly if run sequentially with %do%.

Can anyone explain this behavior, or offer a solution?
Thanks

Upvotes: 2

Views: 2244

Answers (2)

user16482
user16482

Reputation: 1

If you want your results put into a new stack instead of separate files:

# libraries
library(raster)
library(doParallel)

# fake data
RasterStack1 <- stack(system.file("external/rlogo.grd", package="raster")) 
RasterStack1 <- addLayer(RasterStack1, RasterStack1)
plot(RasterStack1)

# make cluster
cl <- makeCluster(2)
registerDoParallel(cl)

# parallel loop
RasterStack2 <- foreach(rasname=iter(names(RasterStack1)), 
                    .packages="raster") %dopar% {
RasterStack1[[rasname]]*3
}

# make result a stack
RasterStack2 <- stack(RasterStack2)
plot(RasterStack2)

# stop cluster
stopCluster(cl)

Upvotes: 0

thie1e
thie1e

Reputation: 3688

It seems that there is just the . in front of packages missing. This works for me:

foreach(rasname = iter(names(RasterStack)), .packages = "raster") %dopar% {
  ras <- RasterStack[[rasname]] * 3
  save(ras, file = paste0(rasname, "_new.Rras"))
}

Upvotes: 4

Related Questions