CODEWITHSUNDEEP
rwindowsparallel-processingenvironment-variables
Nucore
Nucore

Reputation: 69

How to clusterExport in a foreach loop (OS:Windows)

I have the following problem. I need the timeindex of variables in my Global Environment, but when I want to export them into my cluster during the parallel processing from my Global Environment, I'm getting the following message:

Error in { : task 1 failed - "object 'Szenario' not found"

A minimal example of my original code, which produces the error:

    Historical <- structure(c(18.5501872568473, 24.3295787432955, 14.9342384460288, 
    13.0653757599636, 8.67294618896797, 13.4587662721594, 20.792126254714, 
    17.5162747884424, 28.8253151239752, 23.0568765432192), index = structure(c(-7305, 
    -7304, -7303, -7302, -7301, -7300, -7299, -7298, -7297, -7296
    ), class = "Date"), class = "zoo")

    Szenario <- structure(c(10.2800684124652, 14.5495871489361, 9.8565852930294, 
    21.1654540678686, 21.1936990312861, 12.4209005842752, 9.77473132000267, 
    17.1997402736739, 17.884107611858, 13.622588360244), index = structure(c(13149, 
    13150, 13151, 13152, 13153, 13154, 13155, 13156, 13157, 13158
    ), class = "Date"), class = "zoo")

    library(doParallel)   
    library(foreach)     
    library(raster)
    library(zoo)
    library(parallel)

    # Parallelisation Settings
    # Definition of how many cores you want to use
    UseCores <- detectCores() -2 # -1 at max because one core has to be used for other tasks 
    # Register CoreCluster
    cl <- makeCluster(UseCores)
    registerDoParallel(cl)

foreach(fn=1:1) %dopar% {

  library(raster) # needed libraries have to be loaded inside the loop, while parallel processing occurs
  library(zoo)
  library(base)
  library(parallel)

  #In my original script, I'm looping through Filenames, which are called like my variables in my Global environment (without .tif at the end of the filename), variables names are saved as characters
  file.referenz.name <- c("Historical")
  file.szenario.name <- c("Szenario")

  #Create timeindex für rasterstacks to subset later on with them (getZ, setZ)

  clusterExport(cl, varlist = c(file.szenario.name, file.referenz.name), envir = .GlobalEnv)
  time.index.szenario <- index(get(file.szenario.name))
  time.index.referenz <- index(get(file.referenz.name))

}

#end cluster
stopCluster(cl)

Upvotes: 0

Views: 917

Answers (1)

CPak
CPak

Reputation: 13581

Try this

foreach(fn=1:1, .export=c("Szenario"), 
                .packages=c("raster", "zoo", "base", "parallel") %dopar%

And it's confusing to clusterExport inside a %dopar% {}. You can either clusterExport to each cl before foreach, or simply use .export in foreach

You can remove library statements in the %dopar% {}.

Upvotes: 1

Related Questions