CODEWITHSUNDEEP
rforeachrasterdoparallel
lavp_theOG
lavp_theOG

Reputation: 81

Error in makePSOCKcluster(names = spec, ...) : Cluster setup failed. 3 of 3 workers failed to connect

I am trying to recruit more cores to increases my processing time for some lidar data I am analyzing but I keep getting "Error in makePSOCKcluster(names = spec, ...) : Cluster setup failed. 3 of 3 workers failed to connect." after I run this:

 UseCores <-detectCores() -1
cl <- makeCluster(UseCores)
registerDoParallel(cl) 
foreach(i=1:lengthcanopy_list)) %dopar% {
  library(raster)
  ttops <- vwf(CHM = canopy_test, winFun = lin, minHeight = 2, maxWinDiameter = NULL)
}

Why am I getting this error and what can I do to fix it?

Upvotes: 4

Views: 6412

Answers (2)

Dan Hicks
Dan Hicks

Reputation: 294

Assuming you're working in RStudio, the problem was most likely this bug: changes in parallel created a conflict between R 4.0 and RStudio 1.3.something. Your code isn't a minimal working example so I can't check it on my end, but I just confirmed that cl = makeCluster(1) behaves as expected in RStudio 1.4.1103, R 4.0.2, Mac OS 10.15.7. So try updating RStudio and checking your code again.

Upvotes: 4

Garini
Garini

Reputation: 1204

It seems a problem relative to recent versions of R. Until further updates, looking at this issue on GitHub it seems there are two workarounds as follows.

Directly use this to create the cluster:

cl <- parallel::makeCluster(2, setup_strategy = "sequential")

Or for a long term solution add the following to your ~/.Rprofile

## WORKAROUND: https://github.com/rstudio/rstudio/issues/6692
## Revert to 'sequential' setup of PSOCK cluster in RStudio Console on macOS and R 4.0.0
if (Sys.getenv("RSTUDIO") == "1" && !nzchar(Sys.getenv("RSTUDIO_TERM")) && 
    Sys.info()["sysname"] == "Darwin" && getRversion() >= "4.0.0") {
  parallel:::setDefaultClusterOptions(setup_strategy = "sequential")
}

Even if this workaround was necessary for Rstudio users, it could be of general use, as it is useful also on my GitLab registered runner tests.

Upvotes: 6

Related Questions