CODEWITHSUNDEEP
rparallel-processing
jtay
jtay

Reputation: 63

R - Parallelization in EasyABC. Error: ... could not find function

I am trying to run the ABC_sequential() function from the package EasyABC in parallel in R. But I am getting the error:

Error in checkForRemoteErrors(val) : 2 nodes produced errors; first error: could not find function "f"

I think this is because ABC_sequential() is ultimately calling parLapplyLB() (https://github.com/cran/EasyABC/blob/master/R/EasyABC-internal.R) and I have to export the functions using clusterExport()? (parSapply not finding objects in global environment)

Because the function calls makeCluster() within it, it seems like I may have to modify the package to add clusterExport(cl, "f")? However, as I am a fairly new, I haven't looked into modifying packages for my needs (and I am suspecting it may be more complicated than adding the one line of code). I am wondering if there is a better/easier workaround to getting my function onto the parallel nodes? Below is a simplified reproducible example based on the parallel example given in the R help for ABC_sequential: 

library(EasyABC)

f <- function(x){
    x = x^2     
}

toy_model_parallel <- function(x){ 
set.seed(x[1])
2 * x[2] + f(2) + rnorm(1,0,0.1)
}

sum_stat_obs <- 6.5
pacc <- .4
toy_prior <- list(c("unif",0,1)) # a uniform prior distribution between 0 and 1

# this line of code gives the checkForRemoteErrors(val) error
ABC_Lenormand <- ABC_sequential(method="Lenormand", model=toy_model_parallel, prior=toy_prior, nb_simul=20, summary_stat_target=sum_stat_obs, p_acc_min=pacc, use_seed=TRUE, n_cluster=2)
}

Any advice is greatly appreciated.

Upvotes: 4

Views: 734

Answers (1)

Steve Weston
Steve Weston

Reputation: 19677

You could define any necessary auxiliary functions inside the model function. In this case:

toy_model_parallel <- function(x){ 
    f <- function(x){
        x = x^2
    }
    set.seed(x[1])
    2 * x[2] + f(2) + rnorm(1,0,0.1)
}

It looks like you need to do any worker initialization at the beginning of this function. So if your function needs to call functions from another package, you'd also need to load that package at the beginning of the model function.

I suggest that you send an email to the package developers to see if they have a better solution to this problem. If they don't, you might request that they add support for a user specified cluster object.

Upvotes: 0

Related Questions