CODEWITHSUNDEEP
rparallel-processing
lolo
lolo

Reputation: 646

parLapply in R - dataframes

I have a problem while I try to recover the data frame already ejecuted by the parLapply algorithm in R.

I attached the next example with the iris data set to show the issue. I created a function which does a liner model to each iris Species and then aply a loop over them.

uniques<-unique(iris$Species)

model<-function(i){
    table<-iris[iris$Species==uniques[i],]
    fit<-lm(Petal.Width ~ Petal.Length + Sepal.Width + Sepal.Length, data=table)
    predicted_df <- data.frame(pred = predict(fit, table), table)
    assign(paste0("predicted_df_",i),predicted_df,envir = .GlobalEnv)
}

#Loop over Species
loop<-  for (i in 1:3){
  model(i)
}

Here, the tree data bases ("predicted_df_1/2/3") correctly appeared in the local environment.

When I execute the same thing but with the parLapply algorithm I can not find where are the data frames or how to bring them to the local environment. No error is display.

library("foreach")
library("doParallel")

cl <- makeCluster(mc <- getOption("cl.cores", 4))
clusterExport(cl=cl, varlist=c("iris"))
clusterEvalQ(cl, library(DAAG))

registerDoParallel(cl) # register the cluster
system.time(
  df <- parLapply(cl, 1:3, 
                    function(i) {
                      tryCatch({  model(i)}, error=function(e){cat("ERROR :",conditionMessage(e), "\n")})
                    })
)
stopCluster(cl)

How someone helps me with this task? Thanks!

Upvotes: 0

Views: 1774

Answers (1)

F. Priv&#233;
F. Priv&#233;

Reputation: 11728

Simply use 

model<-function(i){
    table<-iris[iris$Species==uniques[i],]
    fit<-lm(Petal.Width ~ Petal.Length + Sepal.Width + Sepal.Length, data=table)
    data.frame(pred = predict(fit, table), table)
}

then you will get a list of size 3 from parLapply with the predictions you want.

You can't assign to the global environment when you're using parallelism because this is not your main R session that does the work.

Upvotes: 1

Related Questions