CODEWITHSUNDEEP
rparallel-processingdata.table
navo
navo

Reputation: 211

Cannot load library data.table to R parallel

R throws an error when I use the below code to load library data.table to the cpu cluster. But data.table package is installed on R and it's working fine when used outside parallel code.

no_cores <- detectCores() - 1
cl <- makeCluster(no_cores,outfile="out.txt")
clusterEvalQ(cl, library(data.table))

Error :-

clusterEvalQ(cl, library(data.table)) Error in checkForRemoteErrors(lapply(cl, recvResult)) : 3 nodes produced errors; first error: there is no package called 'data.table'

Upvotes: 2

Views: 2252

Answers (1)

user5099519
user5099519

Reputation:

Building on what HenrikB said above in the comments, I got rid of this problem by adding my .libPaths() call to clusterEvalQ():

.libPaths("C:/programs/rlib")
library(parallel)
no_cores<-detectCores()-1

cl<-makeCluster(no_cores)
#this is needed to see the package
clusterEvalQ(cl, .libPaths("C:/programs/rlib"))

# I'm using a function that uses the stringdist library
clusterEvalQ(cl, library(stringdist))

#You need to load your data into the cluster also
clusterExport(cl, "unmatched")
clusterExport(cl, "matched")

#now we're going to run it, amatch is a function in the stringdist lib

parLapply(cl, unmatched,function(x) amatch(x,matched, maxDist = Inf))

Upvotes: 3

Related Questions