CODEWITHSUNDEEP
rparallel-processing
Robert Kubrick
Robert Kubrick

Reputation: 8723

R: resourcing code in snow workers

I'm using snow parApply() to distribute processing tasks to a number of workes on the local machine. The problem is that if I change the code in one of the functions the workers will not be aware of the changes.

How can I 'resource' the source code files in the workers?

EDIT

I can't call source() on my cluster to re-eval all my functions:

cl = makeSOCKcluster(rep("localhost", 5))
> clusterCall(cl, getwd)
[[1]]
[1] "/home/user"

[[2]]
[1] "/home/user"

[[3]]
[1] "/home/user"

[[4]]
[1] "/home/user"

[[5]]
[1] "/home/user"

> clusterCall(cl, source, 'ets.load.R')
Error in checkForRemoteErrors(lapply(cl, recvResult)) : 
  5 nodes produced errors; first error: cannot open the connection

Upvotes: 0

Views: 545

Answers (1)

Hansi
Hansi

Reputation: 2644

Update it in each worker using parallel::clusterCall

Upvotes: 3

Related Questions